NumPy library - 6 (Array Manipulation)

Array manipulation as the name suggests deals with creation of an array using already created arrays usually by joining or splitting of already defined arrays. Let's see how array manipulation is done using:

1. Joining Arrays

A new array can be formed by merging multiple arrays, this new array then contains all of the arrays. This is accomplished in NumPy by using stacking which provides methods to merge arrays.

a. vstack() function

In order to perform vertical stacking we use this function which combines the second array as new rows of the first array. In this case, the array grows in a vertical direction hence it is known as vertical stacking. The following program shows vertical stacking:

import numpy as np

arr1 = np.random.random(9).reshape(3,3)
arr2 = np.zeros((3, 3))
print('The first array is \n')
print(arr1)
print('\nThe second array is\n')
print(arr2)
print('\nThe  vertical stack array is\n')
print(np.vstack((arr1, arr2)))

The output of the program is shown below:

The first array is

[[0.14500418 0.61535875 0.92246045]
 [0.79186282 0.22879715 0.30451435]
 [0.30571872 0.60974898 0.93925356]]

The second array is

[[0. 0. 0.]
 [0. 0. 0.]
 [0. 0. 0.]]

The  vertical stack array is

[[0.14500418 0.61535875 0.92246045]
 [0.79186282 0.22879715 0.30451435]
 [0.30571872 0.60974898 0.93925356]
 [0.         0.         0.        ]
 [0.         0.         0.        ]
 [0.         0.         0.        ]]

------------------
(program exited with code: 0)

Press any key to continue . . .

b. hstack() function

In order to perform horizontal stacking we use this function which combines the second array as new columns of the first array. In this case, the array grows in a horizontal direction hence it is known as horizontal stacking. The following program shows horizontal stacking:

 import numpy as np

arr1 = np.random.random(9).reshape(3,3)
arr2 = np.zeros((3, 3))
print('The first array is \n')
print(arr1)
print('\nThe second array is\n')
print(arr2)
print('\nThe  horizontal stack array is\n')
print(np.hstack((arr1, arr2)))

The output of the program is shown below:

The first array is

[[0.34118011 0.46964606 0.92296117]
 [0.3612867  0.49396367 0.93493638]
 [0.6506944  0.02496858 0.88195055]]

The second array is

[[0. 0. 0.]
 [0. 0. 0.]
 [0. 0. 0.]]

The  horizontal stack array is

[[0.34118011 0.46964606 0.92296117 0.         0.         0.        ]
 [0.3612867  0.49396367 0.93493638 0.         0.         0.        ]
 [0.6506944  0.02496858 0.88195055 0.         0.         0.        ]]

------------------
(program exited with code: 0)

Press any key to continue . . . 

c. column_stack()

This function is used with one-dimensional arrays, which are stacked as columns in order to form a new two-dimensional array. The following program shows column stacking:

import numpy as np

arr1 = np.arange(3)
arr2 = np.arange(3)
arr3 = np.arange(3)
print('The first array is \n')
print(arr1)
print('\nThe second array is\n')
print(arr2)
print('\nThe third array is\n')
print(arr3)
print('\nThe  column stack array is\n')
print(np.column_stack((arr1,arr2,arr3)))

The output of the program is shown below:

The first array is

[0 1 2]

The second array is

[0 1 2]

The third array is

[0 1 2]

The  column stack array is

[[0 0 0]
 [1 1 1]
 [2 2 2]]

------------------
(program exited with code: 0)

Press any key to continue . . .

d. column_stack()

This function is used with one-dimensional arrays, which are stacked as columns in order to form a new two-dimensional array. The following program shows column stacking: 

import numpy as np

arr1 = np.arange(3)
arr2 = np.arange(3)
arr3 = np.arange(3)
print('The first array is \n')
print(arr1)
print('\nThe second array is\n')
print(arr2)
print('\nThe third array is\n')
print(arr3)
print('\nThe  row stack array is\n')
print(np.row_stack((arr1,arr2,arr3))) 

The output of the program is shown below:

The first array is

[0 1 2]

The second array is

[0 1 2]

The third array is

[0 1 2]

The  row stack array is

[[0 1 2]
 [0 1 2]
 [0 1 2]]
------------------
(program exited with code: 0)

Press any key to continue . . . 

2. Splitting Arrays

A new array can be formed by dividingan array, this new array into several parts. This is accomplished in NumPy by using splitting which provides functions to split arrays.

a.  hsplit() function

In order to split the array horizontally, i.e the width of the array is divided into two parts this function is used. The following program shows horizontal splitting:

import numpy as np

arr1 = np.arange(16).reshape((4, 4))

print('The original array is \n')
print(arr1)

[arr2,arr3] = np.hsplit(arr1,2)

print('\nThe  horizontally split arrays are\n')
print('The first array  \n')
print(arr2)
print('\nThe second array  \n')
print(arr3)

The output of the program is shown below which  shows that the 4x4 matrix arr1 will be split into two 2x4 matrices:

The original array is

[[ 0  1  2  3]
 [ 4  5  6  7]
 [ 8  9 10 11]
 [12 13 14 15]]

The  horizontally split arrays are

The first array

[[ 0  1]
 [ 4  5]
 [ 8  9]
 [12 13]]

The second array

[[ 2  3]
 [ 6  7]
 [10 11]
 [14 15]]
------------------
(program exited with code: 0)

Press any key to continue . . .

b. vsplit() function

In order to split the array vertically, i.e the height of the array is divided into two parts this function is used. The following program shows vertical splitting:

import numpy as np

arr1 = np.arange(16).reshape((4, 4))

print('The original array is \n')
print(arr1)

[arr2,arr3] = np.vsplit(arr1,2)

print('\nThe  vertically split arrays are\n')
print('The first array \n')
print(arr2)
print('\nThe second array \n')
print(arr3) 

The output of the program is shown below which  shows that the 4x4 matrix arr1 will be split into two 2x4 matrices:

The original array is

[[ 0  1  2  3]
 [ 4  5  6  7]
 [ 8  9 10 11]
 [12 13 14 15]]

The  vertically split arrays are

The first array

[[0 1 2 3]
 [4 5 6 7]]

The second array

[[ 8  9 10 11]
 [12 13 14 15]]

------------------
(program exited with code: 0)

Press any key to continue . . . 

c. split() function

Using this function we can split the array into non-symmetrical parts. We pass the array as an argument and also specify the indexes of the parts to be divided. An additional argument axis is also passed, if the the option axis = 1 is used then the indexes will be columns; if instead the option is axis = 0, then they will be row indexes.

The following program divides the matrix into three parts with axis =1, the first of which will include the first column, the second will include the second and the third column, and the third will include the last column:

import numpy as np

arr1 = np.arange(16).reshape((4, 4))

print('The original array is \n')
print(arr1)

[arr2,arr3,arr4] = np.split(arr1,[1,3],axis=1)

print('\nThe splitted arrays are\n')
print('The first array \n')
print(arr2)
print('\nThe second array \n')
print(arr3)
print('\nThe second array \n')
print(arr4)

The output of the program is shown below:

The original array is

[[ 0  1  2  3]
 [ 4  5  6  7]
 [ 8  9 10 11]
 [12 13 14 15]]

The splitted arrays are

The first array

[[ 0]
 [ 4]
 [ 8]
 [12]]

The second array

[[ 1  2]
 [ 5  6]
 [ 9 10]
 [13 14]]

The second array

[[ 3]
 [ 7]
 [11]
 [15]]

------------------
(program exited with code: 0)

Press any key to continue . . .

The following program divides the matrix into three parts with axis =0, the first of which will include the first row , the second will include the second and the third row, and the third will include the last row:

import numpy as np

arr1 = np.arange(16).reshape((4, 4))

print('The original array is \n')
print(arr1)

[arr2,arr3,arr4] = np.split(arr1,[1,3],axis=0)

print('\nThe splitted arrays are\n')
print('The first array \n')
print(arr2)
print('\nThe second array \n')
print(arr3)
print('\nThe second array \n')
print(arr4)

The output of the program is shown below:

The original array is

[[ 0  1  2  3]
 [ 4  5  6  7]
 [ 8  9 10 11]
 [12 13 14 15]]

The splitted arrays are

The first array

[[0 1 2 3]]

The second array

[[ 4  5  6  7]
 [ 8  9 10 11]]

The second array

[[12 13 14 15]]

------------------
(program exited with code: 0)

Press any key to continue . . . 

We can also use the hsplit() and vsplit() functions to achieve this kind of splitting. The following programs implement them:

1. hsplit()

import numpy as np

arr1 = np.arange(16).reshape((4, 4))

print('The original array is \n')
print(arr1)

[arr2,arr3,arr4] = np.hsplit(arr1,[1,3])

print('\nThe splitted arrays are\n')
print('The first array \n')
print(arr2)
print('\nThe second array \n')
print(arr3)
print('\nThe second array \n')
print(arr4)

The output of the program is shown below:

The original array is

[[ 0  1  2  3]
 [ 4  5  6  7]
 [ 8  9 10 11]
 [12 13 14 15]]

The splitted arrays are

The first array

[[ 0]
 [ 4]
 [ 8]
 [12]]

The second array

[[ 1  2]
 [ 5  6]
 [ 9 10]
 [13 14]]

The second array

[[ 3]
 [ 7]
 [11]
 [15]]


------------------
(program exited with code: 0)

Press any key to continue . . .

2. vsplit()

import numpy as np

arr1 = np.arange(16).reshape((4, 4))

print('The original array is \n')
print(arr1)

[arr2,arr3,arr4] = np.vsplit(arr1,[1,3])

print('\nThe splitted arrays are\n')
print('The first array \n')
print(arr2)
print('\nThe second array \n')
print(arr3)
print('\nThe second array \n')
print(arr4)

The output of the program is shown below:

The original array is

[[ 0  1  2  3]
 [ 4  5  6  7]
 [ 8  9 10 11]
 [12 13 14 15]]

The splitted arrays are

The first array

[[0 1 2 3]]

The second array

[[ 4  5  6  7]
 [ 8  9 10 11]]

The second array

[[12 13 14 15]]


------------------
(program exited with code: 0)

Press any key to continue . . .


3.  Copying arrays

In order to generate a complete and distinct array using an existing array, use the copy() function as shown in the following program:

import numpy as np

arr1 = np.arange(3)

print('The original array is \n')
print(arr1)

arr2 = arr1.copy()

print('\nThe copied array is\n')
print(arr2)
arr1[0] = 7
print('\nThe first array \n')
print(arr1)
print('\nThe second array \n')
print(arr2)

The output of the program is shown below:

The original array is

[0 1 2]

The copied array is

[0 1 2]

The first array

[7 1 2]

The second array

[0 1 2]
------------------
(program exited with code: 0)

Press any key to continue . . .

In case if we assign arr1 to arr2 (arr2 = arr1) then we are not copying it and changing any element in arr1 will also change the element's value in arr2. Same holds good when we slice an array. The following program proves this:

import numpy as np

arr1 = np.arange(5)

print('The original array is \n')
print(arr1)

arr2 = arr1

print('\nThe copied array is\n')
print(arr2)

arr1[0] = 7
print('\nThe first array \n')
print(arr1)
print('\nThe second array \n')
print(arr2)

arr3 = arr1[0:1]

print('\nThe slice \n')
print(arr3)

arr1[0] = 8
print('\nThe modified first array \n')
print(arr1)

print('\nThe modified slice \n')
print(arr3)

The output of the program is shown below:

The original array is

[0 1 2 3 4]

The copied array is

[0 1 2 3 4]

The first array

[7 1 2 3 4]

The second array

[7 1 2 3 4]

The slice

[7]

The modified first array

[8 1 2 3 4]

The modified slice

[8]


------------------
(program exited with code: 0)

Press any key to continue . . .

Thus we can see that none of the NumPy assignments produces copies of arrays, nor any element contained in them whereas the copy() creates a distinct array.

Here I am ending today's post. In the next post we shall further explore NumPy library and discuss about Vectorization and broadcasting. Till we meet next keep practicing and learning Python as Python is easy to learn!

Read More

NumPy library - 5 (Indexing, Slicing, and Iterating) continued..

Element extraction using the Conditional and Boolean operators

Apart from using the numerical indexes we can use the conditions and Boolean operators to selectively extract the elements in an array. In the following example we'll select all values that are less than 0.5 in a matrix containing random numbers between 0 and 1. See the program below:

import numpy as np

arr1 = np.random.random((3, 3))
print(arr1)
print('\n')
print(arr1<0.5)
print('\n')
print(arr1[arr1<0.5])

The output of the program is:

[[0.08995485 0.45831529 0.59991243]
 [0.99256889 0.31161011 0.24247865]
 [0.8438997  0.20548216 0.15781144]]


[[ True  True False]
 [False  True  True]
 [False  True  True]]


[0.08995485 0.45831529 0.31161011 0.24247865 0.20548216 0.15781144]

------------------
(program exited with code: 0)

Press any key to continue . . . 

In the program we first defined a matrix of random numbers and printed the matrix. Next we applied an operator condition and received as a return value, a Boolean array containing true values in the positions in which the condition arr1<0.5  is satisfied. But we are interested in the extraction of elements smaller than 0.5 and the Boolean arrays are used implicitly for making selections of parts of
arrays, thus we inserted our condition directly inside the square brackets and extract all elements smaller than 0.5, so as to obtain a new array as shown in the third output.

Changing shapes of array

1. reshape()

Using this function it is possible to convert a one-dimensional array into a matrix. We have used this function in previous programs and the following program also shows how to use it:

import numpy as np

arr1 = np.random.random(9)
print(arr1)
print('\n')

print(arr1.reshape(3,3))

The output of the program is:

[0.29559686 0.78175579 0.68994419 0.15964917 0.60401985 0.50051105
 0.5012908  0.13863332 0.72451119]


[[0.29559686 0.78175579 0.68994419]
 [0.15964917 0.60401985 0.50051105]
 [0.5012908  0.13863332 0.72451119]]

------------------
(program exited with code: 0)

Press any key to continue . . . 

The reshape() function returns a new array and can thus create new objects.

2. Modifying the shape attribute

It is also possible to modify the object by modifying it's shape attribute. This is done by assigning a
tuple containing the new dimensions directly to its shape attribute as show in the following program:

import numpy as np

arr1 = np.random.random(12)
print(arr1)
print('\n')
arr1.shape = (3,4)
print(arr1)


The output of the program is:

[0.00256283 0.22191957 0.05906521 0.97575381 0.34349577 0.8302148
 0.263992   0.70762576 0.89476555 0.95956978 0.5096924  0.01297329]


[[0.00256283 0.22191957 0.05906521 0.97575381]
 [0.34349577 0.8302148  0.263992   0.70762576]
 [0.89476555 0.95956978 0.5096924  0.01297329]]

------------------
(program exited with code: 0)

Press any key to continue . . . 

In this case no object is returned as the original array changes it's shape.

3. The ravel() function

To convert a two-dimensional array into a one-dimensional array, use the ravel() function as shown in the following program:

import numpy as np
arr1 = np.random.random(12).reshape(3,4)
print('The original array\n')
print(arr1)
print('\nThe converted array\n')
arr1 = arr1.ravel()
print(arr1)

The output of the program is:

The original array

[[0.09085803 0.78986751 0.81665587 0.98706791]
 [0.75774707 0.75674414 0.17217261 0.28350861]
 [0.8894444  0.70953447 0.90111085 0.68554993]]

The converted array

[0.09085803 0.78986751 0.81665587 0.98706791 0.75774707 0.75674414
 0.17217261 0.28350861 0.8894444  0.70953447 0.90111085 0.68554993]
------------------
(program exited with code: 0)

Press any key to continue . . .

Other way to convert the array is using the shape attribute as shown in the program:

import numpy as np

arr1 = np.random.random(12).reshape(3,4)
print('The original array\n')
print(arr1)
print('\nThe converted array\n')
arr1.shape =(12)
print(arr1)

The output of the program is:

The original array

[[0.93676251 0.87865295 0.64477846 0.50678966]
 [0.07506221 0.44684496 0.06758759 0.70553111]
 [0.25710731 0.42245553 0.5651178  0.82364247]]

The converted array

[0.93676251 0.87865295 0.64477846 0.50678966 0.07506221 0.44684496
 0.06758759 0.70553111 0.25710731 0.42245553 0.5651178  0.82364247]
------------------
(program exited with code: 0)

Press any key to continue . . .

4. The transpose() function

This function is used in transposing a matrix, which is basically inverting the columns with the rows. See the following program:

import numpy as np

arr1 = np.random.random(9).reshape(3,3)
print('The original Matrix is \n')
print(arr1)
print('\nThe transposed Matrix is\n')

print(arr1.transpose())

The output of the program is:

The original Matrix is

[[5.46213281e-04 8.42534325e-01 8.56678343e-01]
 [3.34582626e-01 3.34379643e-01 4.54937957e-01]
 [2.46151757e-01 4.48300885e-01 7.83050542e-01]]

The transposed Matrix is

[[5.46213281e-04 3.34582626e-01 2.46151757e-01]
 [8.42534325e-01 3.34379643e-01 4.48300885e-01]
 [8.56678343e-01 4.54937957e-01 7.83050542e-01]]

------------------
(program exited with code: 0)

Press any key to continue . . .

Here I am ending today's post. In the next post we shall further explore NumPy library and discuss about Array manipulation. Till we meet next keep practicing and learning Python as Python is easy to learn!

Read More

NumPy library - 4 (Indexing, Slicing, and Iterating)

This post will help you to learn how to select elements through indexes and slices in order to obtain the values contained in them. This value can be used to make assignments in order to change their values and to make iterations within them.

Indexing

You might be familiar with the indexing in arrays if you have learnt any programming language. Array indexing always uses square brackets ([ ]) to index the elements of the array so
that the elements can then be referred individually for various purposes such as:

• extracting a value
• selecting items
• assigning a new value

An array index starts at zero, thus if I create an array :

arr1 = np.arange(1,5)

The created array [1 2 3 4] has four elements whose indices are 0,1,2,3. In order to access a single element of an array, you can refer to its index. See the program below :

import numpy as np

arr1 = np.arange(1,5)
print(arr1)
print("\nThe first Element is at [0], it's value is\n")
print(arr1[0])
print("\nThe second Element is at [1], it's value is\n")
print(arr1[1])
print("\nThe third Element is at [2], it's value is\n")
print(arr1[2])
print("\nThe fourth Element is at [3], it's value is\n")
print(arr1[3])

The output of the program is shown below:

[1 2 3 4]

The first Element is at [0], it's value is

1

The second Element is at [1], it's value is

2

The third Element is at [2], it's value is

3

The fourth Element is at [3], it's value is

4
------------------
(program exited with code: 0)

Press any key to continue . . .

In order to access the last element we can us negative indexes which have the same incremental sequence from 0 to –1, –2, and so on, but in practice they cause the final element to move gradually toward the initial element, which will be the one with the more negative index value. Thus arr1[-1] will give the last element of the array. See the program below:

import numpy as np

arr1 = np.arange(1,5)
print(arr1)
print("\nThe last Element is at [-1], it's value is\n")
print(arr1[-1])
print("\nThe second last Element is at [-2], it's value is\n")
print(arr1[-2])
print("\nThe third last Element is at [-3], it's value is\n")
print(arr1[-3])
print("\nThe first Element is at [-4], it's value is\n")
print(arr1[-4])


The output of the program is shown below:

[1 2 3 4]

The last Element is at [-1], it's value is

4

The second last Element is at [-2], it's value is

3

The third last Element is at [-3], it's value is

2

The first Element is at [-4], it's value is

1
------------------
(program exited with code: 0)

Press any key to continue . . .

It is also possible to select multiple elements of an array as shown in the following program:

import numpy as np

arr1 = np.arange(1,9)
print(arr1)
print("\nThe values of selected Elements  [1,2,3,-1,-2] are\n")
print(arr1[[1,2,3,-1,-2]])
 
The output of the program is shown below:

[1 2 3 4 5 6 7 8]

The values of selected Elements  [1,2,3,-1,-2] are

[2 3 4 8 7]

------------------
(program exited with code: 0)

Press any key to continue . . .

Now let's focus on two dimensional arrays (matrices) which are represented as rectangular arrays consisting of rows and columns, defined by two axes, where axis 0 is represented by the rows and axis 1 is represented by the columns. The indexing in this case is represented by a pair of values: the first value is the index of the row and the second is the index of the column, which are inside a square bracket as shown: [row index, column index].  In this case also indexing starts from zero hence the first item will [0,0], second item in first row and second column will be [0,1] and so on..

The following program creates a two dimensional array and access the element from a specified location:

import numpy as np

arr = np.arange(1,10).reshape(3,3)
print(arr)

print("\nThe elements of the array are:\n")
print(arr[0,0])
print(arr[0,1])
print(arr[0,2])
print(arr[1,0])
print(arr[1,1])
print(arr[1,2])
print(arr[2,0])
print(arr[2,1])
print(arr[2,2])

The output of the program is shown below:

[[1 2 3]
 [4 5 6]
 [7 8 9]]

The elements of the array are:

1
2
3
4
5
6
7
8
9
------------------
(program exited with code: 0)

Press any key to continue . . . 

Slicing

We have seen Python list slicing before and the same concept is used to extract portions of an array to generate new arrays. In case of Python lists the resulting array after slicing are copies but in NumPy the resulting arrays are views of the same underlying buffer. The syntax of slicing is [index of the starting element:index of the final element]

The following program extract a portion of the array from second to the sixth element:

import numpy as np

arr = np.arange(1,10)
print(arr)
print("\nThe elements of the array are:\n")
print(arr[1:5])

The output of the program is shown below:

[1 2 3 4 5 6 7 8 9]

The elements of the array are:

[2 3 4 5]
------------------
(program exited with code: 0)

Press any key to continue . . . 

Slicing can be done by specifying different slicing conditions for example we can use a third number that defines the gap in the sequence of the elements. Thus to access the elements in an alternating fashion we can use arr[1:5:2] in our previous program as shown below:

import numpy as np

arr = np.arange(1,10)
print(arr)

print("\nThe elements of the array in alternate fashion are:\n")
print(arr[1:7:2])

The output of the program is shown below:

[1 2 3 4 5 6 7 8 9]

The elements of the array in alternate fashion are:

[2 4 6]

------------------
(program exited with code: 0)

Press any key to continue . . .

Now let's see what happens when we omit the first,second and last values in the slicing condition:

import numpy as np

arr = np.arange(1,10)
print(arr)

print("\nThe elements of the array as per slicing conditions are:\n")
print(arr[1:7:2])
print("\nThe elements of the array If you omit the first number are:\n")
print(arr[:7:2])
print("\nThe elements of the array If you omit the second number are:\n")
print(arr[1::2])
print("\nThe elements of the array If you omit the third number are:\n")
print(arr[1:7:])
print("\nThe elements of the array If you omit the first and second number are:\n")
print(arr[::2])
print("\nThe elements of the array If you omit the second and third number are:\n")
print(arr[1::])
print("\nThe elements of the array If you omit the first and third number are:\n")
print(arr[:7:])

The output of the program is shown below:

[1 2 3 4 5 6 7 8 9]

The elements of the array as per slicing conditions are:

[2 4 6]

The elements of the array If you omit the first number are:

[1 3 5 7]

The elements of the array If you omit the second number are:

[2 4 6 8]

The elements of the array If you omit the third number are:

[2 3 4 5 6 7]

The elements of the array If you omit the first and second number are:

[1 3 5 7 9]

The elements of the array If you omit the second and third number are:

[2 3 4 5 6 7 8 9]

The elements of the array If you omit the first and third number are:

[1 2 3 4 5 6 7] 

------------------
(program exited with code: 0)

Press any key to continue . . .

The output of the program shows that if we omit the first number, NumPy implicitly interprets this number as 0 (i.e., the initial element of the array). If we omit the second number, this will be interpreted as the maximum index of the array; and if we omit the last number this will be interpreted as 1 which means all the elements will be considered without intervals.

Now let's see how slicing is done with the two-dimensional arrays. In this case the slicing syntax is separately defined for the rows and columns. The following program better explains how to perform slicing in two-dimensional arrays:

import numpy as np

arr = np.arange(1,10).reshape(3,3)
print(arr)
print("\nThe elements of the first row are:\n")
print(arr[0,:])
print("\nThe elements of the first column are:\n")
print(arr[:,0])
print("\nThe elements of the second row are:\n")
print(arr[1,:])
print("\nThe elements of the second column are:\n")
print(arr[:,1])
print("\nThe elements of the second row are:\n")
print(arr[2,:])
print("\nThe elements of the second column are:\n")
print(arr[:,2])

The output of the program is shown below:

[[1 2 3]
 [4 5 6]
 [7 8 9]]

The elements of the first row are:

[1 2 3]

The elements of the first column are:

[1 4 7]

The elements of the second row are:

[4 5 6]

The elements of the second column are:

[2 5 8]

The elements of the second row are:

[7 8 9]

The elements of the second column are:

[3 6 9]

------------------
(program exited with code: 0)

Press any key to continue . . .

In order to extract a smaller matrix from the original need to explicitly define all intervals with indexes that define them as shown in the program:

import numpy as np

arr = np.arange(1,10).reshape(3,3)
print(arr)
print('\n')
print(arr[0:1,0:2])
print('\n')
print(arr[0:2,0:2])
print('\n')
print(arr[1:2,1:2])

The output of the program is shown below:

[[1 2 3]
 [4 5 6]
 [7 8 9]]

[[1 2]]

[[1 2]
 [4 5]]


[[5]]

------------------
(program exited with code: 0)

Press any key to continue . . .

In case if the indexes of the rows or columns to be extracted are not contiguous, we can specify an array of indexes as shown in the following program:

import numpy as np

arr = np.arange(1,10).reshape(3,3)
print(arr)
print('\n')
print(arr[[0,2],0:2])

The output of the program is shown below:

[[1 2 3]
 [4 5 6]
 [7 8 9]]


[[1 2]
 [7 8]]

------------------
(program exited with code: 0)

Press any key to continue . . .

Iterating an Array

The iteration of the items in an array can be done using the For loop . Let's perform iteration of a one dimensional array, see the following program:

import numpy as np

arr = np.arange(1,10)
print(arr)
print('\nPrinting array items\n')

for i in arr:
  
    print(i)





The output of the program is shown below:

[1 2 3 4 5 6 7 8 9]

Printing array items
 
1
2
3
4
5
6
7
8
9

------------------
(program exited with code: 0)

Press any key to continue . . . 

If we apply the for loop to a two-dimensional array it will always perform a scan according to the first axis as shown in the program:

import numpy as np

arr = np.arange(1,10).reshape(3,3)
print(arr)
print('\nPrinting array items\n')

for i in arr:
   
    print(i)


The output of the program is shown below:

[[1 2 3]
 [4 5 6]
 [7 8 9]]

Printing array items

[1 2 3]
[4 5 6]
[7 8 9]
------------------
(program exited with code: 0)

Press any key to continue . . . 

Thus an element by element iteration is not done. NumPy has special construct to perform this iteration as shown in this program:

import numpy as np

arr = np.arange(1,10).reshape(3,3)
print(arr)
print('\nPrinting array items\n')

for i in arr.flat:
   
    print(i)

The output of the program is shown below:

[[1 2 3]
 [4 5 6]
 [7 8 9]]

Printing array items

1
2
3
4
5
6
7
8
9
------------------
(program exited with code: 0)

Press any key to continue . . .

NumPy has a special apply_along_axis() function to manage the iterations. This function takes three arguments: the aggregate function, the axis on which to apply the iteration, and the array. If the option axis equals 0, then the iteration evaluates the elements column by column, whereas if axis equals 1 then the iteration evaluates the elements row by row. The following program uses this function:

import numpy as np

arr1 = np.arange(1,10).reshape(3,3)
print(arr1)
print('\nPrinting array items\n')
print(np.apply_along_axis(np.mean, axis=0, arr=arr1))
print('\nPrinting array items\n')
print(np.apply_along_axis(np.mean, axis=1, arr=arr1))

The output of the program is shown below:

[[1 2 3]
 [4 5 6]
 [7 8 9]]

Printing array items

[4. 5. 6.]

Printing array items

[2. 5. 8.]

------------------
(program exited with code: 0)

Press any key to continue . . .

The above program calculate the average values first by column and then by row. We can also use ufunctions as the first argument in the apply_along_axis() function as shown in the following program:

import numpy as np

arr1 = np.arange(1,10).reshape(3,3)
print(arr1)
print('\nPrinting array items\n')
print(np.apply_along_axis(np.sqrt, axis=0, arr=arr1))
print('\nPrinting array items\n')
print(np.apply_along_axis(np.max, axis=0, arr=arr1))
print('\nPrinting array items\n')
print(np.apply_along_axis(np.min, axis=0, arr=arr1))
print('\nPrinting array items\n')
print(np.apply_along_axis(np.log, axis=0, arr=arr1))

The output of the program is shown below:

[[1 2 3]
 [4 5 6]
 [7 8 9]]

Printing array items

[[1.         1.41421356 1.73205081]
 [2.         2.23606798 2.44948974]
 [2.64575131 2.82842712 3.        ]]

Printing array items

[7 8 9]

Printing array items

[1 2 3]

Printing array items

[[0.         0.69314718 1.09861229]
 [1.38629436 1.60943791 1.79175947]
 [1.94591015 2.07944154 2.19722458]]


------------------
(program exited with code: 0)

Press any key to continue . . .

Here I am ending today's post. In the next post we shall further explore NumPy library and continue with indexing and slicing. Till we meet next keep practicing and learning Python as Python is easy to learn!

Read More

NumPy library -3 (Basic operations on NumPy)

In the previous posts we learnt how to create NumPy arrays and define items in it. In this post we shall discuss how to apply various operations to them.

Arithmetic Operators

The common arithmetic operators used with arrays are addition and multiplication. The following program show adding an array with a scalar:

import numpy as np

arr = np.arange(5)
print(arr)
arr= arr+5
print('\n')
print(arr)

The output of the program is shown below:

[0 1 2 3 4]


[5 6 7 8 9]
------------------
(program exited with code: 0)

Press any key to continue . . .

Similarly multiplication of array with a scalar can be done as follows:

import numpy as np

arr = np.arange(5)
print(arr)
arr= arr*5
print('\n')
print(arr)

The output of the program is shown below:

[0 1 2 3 4]

[ 0  5 10 15 20]
------------------
(program exited with code: 0)

Press any key to continue . . .

These operators can also be used between two arrays. In NumPy, these operations are element-wise, that is, the operators are applied only between corresponding elements. These are objects that occupy the same position, so that the end result will be a new array containing the results in the same location of the operands. The following program shows addition and multiplication of two arrays:

import numpy as np

arr1 = np.arange(5)
arr2 = np.arange(5,10)

print('Addition of two arrays\n')
print(arr1+arr2)

print('\nMultiplication of two arrays\n')
print(arr1*arr2)

The output of the program is shown below:

Addition of two arrays

[ 5  7  9 11 13]

Multiplication of two arrays

[ 0  6 14 24 36]

------------------
(program exited with code: 0)

Press any key to continue . . .

We can also multiply the array by the sine or the square root of the elements of array b as shown in the following program:

import numpy as np

arr1 = np.arange(5)
arr2 = np.arange(5,10)

print('Adding the array by the sine value of the elements of array of another array\n')
print(arr1 + np.sin(arr2))

print('\nAdding the array by the square root of the elements of array\n')
print(arr1 + np.sqrt(arr2))

print('\nMultiplying the array by the sine value of the elements of array of another array\n')
print(arr1 * np.sin(arr2))

print('\nMultiplying the array by the square root of the elements of array\n')
print(arr1 * np.sqrt(arr2))

The output of the program is shown below:

Adding the array by the sine value of the elements of array of another array

[-0.95892427  0.7205845   2.6569866   3.98935825  4.41211849]

Adding the array by the square root of the elements of array

[2.23606798 3.44948974 4.64575131 5.82842712 7.        ]

Multiplying the array by the sine value of the elements of array of another arra
y

[-0.         -0.2794155   1.3139732   2.96807474  1.64847394]

Multiplying the array by the square root of the elements of array

[ 0.          2.44948974  5.29150262  8.48528137 12.        ]

------------------
(program exited with code: 0)

Press any key to continue . . .

These operators can also be used with the multidimensional arrays. The following program shows addition and multiplication of the multidimensional arrays:

import numpy as np

arr1 = np.arange(0,9).reshape(3,3)
arr2 = np.ones((3,3))

print(arr1)
print('\n')
print(arr2)

print('\nAdding the arrays\n')
print(arr1 + arr2)


print('\nMultiplying the arrays\n')
print(arr1 * arr2)

The output of the program is shown below:

[[0 1 2]
 [3 4 5]
 [6 7 8]]


[[1. 1. 1.]
 [1. 1. 1.]
 [1. 1. 1.]]

Adding the arrays

[[1. 2. 3.]
 [4. 5. 6.]
 [7. 8. 9.]]

Multiplying the arrays

[[0. 1. 2.]
 [3. 4. 5.]
 [6. 7. 8.]]

------------------
(program exited with code: 0)

Press any key to continue . . .  

We have converted the one dimensional array arr1 into a 3,3 array using the reshape() function. In this case also element wise operation takes place. If a matrix product is required then we can use the dot() function as shown in the following program:

import numpy as np

arr1 = np.arange(0,9).reshape(3,3)
arr2 = np.ones((3,3))

print(arr1)
print('\n')
print(arr2)

print('\nMatrix product of the arrays\n')
print(np.dot(arr1,arr2))

The output of the program is shown below:

[[0 1 2]
 [3 4 5]
 [6 7 8]]


[[1. 1. 1.]
 [1. 1. 1.]
 [1. 1. 1.]]

Matrix product of the arrays

[[ 3.  3.  3.]
 [12. 12. 12.]
 [21. 21. 21.]]
------------------
(program exited with code: 0)

Press any key to continue . . . 

I hope you are familiar with the matrix product which is the sum of the products of each element of
the corresponding row of the first matrix with the corresponding element of the corresponding column of the second matrix.

There is an alternative way to write the matrix product which is to see the dot() function as an
object’s function of one of the two matrices, i.e. arr1.dot(arr2). In our previous program use this and verify if the resulting matrix is the same as obtained with np.dot(arr1,arr2). What if the order of the operands are changed? i.e result of arr2.dot(arr1) ..The following program shows the result:

import numpy as np

arr1 = np.arange(0,9).reshape(3,3)
arr2 = np.ones((3,3))

print(arr1)
print('\n')
print(arr2)

print('\nMatrix product of the arrays\n')
print(np.dot(arr1,arr2))
print("\nMatrix product of the arrays using dot function as an object's function of one of the two matrices \n")
print(arr1.dot(arr2))
print('\nMatrix product of the arrays by changing the order of the operands\n')
print(arr2.dot(arr1))

The output of the program is shown below:

[[0 1 2]
 [3 4 5]
 [6 7 8]]


[[1. 1. 1.]
 [1. 1. 1.]
 [1. 1. 1.]]

Matrix product of the arrays

[[ 3.  3.  3.]
 [12. 12. 12.]
 [21. 21. 21.]]

Matrix product of the arrays using dot function as an object's function of one o
f the two matrices

[[ 3.  3.  3.]
 [12. 12. 12.]
 [21. 21. 21.]]

Matrix product of the arrays by changing the order of the operands

[[ 9. 12. 15.]
 [ 9. 12. 15.]
 [ 9. 12. 15.]]
------------------
(program exited with code: 0)

Press any key to continue . . .

The above result shows that the matrix product is not a commutative operation, thus the order of the
operands is important.

Increment and Decrement Operators

In Python the increment and decrement operations is conducted using += and -= operators. The following example shows their usage:

import numpy as np

arr1 = np.arange(5)

print(arr1)
print('\n')
arr1 += 1
print(arr1)
print('\n')
arr1 -= 1
print(arr1)

The output of the program is shown below:

[0 1 2 3 4]

[1 2 3 4 5]

[0 1 2 3 4]
------------------
(program exited with code: 0)

Press any key to continue . . . 

These operators instead of creating a new array with the results, reassign the results to the same array. Thus we can use them every time we want to change the values in an array without generating a new one. The following program shows how this can be done:

import numpy as np

arr1 = np.arange(5)

print(arr1)
print('\n')
arr1 += 5
print(arr1)
print('\n')
arr1 *= 5
print(arr1)

The output of the program is shown below:

[0 1 2 3 4]

[5 6 7 8 9]
 
[25 30 35 40 45]
------------------
(program exited with code: 0)

Press any key to continue . . .

Universal Functions (ufunc)

These functions operate on an array in an element by element fashion. Thus they act individually on each single element of the input array to generate a corresponding result in a new output array of the same size as the input. The most common examples of these are the mathematical and trigonometric operations. The following program shows some of these functions:

import numpy as np

arr1 = np.arange(1,5)
print(arr1)
print('\n')
print(np.sqrt(arr1))
print('\n')
print(np.log(arr1))
print('\n')
print(np.sin(arr1))

The output of the program is shown below:

[1 2 3 4]

[1.         1.41421356 1.73205081 2.        ]

[0.         0.69314718 1.09861229 1.38629436]

[ 0.84147098  0.90929743  0.14112001 -0.7568025 ]

------------------
(program exited with code: 0)

Press any key to continue . . . 

Aggregate functions

The Aggregate functions perform an operation on an array and produce a single result. Therefore, the sum of all the elements in an array is an aggregate function. Many Aggregate functions are implemented within the class ndarray. The following program shows some of these functions:

import numpy as np

arr1 = np.arange(1,5)
print(arr1)
print('\n')
print(arr1.sum())
print('\n')
print(arr1.min())
print('\n')
print(arr1.max())
print('\n')
print(arr1.mean())
print('\n')
print(arr1.std())

The output of the program is shown below:

[1 2 3 4]

10

1

4

2.5

1.118033988749895

------------------
(program exited with code: 0)

Press any key to continue . . .

Here I am ending today's post. In the next post we shall further explore NumPy library and see how to manipulate the array objects, i.e how to select elements through indexes and slices, in order to obtain the values contained in them or to make assignments in order to change their values. Till we meet next keep practicing and learning Python as Python is easy to learn!

Read More

NumPy library -2 (functions that generate ndarrays)

The NumPy library provides a set of functions that generate ndarrays with initial content, created with different values depending on the function. We have already seen some of these functions in the previous post which allows a single line of code to generate large amounts of data.

1. The zeros() function

This function creates a full array of zeros with dimensions defined by the shape argument. Let's create a two-dimensional array 3x3 using the zeros() function:

import numpy as np

arr = np.zeros((3, 3))
print(arr)

The output shows the created two-dimensional array 3x3 with all zeros:

[[0. 0. 0.]
 [0. 0. 0.]
 [0. 0. 0.]]
------------------
(program exited with code: 0)

Press any key to continue . . .

The created array is with the float64 data type.

2. The ones() function

This function creates a full array of ones with dimensions defined by the shape argument. Let's create a two-dimensional array 3x3 using the ones() function:

import numpy as np

arr = np.ones((3, 3))
print(arr)

The output shows the created two-dimensional array 3x3 with all ones:

[[1. 1. 1.]
 [1. 1. 1.]
 [1. 1. 1.]]
------------------
(program exited with code: 0)

Press any key to continue . . .

The created array is with the float64 data type.

3. The arange() function

This function generates NumPy arrays with numerical sequences that respond to particular rules depending on the passed arguments. For example, if you want to generate a sequence of values between 0 and 10, you will be passed only one argument to the function, that is the value with which you want to end the sequence. See the following code:

import numpy as np

arr = np.arange(0, 10)
print(arr)

The output shows the created array:

[0 1 2 3 4 5 6 7 8 9]

------------------
(program exited with code: 0)

Press any key to continue . . .
In case if we wish to start from any other number than 0 then we can modify the first argument of the arange() as shown below:

import numpy as np

arr = np.arange(5, 10)
print(arr)

The output shows the created array:

[ 5 6 7 8 9]



------------------
(program exited with code: 0)

Press any key to continue . . . 

We can also specify a third argument the range function which represent the gap between one value and the next one in the sequence of values. Thus it is possible to generate a sequence of values with precise intervals between them. The following program creates an array of numbers in range 0 to 20 with an interval of 4:

import numpy as np

arr = np.arange(0, 20, 4)

print(arr)

The output shows the created array:


[ 0  4  8 12 16]

------------------
(program exited with code: 0)

Press any key to continue . . .

The third argument can also be a float type as shown in the following program:

import numpy as np

arr = np.arange(0, 2, 0.4)

print(arr)

The output shows the created array:

[0.  0.4 0.8 1.2 1.6]

------------------
(program exited with code: 0)

Press any key to continue . . .

The output shows the created arrays with arange() function are by default one dimensional arrays. To generate two-dimensional arrays using the arange() function, combine it with the reshape() function. This function divides a linear array in different parts in the manner specified by the shape argument. See the example below:

import numpy as np

arr = np.arange(0, 12).reshape(3, 4)
print(arr)

The output shows the created array:

[[ 0  1  2  3]
 [ 4  5  6  7]
 [ 8  9 10 11]]
------------------
(program exited with code: 0)

Press any key to continue . . .

4. The linspace() function

This function takes as its first two arguments the initial and end values of the sequence, the third argument defines the number of elements into which we want the interval to be split.

import numpy as np

arr = np.linspace(0,10,5)
print(arr)

The output shows the created array:

[ 0.   2.5  5.   7.5 10. ]
------------------
(program exited with code: 0)

Press any key to continue . . . 

5. The linspace() function

This function will generate an array with many elements as specified in the argument. The obtained arrays contains random values. See the following program:

import numpy as np

arr = np.random.random(5)
print(arr)

The output shows the created array:

[0.09067762 0.15551911 0.23330312 0.01154011 0.87073787]

------------------
(program exited with code: 0)

Press any key to continue . . .

Here I am ending today's post. In the next post we shall further explore NumPy library and see how to apply various operations to them . Till we meet next keep practicing and learning Python as Python is easy to learn!

Read More

NumPy library -1 (Introduction)

NumPy is the foundation library for scientific computing in Python since it provides data structures and high-performing functions that the basic package of the Python cannot provide. NumPy defines a specific data structure that is an N-dimensional array defined as ndarray. Knowledge of NumPy  is essential in terms of numerical calculations since its correct use can greatly influence the performance of your computations.

This package provides some features that will be added to the standard Python:

• Ndarray: A multidimensional array much faster and more efficient than those provided by the basic package of Python.

• Element-wise computation: A set of functions for performing this type of calculation with arrays and mathematical operations between arrays.

• Reading-writing datasets: A set of tools for reading and writing data stored in the hard disk.

• Integration with other languages such as C, C++, and FORTRAN: A set of tools to integrate code developed with these programming languages.

The installation of NumPy is easy using the PIP-

pip install NumPy

On Windows with Anaconda, we can use:

conda install numpy

NumPy is important for numerical computations in Python because it is designed for efficiency on large arrays of data. There are a number of reasons for this:

1. NumPy internally stores data in a contiguous block of memory, independent of other built-in Python objects. NumPy’s library of algorithms written in the C language can operate on this memory without any type checking or other overhead. NumPy arrays also use much less memory than built-in Python sequences.

2. NumPy operations perform complex computations on entire arrays without the need for Python for loops.

3. NumPy-based algorithms are generally 10 to 100 times faster (or more) than their pure Python counterparts and use significantly less memory.



Ndarray

The main object of NumPy library is ndarray (which stands for N-dimensional array) is a multidimensional homogeneous array with a predetermined number of items: homogeneous because virtually all the items in it are of the same type and the same size. In fact, the data type is specified by another NumPy object called dtype (data-type); each ndarray is associated with only one type of dtype.

The number of the dimensions and items in an array is defined by its shape, a tuple of N-positive integers that specifies the size for each dimension. The dimensions are defined as axes and the number of axes as rank. The size of NumPy arrays is fixed, that is, once we define their size at the time of creation, it remains unchanged. This behavior is different from Python lists, which can grow or shrink in size.

We can define a ndarray using the array() function which takes Python list containing the elements to be included in it as an argument.

Our first program aims to prove that NumPy-based algorithms are generally 10 to 100 times faster (or more) than their pure Python counterparts. I'll be using IPython often in the future so its better to install iPython as shown below:

Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation.  All rights reserved.

C:\Users\Python>pip install ipython
Collecting ipython
  Downloading https://files.pythonhosted.org/packages/f0/b4/a9ea018c73a84ee6280b
2e94a1a6af8d63e45903eac2da0640fa63bca4db/ipython-7.2.0-py3-none-any.whl (765kB)
    100% |████████████████████████████████| 768kB 588kB/s
Collecting colorama; sys_platform == "win32" (from ipython)
  Downloading https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc9
4c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting prompt-toolkit<2.1.0,>=2.0.0 (from ipython)
  Downloading https://files.pythonhosted.org/packages/65/c2/e676da701cda11b32ff4
2eceb44aa7d8934b597d604bb5e94c0283def064/prompt_toolkit-2.0.8-py3-none-any.whl (
342kB)
    100% |████████████████████████████████| 348kB 320kB/s
Collecting backcall (from ipython)
  Downloading https://files.pythonhosted.org/packages/84/71/c8ca4f5bb1e08401b916
c68003acf0a0655df935d74d93bf3f3364b310e0/backcall-0.1.0.tar.gz
Collecting pickleshare (from ipython)
  Downloading https://files.pythonhosted.org/packages/9a/41/220f49aaea88bc6fa6cb
a8d05ecf24676326156c23b991e80b3f2fc24c77/pickleshare-0.7.5-py2.py3-none-any.whl
Collecting pygments (from ipython)
  Downloading https://files.pythonhosted.org/packages/13/e5/6d710c9cf96c31ac8265
7bcfb441df328b22df8564d58d0c4cd62612674c/Pygments-2.3.1-py2.py3-none-any.whl (84
9kB)
    100% |████████████████████████████████| 849kB 522kB/s
Collecting jedi>=0.10 (from ipython)
  Downloading https://files.pythonhosted.org/packages/c2/bc/54d53f5bc4658380d0ec
a9055d72be4df45e5bfd91a4bac97da224a92553/jedi-0.13.2-py2.py3-none-any.whl (177kB
)
    100% |████████████████████████████████| 184kB 386kB/s
Requirement already satisfied: setuptools>=18.5 in c:\users\python\appdata\local
\programs\python\python36\lib\site-packages (from ipython) (28.8.0)
Collecting traitlets>=4.2 (from ipython)
  Downloading https://files.pythonhosted.org/packages/93/d6/abcb22de61d78e2fc395
9c964628a5771e47e7cc60d53e9342e21ed6cc9a/traitlets-4.3.2-py2.py3-none-any.whl (7
4kB)
    100% |████████████████████████████████| 81kB 239kB/s
Collecting decorator (from ipython)
  Downloading https://files.pythonhosted.org/packages/f1/cd/7c8240007e9716b14679
bc217a1baefa4432aa30394f7e2ec40a52b1a708/decorator-4.3.2-py2.py3-none-any.whl
Requirement already satisfied: six>=1.9.0 in c:\users\python\appdata\roaming\pyt
hon\python36\site-packages (from prompt-toolkit<2.1.0,>=2.0.0->ipython) (1.11.0)

Collecting wcwidth (from prompt-toolkit<2.1.0,>=2.0.0->ipython)
  Downloading https://files.pythonhosted.org/packages/7e/9f/526a6947247599b084ee
5232e4f9190a38f398d7300d866af3ab571a5bfe/wcwidth-0.1.7-py2.py3-none-any.whl
Collecting parso>=0.3.0 (from jedi>=0.10->ipython)
  Downloading https://files.pythonhosted.org/packages/19/b1/522b2671cc6d134c9d3f
5dfc0d02fee07cab848e908d03d2bffea78cca8f/parso-0.3.4-py2.py3-none-any.whl (93kB)

    100% |████████████████████████████████| 102kB 652kB/s
Collecting ipython-genutils (from traitlets>=4.2->ipython)
  Downloading https://files.pythonhosted.org/packages/fa/bc/9bd3b5c2b4774d5f33b2
d544f1460be9df7df2fe42f352135381c347c69a/ipython_genutils-0.2.0-py2.py3-none-any
.whl
Building wheels for collected packages: backcall
  Running setup.py bdist_wheel for backcall ... done
  Stored in directory: C:\Users\Python\AppData\Local\pip\Cache\wheels\98\b0\dd\2
9e28ff615af3dda4c67cab719dd51357597eabff926976b45
Successfully built backcall
Installing collected packages: colorama, wcwidth, prompt-toolkit, backcall, pick
leshare, pygments, parso, jedi, ipython-genutils, decorator, traitlets, ipython
Successfully installed backcall-0.1.0 colorama-0.4.1 decorator-4.3.2 ipython-7.2
.0 ipython-genutils-0.2.0 jedi-0.13.2 parso-0.3.4 pickleshare-0.7.5 prompt-toolk
it-2.0.8 pygments-2.3.1 traitlets-4.3.2 wcwidth-0.1.7

Now from command window ipython can be started as shown below:



C:\Users\Python>ipython
Python 3.6.5rc1 (v3.6.5rc1:f03c5148cf, Mar 14 2018, 03:12:11) [MSC v.1913 64 bit
 (AMD64)]
Type 'copyright', 'credits' or 'license' for more information
IPython 7.2.0 -- An enhanced Interactive Python. Type '?' for help.





In [1]: import numpy as np

In [2]: a = np.arange(1000000)

In [3]: l = list(range(1000000))

In [4]: %time for _ in range(10): a2 = a * 2
Wall time: 20 ms

In [5]: %time for _ in range(10): l2 = [x * 2 for x in l]
Wall time: 1.09 s

The above program creates an array of one million integers, and the equivalent Python list 'l'. When we notice the wall time we can see that NumPy-based algorithms are generally 10 to 100 times faster (or more) than their pure Python counterparts (lists).

Another program which proves the above statement is shown below:

In [6]: arr = np.arange(1e7)

In [7]: larr = arr.tolist()

In [8]: def list_times(alist, scalar):
   ...:     for i, val in enumerate(alist):
   ...:         alist[i] = val * scalar
   ...:     return alist
   ...:

In [9]: timeit arr * 1.1
36.2 ms ± 173 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)

In [10]: timeit list_times(larr, 1.1)
1.08 s ± 20.6 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

As you can see the ndarray operation is ∼ 25 faster than the Python loop in this example.


Now let's focus on ndarrays and see array creation and data typing fundamentals. There are many ways to create an array in NumPy, and here we will discuss the ones that are most useful.

1. Creating array from lists

First we create a list and then wrap it with the np.array() function as shown in the code below:

alist = [1, 2, 3]
arr = np.array(alist)

We can also pass the list as an argument to the np.array() as shown below:

a = np.array([1, 2, 3])

The following program creates an array and print it:

import numpy as np

alist = [1, 2, 3]
arr = np.array(alist)
a = np.array([1, 2, 3])
print(arr)
print(a) 

The output of the program is shown below:

[1 2 3]
[1 2 3]
------------------
(program exited with code: 0)

Press any key to continue . . .

As we can see the both the arrays are same which contains the list elements. We can check that a newly created object is an ndarray by passing the new variable to the type() function:

print(type(arr))
print(type(a)) 

The output of the above program using type() is shown below:

[1 2 3]
[1 2 3]
<class 'numpy.ndarray'>
<class 'numpy.ndarray'>
------------------
(program exited with code: 0)

Press any key to continue . . . 

An ndarray is a generic multidimensional container for homogeneous data; that is, all of the elements must be the same type. Every array has a shape, a tuple indicating the size of each dimension, and a dtype, an object describing the data type of the array. In order to know the associated dtype to the newly created ndarray, we use the dtype attribute as shown below:

print(arr.dtype)
print(a.dtype)

The output of the above program using dtype attribute is shown below:

[1 2 3]
[1 2 3]
<class 'numpy.ndarray'>
<class 'numpy.ndarray'>
int32
int32
------------------
(program exited with code: 0)

Press any key to continue . . .

In order to get the shape of the newly created ndarray, we use the shape attribute, use the ndim
attribute for getting the axes, the size attribute to know the array length as shown below:

import numpy as np

a = np.array([1, 2, 3])
print(a.shape)
print(a.ndim)
print(a.size)

The output of the above program using the attributes is shown below:

(3,)
1
3
------------------
(program exited with code: 0)

Press any key to continue . . .

In order to create a two dimensional array we can use the np.array() function as shown below:

arr = np.array([[1.5, 2.8],[0.5, 4.5]])

See the following program which creates and print the array and its attributes:

import numpy as np

arr = np.array([[1.5, 2.8],[0.5, 4.5]])
print(arr)
print(arr.dtype)
print(arr.shape)
print(arr.ndim)
print(arr.size)

The output of the above program using the attributes is shown below:

[[1.5 2.8]
 [0.5 4.5]]
float64
(2, 2)
2
4
------------------
(program exited with code: 0)

Press any key to continue . . .

The arr array has rank 2 as shown from the output arr.ndim, it has two axes each of length 2. To determine the size of each item in the array, the itemsize attribute is used which defines the size in bytes of each array item. See the following program:

import numpy as np

arr = np.array([[1.5, 2.8],[0.5, 4.5]])
print(arr.itemsize)

The output of the above program is shown below:

8
------------------
(program exited with code: 0)

Press any key to continue . . . 

The array() function, in addition to lists, can accept tuples, sequences of tuples and interconnected lists. See the program below:

import numpy as np

arr = np.array(((1, 2, 3),(4, 5, 6)))

print(arr)
print('\n')

arr = np.array([(1, 2, 3), [4, 5, 6], (7, 8, 9)])

print(arr)


The output of the above program is shown below:

[[1 2 3]
 [4 5 6]]


[[1 2 3]
 [4 5 6]
 [7 8 9]]

------------------
(program exited with code: 0)

Press any key to continue . . .

In the following program we can see some more ways to create an array in NumPy:

import numpy as np

# Creating an array of zeros with five elements
arr = np.zeros(5)
print("Creating an array of zeros with five elements\n")
print(arr)
print('\n')

#create an array going from 0 to 100
arr = np.arange(100)
print("create an array going from 0 to 100\n")
print(arr)
print('\n')

# Or 10 to 100?
arr = np.arange(10,100)
print("Or 10 to 100\n")
print(arr)
print('\n')

# If you want 100 steps from 0 to 1...
arr = np.linspace(0, 1, 100)
print("100 steps from 0 to 1\n")
print(arr)
print('\n')

# Or if you want to generate an array from 1 to 10
# in log10 space in 100 steps...
arr = np.logspace(0, 1, 100, base=10.0)
print("generate an array from 1 to 10 in log10 space in 100 steps\n")
print(arr)
print('\n')

# Creating a 5x5 array of zeros (an image)
image = np.zeros((5,5))
print("Creating a 5x5 array of zeros (an image)\n")
print(image)
print('\n')

# Creating a 5x5x5 cube of 1's
# The astype() method sets the array with integer elements.
cube = np.zeros((5,5,5)).astype(int) + 1
print("Creating a 5x5x5 cube of 1's\n")
print(cube)
print('\n')

# Or even simpler with 16-bit floating-point precision...
cube = np.ones((5, 5, 5)).astype(np.float16)
print("with 16-bit floating-point precision\n")
print(cube)
print('\n')

The output of the above program is shown below:

Creating an array of zeros with five elements

[0. 0. 0. 0. 0.]


create an array going from 0 to 100

[ 0  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47
 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71
 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95
 96 97 98 99]


Or 10 to 100

[10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33
 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57
 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81
 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]


100 steps from 0 to 1

[0.         0.01010101 0.02020202 0.03030303 0.04040404 0.05050505
 0.06060606 0.07070707 0.08080808 0.09090909 0.1010101  0.11111111
 0.12121212 0.13131313 0.14141414 0.15151515 0.16161616 0.17171717
 0.18181818 0.19191919 0.2020202  0.21212121 0.22222222 0.23232323
 0.24242424 0.25252525 0.26262626 0.27272727 0.28282828 0.29292929
 0.3030303  0.31313131 0.32323232 0.33333333 0.34343434 0.35353535
 0.36363636 0.37373737 0.38383838 0.39393939 0.4040404  0.41414141
 0.42424242 0.43434343 0.44444444 0.45454545 0.46464646 0.47474747
 0.48484848 0.49494949 0.50505051 0.51515152 0.52525253 0.53535354
 0.54545455 0.55555556 0.56565657 0.57575758 0.58585859 0.5959596
 0.60606061 0.61616162 0.62626263 0.63636364 0.64646465 0.65656566
 0.66666667 0.67676768 0.68686869 0.6969697  0.70707071 0.71717172
 0.72727273 0.73737374 0.74747475 0.75757576 0.76767677 0.77777778
 0.78787879 0.7979798  0.80808081 0.81818182 0.82828283 0.83838384
 0.84848485 0.85858586 0.86868687 0.87878788 0.88888889 0.8989899
 0.90909091 0.91919192 0.92929293 0.93939394 0.94949495 0.95959596
 0.96969697 0.97979798 0.98989899 1.        ]


generate an array from 1 to 10 in log10 space in 100 steps

[ 1.          1.02353102  1.04761575  1.07226722  1.09749877  1.12332403
  1.149757    1.17681195  1.20450354  1.23284674  1.26185688  1.29154967
  1.32194115  1.35304777  1.38488637  1.41747416  1.45082878  1.48496826
  1.51991108  1.55567614  1.59228279  1.62975083  1.66810054  1.70735265
  1.7475284   1.78864953  1.83073828  1.87381742  1.91791026  1.96304065
  2.009233    2.05651231  2.10490414  2.15443469  2.20513074  2.25701972
  2.3101297   2.36448941  2.42012826  2.47707636  2.53536449  2.59502421
  2.65608778  2.71858824  2.7825594   2.84803587  2.91505306  2.98364724
  3.05385551  3.12571585  3.19926714  3.27454916  3.35160265  3.43046929
  3.51119173  3.59381366  3.67837977  3.76493581  3.85352859  3.94420606
  4.03701726  4.1320124   4.22924287  4.32876128  4.43062146  4.53487851
  4.64158883  4.75081016  4.86260158  4.97702356  5.09413801  5.21400829
  5.33669923  5.46227722  5.59081018  5.72236766  5.85702082  5.9948425
  6.13590727  6.28029144  6.42807312  6.57933225  6.73415066  6.8926121
  7.05480231  7.22080902  7.39072203  7.56463328  7.74263683  7.92482898
  8.11130831  8.30217568  8.49753436  8.69749003  8.90215085  9.11162756
  9.32603347  9.54548457  9.77009957 10.        ]


Creating a 5x5 array of zeros (an image)

[[0. 0. 0. 0. 0.]
 [0. 0. 0. 0. 0.]
 [0. 0. 0. 0. 0.]
 [0. 0. 0. 0. 0.]
 [0. 0. 0. 0. 0.]]


Creating a 5x5x5 cube of 1's

[[[1 1 1 1 1]
  [1 1 1 1 1]
  [1 1 1 1 1]
  [1 1 1 1 1]
  [1 1 1 1 1]]

 [[1 1 1 1 1]
  [1 1 1 1 1]
  [1 1 1 1 1]
  [1 1 1 1 1]
  [1 1 1 1 1]]

 [[1 1 1 1 1]
  [1 1 1 1 1]
  [1 1 1 1 1]
  [1 1 1 1 1]
  [1 1 1 1 1]]

 [[1 1 1 1 1]
  [1 1 1 1 1]
  [1 1 1 1 1]
  [1 1 1 1 1]
  [1 1 1 1 1]]

 [[1 1 1 1 1]
  [1 1 1 1 1]
  [1 1 1 1 1]
  [1 1 1 1 1]
  [1 1 1 1 1]]]


with 16-bit floating-point precision

[[[1. 1. 1. 1. 1.]
  [1. 1. 1. 1. 1.]
  [1. 1. 1. 1. 1.]
  [1. 1. 1. 1. 1.]
  [1. 1. 1. 1. 1.]]

 [[1. 1. 1. 1. 1.]
  [1. 1. 1. 1. 1.]
  [1. 1. 1. 1. 1.]
  [1. 1. 1. 1. 1.]
  [1. 1. 1. 1. 1.]]

 [[1. 1. 1. 1. 1.]
  [1. 1. 1. 1. 1.]
  [1. 1. 1. 1. 1.]
  [1. 1. 1. 1. 1.]
  [1. 1. 1. 1. 1.]]

 [[1. 1. 1. 1. 1.]
  [1. 1. 1. 1. 1.]
  [1. 1. 1. 1. 1.]
  [1. 1. 1. 1. 1.]
  [1. 1. 1. 1. 1.]]

 [[1. 1. 1. 1. 1.]
  [1. 1. 1. 1. 1.]
  [1. 1. 1. 1. 1.]
  [1. 1. 1. 1. 1.]
  [1. 1. 1. 1. 1.]]]


------------------
(program exited with code: 0)

Press any key to continue . . .


NumPy arrays are designed to contain a wide variety of data types. Let us create an array of string type:

import numpy as np

arr = np.array([['a', 'b'],['c', 'd']])
print(arr)

print(arr.dtype)
print(arr.dtype.name)




The above program creates an array of string type and print it. Next we verify the type of array using the dtype method and print the name of stringtype using dtype.name attribute. The output of the above program is shown below:

[['a' 'b']
 ['c' 'd']]
<U1
str32
------------------
(program exited with code: 0)

Press any key to continue . . .

The data types supported by NumPy are:

bool_               Boolean (true or false) stored as a byte
int_                 Default integer type (same as C long; normally either int64 or int32)
intc                 Identical to C int (normally int32 or int64)
intp                Integer used for indexing (same as C size_t; normally either int32 or int64)
int8                Byte (–128 to 127)
int16              Integer (–32768 to 32767)
int32              Integer (–2147483648 to 2147483647)
int64              Integer (–9223372036854775808 to 9223372036854775807)
uint8              Unsigned integer (0 to 255)
uint16            Unsigned integer (0 to 65535)
uint32            Unsigned integer (0 to 4294967295)
uint64            Unsigned integer (0 to 18446744073709551615)
float_             Shorthand for float64
float16           Half precision float: sign bit, 5-bit exponent, 10-bit mantissa
float32           Single precision float: sign bit, 8-bit exponent, 23-bit mantissa
float64          Double precision float: sign bit, 11-bit exponent, 52-bit mantissa
complex_      Shorthand for complex128
complex64    Complex number, represented by two 32-bit floats (real and imaginary components)
complex128  Complex number, represented by two 64-bit floats (real and imaginary components)

The array() function can associate the most suitable type according to the values contained in the sequence of lists or tuples but it is also possible to define the dtype using the dtype option as argument of the function. In the following program we first define and array of integers and later convert the same array with complex values using dtype=complex:

import numpy as np

arr = np.array([[1, 2, 3],[4, 5, 6]])
print(arr)

print("\nArray with complex values\n")

arr = np.array([[1, 2, 3],[4, 5, 6]], dtype=complex)
print(arr)
print('\n')
print(arr.dtype)

The output of the above program is shown below:

[[1 2 3]
 [4 5 6]]

Array with complex values

[[1.+0.j 2.+0.j 3.+0.j]
 [4.+0.j 5.+0.j 6.+0.j]]


complex128
------------------
(program exited with code: 0)

Press any key to continue . . .

Here I am ending today's post. In the next post we shall further explore NumPy library. Till we meet next keep practicing and learning Python as Python is easy to learn!

Read More

Using sqlite with tkinter

In this post we'll see how SQLite3 allows for interaction with databases in Python, through SQL syntax by making a small program. This application "COVRI Training " allows a user to select a course from our training program in programming languages along with the training duration and an option to do project based training along with the basic training.

The GUI for application to be developed is shown below:



Let's start coding to implement the GUI of our COVRI Training application as shown above. First step as always is to import relevant libraries,the Tkinter to build the GUI and SQLite3 for interaction with the database. Tkinter and SQLite3 are libraries that come with the standard Python library.

from tkinter import *
import sqlite3 as sq

Next step will be creating root window:

window = Tk()
window.title("COVRI Training")
window.geometry('800x600+0+0')
header = Label(window, text="Available courses", font=("arial",30,"bold"), fg="goldenrod").pack()

The root window is created and assigned to ‘window’this can be any variable, sometimes I use root. The title of the root window is set to "COVRI Training"and the size of the window is set. We can input text to form a header for the GUI. This is done via the ‘header’ variable. Our header is set to "Available courses".

Once the widow is available we create our widgets which includes list boxes, labels and buttons. As per requirement the following information is needed to store a record in the database:

Select Course
Duration(months)
Want Projects(Y/N) ?

Accordingly we'll set the labels for the above mentioned information:

L1 = Label(window, text = "Select Course", font=("arial", 18)).place(x=10,y=100)
L2 = Label(window, text = "Duration(months)", font=("arial",18)).place(x=10,y=150)
L3 = Label(window, text = "Want Projects(Y/N) ?", font=("arial",18)).place(x=10,y=200)

As we have already discussed about labels I'm not covering the details but one thing to notice is that the place() method is used to set the labels in the window.

In COVRI training I have included 'Python', 'Java', 'PHP', and 'Javascript' as the languages to be taught. I'll create a dictionary to store these language options:

planguages = {'Python', 'Java', 'PHP','Javascript'}

As I prefer the user to select a language from a drop down, I create the drop down list using the ‘OptionMenu’ function. But we need a variable for the languages which will be used in the OptionMenu function thus we define it first:

lvar = StringVar(window)
lvar .set('----')

The arguments for the OptionMenu function are the root window, the variable for the languages (set above) and the dictionary planguages :

langd= OptionMenu(window, comp, *compound)

The option menu is placed next to the Select course label:

langd.place(x=220,y=105)

Now we will set the text boxes so that users can input the relevant data. Tkinter provides the ‘Entry’ function to place text boxes into the GUI. First we declare two variables:

duration = StringVar(window)
wpro = StringVar(window)

Then use them in the Entry widget.

durationT = Entry(window, textvariable=duration)
durationT.place(x=220,y=155)

wproT = Entry(window, textvariable=wpro)
wproT.place(x=220,y=205)

These widgets are placed next to their respective labels in the GUI. So far we have developed our application to accept user input. Now the next challenge is to store the date in to a database from where it can be retrieved as and when required.

We'll begin with created a database using:

con = sq.connect('mycourses.db')

This statement creates a database if it doesn't exists and then establishes a connection to it. Next we create a cursor ( details we have covered in the previous posts)

c = con.cursor()

The cursor will allow us to call on the ‘execute’ method, which will allow for SQL commands. The execute method will allow for a table to be created based off the selected compound lift in the drop down menu.To get the text from the entry boxes and store in the database we define a get():

def get():

       
    c.execute('CREATE TABLE IF NOT EXISTS ' +lvar .get()+ ' (Duration INTEGER, PROJECTS TEXT)') #SQL syntax
    c.execute('INSERT INTO ' +lvar .get()+ ' (Duration, PROJECTS) VALUES (?, ?)',(duration.get(), wpro.get())) #Insert record into database.
    con.commit()


Next we create a submit button which will use this get() and allows user to store his entry in to the database:

button_1 = Button(window, text="Submit",command=get)
button_1.place(x=100,y=300)

To reset the entry fields after the submit button is clicked add this to the get()

   lvar .set('----')
     
    duration.set('')
    wpro.set('')

We also prefer to keep a clear button in case the user wants to change/edit his selection. The functionality of this button will be implemented in a newly defined clear():

def clear():
    lvar .set('----')
    compdb.set('----')
 
    duration.set('')
    wpro.set('')

The clear button can be created as shown below:

button_2 = Button(window,text= "Clear",command=clear)
button_2.place(x=10,y=300)

The application now can get input from a user and store in the database. Next we plan a View button which when clicked allows us to view the entries related to our programming languages.

We create a record function which implements the functionality of the view button. We first create another variable for the second dropdown list:

langdb = StringVar(window)

langdb .set('----')

def record():

    c.execute('SELECT * FROM ' +langdb .get()) #Select from which ever language is selected

    frame = Frame(window)

    frame.place(x= 400, y = 150)

    

    Lb = Listbox(frame, height = 8, width = 25,font=("arial", 12)) 

    Lb.pack(side = LEFT, fill = Y)

    

    scroll = Scrollbar(frame, orient = VERTICAL)

    scroll.config(command = Lb.yview)

    scroll.pack(side = RIGHT, fill = Y)

    Lb.config(yscrollcommand = scroll.set) 

    

    Lb.insert(0, 'Duration, Projects') 

    

    data = c.fetchall()

    

    for row in data:

        Lb.insert(1,row) # Inserts record row by row in list box

    L4 = Label(window, text = langdb .get()+ '      ', 

               font=("arial", 16)).place(x=400,y=100)

    L5 = Label(window, text = "Details for selected course", 

               font=("arial", 16)).place(x=400,y=350)

    con.commit()

The view button can be created as shown below:

button_3 = Button(window,text="View",command=record)
button_3.place(x=10,y=350)

To display the output for the view button we have created a frame. Within this frame a list box with scroll bar is created. In this list we first enter the Duration and Project labels by using:

Lb.insert(0, 'Duration  Projects')

Then we fetch the data from the table using:

data = c.fetchall()

Then using a for loop we insert the data in the list box:

 for row in data:
        Lb.insert(1,row)

It's better to display the name of  selected language above the list box for this we create another label and place above the list box:

L4 = Label(window, text = compdb.get()+ '      ',
               font=("arial", 16)).place(x=400,y=100)

Similarly we display a message "Details for selected course" below the list box with the help of a new label created as shown below:

L5 = Label(window, text = "Details for selected course",
               font=("arial", 16)).place(x=400,y=350)

Remember to keep the created window open using the window.mainloop().

This application is a good exercise to use whatever we've covered so far for tkinter and sqlite. Once the program is completed it should give the following output:



The complete functionality is shown in the following screenshots:



Enter a few more records for Java and submit to database. Then view the records as shown:



Once view is clicked the output changes to:



Here today's topic comes to end. Hopefully you should be able to develop this simple application and test how far you have mastered tkinter and sqlite. This application is built just to explain how to use tkinter with sqlite database and can be made much better.

Till we meet next, keep practicing and learning Python as Python is easy to learn!

Read More