Standard modules

Python comes with a library of over 200 standard modules. The exact number varies from one distribution to the other. These modules can be imported into your program. The list of these modules is very extensive but only a few commonly used modules are mentioned here as an example of standard modules:

• math: This module provides mathematical functions for arithmetic operations.

• random: This module is helpful to generate pseudo-random numbers using different types of distributions.

• statistics: This module offers statistics functions such as mean, median, and variance.

• base64: This module provides functions to encode and decode data.

• calendar: This module offers functions related to the calendar, which is helpful for calendar-based computations.

• collections: This module contains specialized container data types other than the general-purpose built-in containers (such as dict, list, or set). These specialized data types include deque, Counter, and ChainMap.

• csv: This module helps in reading from and writing to comma-based delimited files.

• datetime: This module offers general-purpose data and time functions.

• decimal: This module is specific for decimal-based arithmetic operations.

• logging: This module is used to facilitate logging into your application.

• os and os.path: These modules are used to access operating system-related functions.

• socket: This module provides low-level functions for socket-based network communication.

• sys: This module provides access to a Python interpreter for low-level variables and functions.

• time: This module offers time-related functions such as converting to different time units.

Read More

Loading and initializing a module

Whenever the Python interpreter interacts with an import or equivalent statement, it does three operations, which are described in the next sections.

Loading a module

The Python interpreter searches for the specified module on a sys.path (to be discussed in the Accessing packages from any location section) and loads the source code. This has been explained in the Learning how import works section.

Setting special variables

In this step, the Python interpreter defines a few special variables, such as __name__, which basically defines the namespace that a Python module is running in. The __name__ variable is one of the most important variables.

In the case of our example of the calcmain1.py, mycalculator.py, and myrandom.py modules, the __name__ variable will be set for each module as follows:

There are two cases of setting the __name__ variable, which are described next.

Case A – module as the main program

If you are running your module as the main program, the __name__ variable will be set to the __main__ value regardless of whatever the name of the Python file or module is. For example, when calcmain1.py is executed, the interpreter will assign the hardcoded __main__ string to the __name__ variable. If we run myrandom.py or mycalculator.py as the main program, the __name__ variable will automatically

get the value of __main__.

Therefore, we added an if __name__ == '__main__' line to all main scripts to check whether this is the main execution program.

Case B – module is imported by another module

In this case, your module is not the main program, but it is imported by another module. In our example, myrandom and mycalculator are imported in calcmain1.py. As soon as the Python interpreter finds the myrandom.py and mycalculator.py files, it will assign the myrandom and mycalculator names from the import statement to the __name__ variable for each module. This assignment is done prior to executing the code inside these modules. This is reflected in Table shown above.

Executing the code

After the special variables are set, the Python interpreter executes the code in the file line by line. It is important to know that functions (and the code under the classes) are not executed unless they are not called by other lines of code. Here is a quick analysis of the three modules from the execution point of view when calcmain1.py is run:

• mycalculator.py: After setting the special variables, there is no code to be executed in this module at the initialization time.

• myrandom.py: After setting the special variables and the import statement, there is no further code to be executed in this module at initialization time.

• calcmain1.py: After setting the special variables and executing the import statements, it executes the following if statement: if __name__ == "__main__":. This will return true because we launched the calcmain1.py file.

Inside the if statement, the my_main () function will be called, which in fact then calls methods from the myrandom.py and mycalculator.py modules.

We can add an if __name__ == "__main__" statement to any module regardless of whether it is the main program or not. The advantage of using this approach is that the module can be used both as a module or as a main program. There is also another application of using this approach, which is to add unit tests within the module. Some of the other noticeable special variables are as follows:

• __file__: This variable contains the path to the module that is currently being imported.

• __doc__: This variable will output the docstring that is added in a class or a method. A docstring is a comment line added right after the class or method definition.

• __package__: This is used to indicate whether the module is a package or not. Its value can be a package name, an empty string, or none.

• __dict__: This will return all attributes of a class instance as a dictionary.

• dir: This is actually a method that returns every associated method or attribute as a list.

• Locals and globals: These are also used as methods that display the local and global variables as dictionary entries.

Read More

Absolute versus relative import

We have fairly a good idea of how to use import statements. Now it is time to understand absolute and relative imports, especially when we are importing custom or project-specific modules. To illustrate the two concepts, let's take an example of a project with different packages, sub-packages, and modules, as shown next:

project

├── pkg1

│ ├── module1.py

│ └── module2.py (contains a function called func1 ())

└── pkg2

├── __init__.py

├── module3.py

└── sub_pkg1

└── module6.py (contains a function called func2 ())

├── pkg3

│ ├── module4.py

│ ├── module5.py

└── sub_pkg2

└── module7.py

Using this project structure, we will discuss how to use absolute and relative imports.

Absolute import

We can use absolute paths starting from the top-level package and drilling down to the sub-package and module level. A few examples of importing different modules are shown here:

from pkg1 import module1

from pkg1.module2 import func1

from pkg2 import module3

from pkg2.sub_pkg1.module6 import func2

from pkg3 import module4, module5

from pkg3.sub_pkg2 import module7

For absolute import statements, we must give a detailed path for each package or file, from the top-level package folder, which is similar to a file path.

Absolute imports are recommended because they are easy to read and easy to follow the exact location of imported resources. Absolute imports are least impacted by project sharing and changes in the current location of import statements. In fact, PEP 8 explicitly recommends the use of absolute imports.

Sometimes, however, absolute imports are quite long statements depending on the size of the project folder structure, which is not convenient to maintain.

Relative import

A relative import specifies the resource to be imported relative to the current location, which is mainly the current location of the Python code file where the import statement is used.

For the project examples discussed earlier, here are a few scenarios of relative import. The equivalent relative import statements are as follows:

• Scenario 1: Importing funct1 inside module1.py:

from .module2 import func1

We used one dot (.) only because module2.py is in the same folder as module1.py.

• Scenario 2: Importing module4 inside module1.py:

from ..pkg3 import module4

In this case, we used two dots (..) because module4.py is in the sibling folder of module1.py.

• Scenario 3: Importing Func2 inside module1.py:

from ..pkg2.sub_pkg_1.module2 import Func2

For this scenario, we used two dots (..) because the target module (module2.py) is inside a folder that is in the sibling folder of module1.py. We used one dot to access the sub_pkg_1 package and another dot to access module2.

One advantage of relative imports is that they are simple and can significantly reduce long import statements. But relative import statements can be messy and difficult to maintain when projects are shared across teams and organizations. Relative imports are not easy to read and manage.

Read More

Using the __import__ statement and importlib.import_module statement

The __import__ statement is a low-level function in Python that takes a string as input and triggers the actual import operation. Low-level functions are part of the core Python language and are typically meant to be used for library development or for accessing operating system resources, and are not commonly used for application development.

We can use this keyword to import the random library in our myrandom.py module as follows:

#import random

random = __import__('random')

The rest of the code in myrandom.py can be used as it is without any change. I have illustrated a simple case of using the __import__ method for academic reasons and will skip the advanced details for those of you who are interested in exploring as further reading. The reason for this is that the __import__ method is not recommended to be used for user applications; it is designed more for interpreters.

The importlib.import_module statement is the one to be used other than the regular import for advanced functionality.

We can import any module using the importlib library. The importlib library offers a variety of functions, including __import__, related to importing modules in a more flexible way. Here is a simple example of how to import a random module in our myrandom.py module using importlib:

import importlib

random = importlib.import_module('random')

The rest of the code in myrandom.py can be used as it is without any change. The importlib module is best known for importing modules dynamically and is very useful in cases where the name of the module is not known in advance and we need to import the modules at runtime. This is a common requirement for the development of plugins and extensions.

Commonly used functions available in the importlib module are as follows:

• __import__: This is the implementation of the __import__ function, as already discussed.

• import_module: This is used to import a module and is most commonly used to load a module dynamically. In this method, you can specify whether you want to import a module using an absolute or relative path. The import_module function is a wrapper around importlib.__import__. Note that the former function brings back the package or module (for example, packageA.module1), which is specified with the function, while the latter function always returns the top-level package or module (for example, packageA).

• importlib.util.find_spec: This is a replaced method for the find_loader method, which is deprecated since Python release 3.4. This method can be used to validate whether the module exists and it is valid.

• invalidate_caches: This method can be used to invalidate the internal caches of finders stored at sys.meta_path. The internal cache is useful to load the module faster without triggering the finder methods again. But if we are dynamically importing a module, especially if it is created after the interpreter began execution, it is a best practice to call the invalidate_caches method.

This function will clear all modules or libraries from the cache to make sure the requested module is loaded from the system path by the import system.

• reload: As the name suggests, this function is used to reload a previously imported module. We need to provide the module object as an input parameter for this function. This means the import function has to be done successfully.

This function is very helpful when module source code is expected to be edited or changed and you want to load the new version without restarting the program.

Read More

Specific import

We can also import something specific (variable or function or class) from a module instead of importing the whole module. This is achieved using the from statement, such as the following:

from math import pi

Another best practice is to use a different name for an imported module for convenience or sometimes when the same names are being used for different resources in two different libraries. To illustrate this idea, we will be updating our calcmain1.py file (the updated program is calcmain2.py) from the earlier example by using the calc and rand aliases for the mycalculator and myrandom modules, respectively. This change will

make use of the modules in the main script much simpler, as shown next:

# calcmain2.py with alias for modules

import mycalculator as calc

import myrandom as rand

def my_main():

""" This is a main function which generates two random\

numbers and then apply calculator functions on them """

x = rand.random_2d()

y = rand.random_1d()

sum = calc.add(x,y)

diff = calc.subtract(x,y)

print("x = {}, y = {}".format(x,y))

print("sum is {}".format(sum))

print("diff is {}".format(diff))

""" This is executed only if the special variable '__name__' is

set as main"""

if __name__ == "__main__":

my_main()

As a next step, we will combine the two concepts discussed earlier in the next iteration of the calcmain1.py program (the updated program is calcmain3.py). In this update, we will use the from statement with the module names and then import the individual functions from each module. In the case of the add and subtract functions, we used the as statement to define a different local definition of the module resource for illustration purposes.

A code snippet of calcmain3.py is as follows:

# calcmain3.py with from and alias combined

from mycalculator import add as my_add

from mycalculator import subtract as my_subtract

from myrandom import random_2d, random_1d

def my_main():

""" This is a main function which generates two random

numbers and then apply calculator functions on them """

x = random_2d()

y = random_1d()

sum = my_add(x,y)

diff = my_subtract(x,y)

print("x = {}, y = {}".format(x,y))

print("sum is {}".format(sum))

print("diff is {}".format(diff))

print (globals())

""" This is executed only if the special variable '__name__' is

set as main"""

if __name__ == "__main__":

my_main()

As we used the print (globals()) statement with this program, the console output of this program will show that the variables corresponding to each function are created as per our alias. The sample console output is as follows:

{

"__name__":"__main__",

"__doc__":"None",

"__package__":"None",

"__loader__":"<_frozen_importlib_external.\

SourceFileLoader object at 0x1095f1208>",

"__spec__":"None",

"__annotations__":{},

"__builtins__":"<module 'builtins' (built-in)>", "__

file__":"/PythonForGeeks/source_code/chapter2/module1/

main_2.py",

"__cached__":"None",

"my_add":"<function add at 0x109645400>",

"my_subtract":"<function subtract at 0x109645598>",

"random_2d":"<function random_2d at 0x10967a840>",

"random_1d":"<function random_1d at 0x1096456a8>",

"my_main":"<function my_main at 0x109645378>"

}

Note that the variables in bold correspond to the changes we made in the import statements in the calcmain3.py file.

Read More

Using the import statement

The import statement is a common way to import a module. The next code snippet is an example of using an import statement:

import math

The import statement is responsible for two operations: first, it searches for the module given after the import keyword, and then it binds the results of that search to a variable name (which is the same as the module name) in the local scope of the execution. In the next two subsections, we will discuss how import works and also how to import specific elements from a module or a package.

Learning how import works

Next, we need to understand how the import statement works. First, we need to remind ourselves that all global variables and functions are added to the global namespace by the Python interpreter at the start of an execution. To illustrate the concept, we can write a small Python program to spit out of the contents of the globals namespace, as shown next:

# globalmain.py with globals() function

def print_globals():

print (globals())

def hello():

print ("Hello")

if __name__ == "__main__":

print_globals()

This program has two functions: print_globals and hello. The print_globals function will spit out the contents of the global namespace. The hello function will not be executed and is added here to show its reference in the console output of the global namespace. The console output after executing this Python code will be similar to the following:

{

"__name__":"__main__",

"__doc__":"None",

"__package__":"None",

"__loader__":"<_frozen_importlib_external.\

SourceFileLoader object at 0x101670208>",

"__spec__":"None",

"__annotations__":{

},

"__builtins__":"<module 'builtins' (built-in)>",

"__file__":"/ PythonForGeeks/source_code/chapter2/\

modules/globalmain.py",

"__cached__":"None",

"print_globals":"<function print_globals at \

0x1016c4378>",

"hello":"<function hello at 0x1016c4400>"

}

The key points to be noticed in this console output are as follows:

• The __name__ variable is set to the __main__ value. This will be discussed in more detail in the Loading and initializing a module post.

• The __file__ variable is set to the file path of the main module here.

• A reference to each function is added at the end.

If we add print(globals()) to our calcmain1.py script, the console output after adding this statement will be similar to the following:

{

"__name__":"__main__",

"__doc__":"None",

"__package__":"None",

"__loader__":"<_frozen_importlib_external.\

SourceFileLoader object at 0x100de1208>",

"__spec__":"None",

"__annotations__":{},

"__builtins__":"<module 'builtins' (built-in)>",

"__file__":"/PythonForGeeks/source_code/chapter2/module1/

main.py",

"__cached__":"None",

"mycalculator":"<module 'mycalculator' from \

'/PythonForGeeks/source_code/chapter2/modules/\

mycalculator.py'>",

"myrandom":"<module 'myrandom' from '/PythonForGeeks/source_

code/chapter2/modules/myrandom.py'>",

"my_main":"<function my_main at 0x100e351e0>"

}

An important point to note is that there are two additional variables (mycalculator and myrandom) added to the global namespace corresponding to each import statement used to import these modules. Every time we import a library, a variable with the same name is created, which holds a reference to the module just like a variable for the global functions (my_main in this case).

We will see, in other approaches of importing modules, that we can explicitly define some of these variables for each module. The import statement does this automatically for us.

Read More

Modularization to Handle Complex Projects

When you start programming in Python, it is very tempting to put all your program code in a single file. There is no problem in defining functions and classes in the same file where your main program is. This option is attractive to beginners because of the ease of execution of the program and to avoid managing code in multiple files. But a single-file program approach is not scalable for medium- to large-size projects. It becomes challenging to keep track of all the various functions and classes that you define.

To overcome the situation, modular programming is the way to go for medium to large projects. Modularity is a key tool to reduce the complexity of a project. Modularization also facilitates efficient programming, easy debugging and management, collaboration, and reusability.

We will cover the following topics in this and future blogs:

• Introduction to modules and packages

• Importing modules

• Loading and initializing a module

• Writing reusable modules

• Building packages

• Accessing packages from any location

• Sharing a package

Let us start with introduction to modules and packages.

Modules in Python are Python files with a .py extension. In reality, they are a way to organize functions, classes, and variables using one or more Python files such that they are easy to manage, reuse across the different modules, and extend as the programs become complex.

A Python package is the next level of modular programming. A package is like a folder for organizing multiple modules or sub-packages, which is fundamental for sharing the modules for reusability. Python source files that use only the standard libraries are easy to share and easy to distribute using email, GitHub, and shared drives, with the only caveat being that there should be Python version compatibility. But this sharing approach will not scale for projects that have a decent number of files and have dependencies on third-party libraries and may be developed for a specific version of Python. To rescue the situation, building and sharing packages is a must for efficient sharing and reusability of Python programs.

Next, we will discuss how to import modules and the different types of import techniques supported in Python.

Importing modules

Python code in one module can get access to the Python code in another module by a process called importing modules.

To elaborate on the different module and package concepts, we will build two modules and one main script that will use those two modules. These two modules will be updated or reused throughout this blog series.

To create a new module, we will create a .py file with the name of the module. We will create a mycalculator.py file with two functions: add and subtract. The add function computes the sum of the two numbers provided to the function as arguments and returns the computed value. The subtract function computes the difference between the two numbers provided to the function as arguments and returns the computed value.

A code snippet of mycalculator.py is shown next:

# mycalculator.py with add and subtract functions

def add(x, y):

"""This function adds two numbers"""

return x + y

def subtract(x, y):

"""This function subtracts two numbers"""

return x - y

Note that the name of the module is the name of the file.

We will create a second module by adding a new file with the name myrandom.py. This module has two functions: random_1d and random_2d. The random_1d function is for generating a random number between 1 and 9 and the random_2d function is for generating a random number between 10 and 99. Note that this module is also using the random library, which is a built-in module from Python.

The code snippet of myrandom.py is shown next:

# myrandom.py with default and custom random functions

import random

def random_1d():

"""This function generates a random number between 0 \

and 9"""

return random.randint (0,9)

def random_2d():

"""This function generates a random number between 10 \

and 99"""

return random.randint (10,99)

To consume these two modules, we also created the main Python script (calcmain1.py), which imports the two modules and uses them to achieve these two calculator functions. The import statement is the most common way to import built-in or custom modules.

A code snippet of calcmain1.py is shown next:

# calcmain1.py with a main function

import mycalculator

import myrandom

def my_main( ):

""" This is a main function which generates two random\

numbers and then apply calculator functions on them """

x = myrandom.random_2d( )

y = myrandom.random_1d( )

sum = mycalculator.add(x, y)

diff = mycalculator.subtract(x, y)

print("x = {}, y = {}".format(x, y))

print("sum is {}".format(sum))

print("diff is {}".format(diff))

""" This is executed only if the special variable '__name__'

is set as main"""

if __name__ == "__main__":

my_main()

In this main script (another module), we import the two modules using the import statement. We defined the main function (my_main), which will be executed only if this script or the calcmain1 module is executed as the main program. The details of executing the main function from the main program will be covered later in the Setting special variables section. In the my_main function, we are generating two random numbers using the myrandom module and then calculating the sum and difference of the two random numbers using the mycalculator module. In the end, we are sending the results to the console using the print statement.

There are other options available to import a module, such as importlib.import_module() and the built-in __import__() function. Let's discuss how import and other alternative options works, in the next blog. Meanwhile keep practicing and experimenting.

Read More

Memoization

 Basically in memoization we maintain a look up table where solutions are

stored so that we don’t have to solve the same sub problem again and again.

Instead we solve it once, and store the values so that they can be reused.

We know that Fibonacci sequence is:

F(n) = F(n-1)+F(n-2) if n>1

= n if n =0,1

So,

F(n):

if n<1:

return n

else :

return F(n-1)+F(n-2)

Here, we are making two recursive calls, and adding them up, and the value is returned.

Look at the following diagram:

Observe that just to find fibonacci(5), fibonacci(2) is computed three times and fibonacci(3) is computed two times. So, as n increases, fibonacci function’s(f(n)) performance goes down. The consumption of time and space would increase exponentially with increase in n. We can save time, by following a simple approach, that is to save a value when it is computed for the first time. So, we can save the values of F(1), F(2), F(3) and F(4) when they are computed for the first time, same way with F(3), F(4)…so on. So, we can say that:

F(n):

if n=<1:

return n

elif F(n) exist :

return F(n-1)

else:

F(n) = F(n-1)+F(n-2)

Save F(n)

Return F(n)

In the following code:

1. The function fibonacci() takes a number and creates a list, fib_num of size num+1. This is because the Fibonnaci series start from 0.

2. It calls the function fib_calculate(), which takes the number num and list fib_num as a parameter.

3. We have saved -1 at all index in the list:

a. If fib_num[num] is >0, that means Fibonacci for this number already exists, and we need not compute it again, and the number can be returned.

b. If num<= 1, then return num.

c. Else if num>=2, calculate fib_calculate(num - 1, fib_num) + fib_calculate(num - 2, fib_num). The value calculated must be stored in list fib_num at index num so that there is no need to calculate it again.

Code:

def fibonacci(num):

fib_num = [-1]*(num + 1)

return fib_calculate(num, fib_num)

def fib_calculate(num, fib_num):

if fib_num[num] >= 0:

return fib_num[num]

if (num<= 1):

fnum = num

return fnum

else:

fnum = fib_calculate(num - 1, fib_num) + fib_calculate(num - 2, fib_num)

fib_num[num] = fnum

return fnum

num = int(input('Enter the number: '))

print("Answer = ",fibonacci(num))

Execution:

num = int(input('Enter the number: '))

print("Answer = ",fibonacci(num))

Output:

Enter the number: 15

Answer = 610

Read More

Types of recursion

 Recursion can be classified as direct recursion or indirect recursion. If a function makes a call to itself, then it is an example of direct recursion.

For example: in the factorial function, the find_factorial() function made a call to itself. Therefore, we can call it an example of direct recursion. However, sometimes there are scenarios where a function f() may call another function f1(), which in return makes a call back to function f(). This is known as Indirect Recursion. As an example, have a look at the following code:

def happy_new_year(n=1):

if(n<=0):

how_many_times()

for i in range(n):

print("Happy New Year")

def how_many_times():

val = input("How many times should I print?:")

if val=='':

happy_new_year()

else:

happy_new_year(int(val))

how_many_times()

In the preceding example, there is a function called happy_new_year() that prints the “Happy New Year” message as many number of times as you want. If no value is provided, then it takes the default value as 1, and prints the message once. If the value is invalid, that is, 0 or below, if calls a function how_many_times() which prompts the user again to provide another value and makes a call back to happy_new_year() with the new value. Have a look at the following figure:

Let’s have a look at some interesting output for this code:

Case 1: Call is made to happy_new_year() with no input value.

Output:

Happy New Year

Explanation: Since no value was provided the message got printed only once as that is the default value.

Case 2: Callis made to happy_new_year()for n=0.

Output: Since n =0, how_many_times()is called again till a value of n greater than 0 is provided.

How many times should I print?:0

How many times should I print?:0

How many times should I print?:7

Happy New Year

Happy New Year

Happy New Year

Happy New Year

Happy New Year

Happy New Year

Happy New Year

>>>

Explanation: The user provided n = 0 twice as a result of which the function happy_new_year() called how_many_times() back each time till the user decided to provide a value greater than 0 which in this case is 7. So, for n <=0, the function prompts the user to provide another value, and the function happy_new_year() is called again with the new input value.

Case 2: Call made with invalid values again and again.

Output:

happy_new_year(-2)

How many times should I print?:-3

How many times should I print?:-8

How many times should I print?:0

How many times should I print?:2

Happy New Year

Happy New Year

Explanation: An invalid value is passed on to happy_new_year() function, which then calls the how_many_times() function which prompts users to provide a valid number. However, the user continues to provide invalid values so both the functions keep on calling each other till a valid value is provided.

Finally let’s have a look at advantages and disadvantages of recursion:

Advantages of recursion

• Requires fewer lines of code. The code looks clean.
• Allows you to break a complex task into simpler tasks.

Disadvantages of recursion

• Forming the logic for recursion can sometimes be difficult.
• Debugging a recursive function can be difficult.
• Recursive functions consume more memory and time.

Read More

Recursion

Recursion in programming can be applied to solve a problem whose solution depends on the solutions to smaller instances of the same problem. We can therefore say that Recursion refers to a function, which can calculate the right answer by first solving a smaller version of its own self and then using that result along with some more computation to get the final answer.

Let’s have a look at our example of finding the factorial of a number. A factorial of a number is the product of that number and all positive integers below it. So, factorial of 5 or 5! can also be represented as follows:

5! = 5 * 4 * 3 * 2 * 1 …….(1)

Now, look at the preceding statement carefully. Since a factorial of a number is product of that number and all positive numbers below it we can say that 4 * 3

* 2 * 1 = 4!. Hence, statement (1) can be rewritten as:

5! = 5 * 4 * 3* 2 * 1 = 5 * 4! ……(2)

Following the definition of factorial, we can now write (2) as:

5! = 5 * 4 * 3 * 2 * 1 =5 * 4 * 3! ……(3)

Similarly,

5! = 5 * 4 * 3 * 2 * 1 =5 * 4 * 3 * 2! …….(4)

So, if we define a method find_factorial(n) to find factorial of number n, it would mean that find_factorial(n) is same as n * find_factorial(n-1) and find_factorial(n-1) is same as (n-1) * find_factorial(n-2), and so on.

Therefore, as we start writing the code for recursion, the following may seem like the right way to start. However, the following code is not complete yet.

def find_factorial(n):

return n*find_factorial(n-1)

At this point, it is important to understand base case or terminal case. Every problem in recursion has a base case. It is one or more special values for which a function can be evaluated without recursion or it is that part of the recursion problem, which cannot be defined in smaller instances of itself. Without a base case a recursive function will never stop executing. A recursive function makes calls to itself, which step by step takes the function to the base case, and the function then stops executing.

So, the preceding code is not complete because it has no base case. As per the definition of factorial, we say that factorial of a number is the product of itself with all the positive numbers below it. This means that the last number to be multiplied is always 1. Thus, the function should stop executing when a call is made to it with value n= 1. This will be the base case for our find_factorial() function.

def find_factorial(n):

if(n==1):

return 1

return n*find_factorial(n-1)

print (find_factorial (5))

So the Algorithm for finding factorial using recursion will be -

Function find_factorial(n):

Step 1: Read the number n provided for finding factorial.

Step 2: Check whether the value provided is equal to 1. If true, return 1.

Step 3: Else return value of n*find_factorial(n-1).

Read More

More on Lambda functions

The answer to the assignment from previous post is

addFunc = lambda a,b: a+b

print(addFunc(5,2))

Now we have a list of countries as:

countries = ['India','Mauritius','France','Turkey','Kenya','Hungary']

Your task is to use lambda function to print the length of each string in the list countries. This is how you can do this -

print(list(map(lambda x: len(x),countries)))

Now let us see how to use sort() with lambda. The syntax for sort is as follows:

list.sort(key = None, reverse= False)

The sort method uses the process of comparing the items to sort the elements of a list. The lambda functions allow key to become more versatile.

Suppose we have the list of country names: 

countries = ['India','Mauritius','France','Turkey','Kenya','Hungary']

Now, if we want to sort these names alphabetically, we can use the procedure given as follows:

countries = ['India','Mauritius','France','Turkey','Kenya','Hungary']

countries.sort()

print(countries)

Or we can use the sort() function given as follows:

countries.sort(key = lambda x:x[0])

print(countries)

So basically, the lambda functions takes each element(x) and sorts with respect to first element of string.

Suppose we have a list of names, and we want to sort the names on the basis of surnames:

We can write:

>>> names = ['Mahatma Gandhi','Jawaharlal Nehru','Subhash Chandra bose','Rani Laxmi Bai','Chandra Shekhar Azaad','Sarojini Naidu']

>>>names.sort(key = lambda x:x.split()[-1])

>>> print(names)

['Chandra Shekhar Azaad', 'Rani Laxmi Bai', 'Mahatma Gandhi', 'Sarojini Naidu', 'Jawaharlal Nehru', 'Subhash Chandra bose']

Here, lambda x:x.split()[-1], x.spilt() breaks each element into individual words. The names (having two parts) is split into a list of two words, and the names having three parts are split into a list of three words. -1 is the index of the last element in the list. So, lambda x:x.split()[-1] is working on last word of each element in the list names. (Notice, the name ‘Subhash Chandra bose’ is placed at the end because the surname does not begin with capital ‘b’. Let’s change the surname to capital B and see the output.)

>>> names = ['Mahatma Gandhi','Jawaharlal Nehru','Subhash Chandra Bose','Rani Laxmi Bai','Chandra Shekhar Azaad','Sarojini Naidu']

>>> names.sort(key = lambda x:x.split()[-1])

>>> print(names)

['Chandra Shekhar Azaad', 'Rani Laxmi Bai', 'Subhash Chandra Bose', 'Mahatma Gandhi', 'Sarojini Naidu', 'Jawaharlal Nehru']

We can also get the same result using the sorted() function.

>>> names = ['Mahatma Gandhi','Jawaharlal Nehru','Subhash Chandra Bose','Rani Laxmi Bai','Chandra Shekhar Azaad','Sarojini Naidu']

>>> print(list(sorted(names, key = lambda x:x.split()[-1])))

['Chandra Shekhar Azaad', 'Rani Laxmi Bai', 'Subhash Chandra Bose', 'Mahatma Gandhi', 'Sarojini Naidu', 'Jawaharlal Nehru']

The difference between the sort() and the sorted() function is that the sort() function modifies the list whereas the sorted() function provides a new list having the sorted values.

Read More

Lambda functions

Lambda functions are Python’s anonymous functions, that is, they are defined without a name. These functions are defined using lambda keyword instead of def. The syntax for lambda function is as follows:

lambda arguments: expression

The most interesting feature about lambda functions is that it can have any number of arguments but only one expression. The expression is evaluated, and the value is returned. Ideally, lambda functions are used if there is a requirement of function objects.

For example:

your_age = lambda yr_of_birth: 2021 - yr_of_birth

print(your_age(1956))

Output:

65

Three functions map(), filter(), and reduce()were created to facilitate functional approach in Python programming. In Python 3 reduce() has been discontinued. These functions can be replaced by List Comprehensions or loops. The following example demonstrate how to use lambda function with filter().

number = [1,2,3,4,5,6,13,7,8,9,0]

odd_number = list(filter(lambda x : (x%2!=0), number))

print(odd_number)

The above example use lambda function with filter() to print odd numbers from a given list. Similarly, map() function applies the same function to each element of a sequence and returns a modified list.

Suppose we need to square every number in list [1,2,3,4,5], the usual way of doing this would be:

list1 = [1,2,3,4,5]

for element in list1:

print(element**2)

We can use map() along with lambda to produce the same result:

print(list(map(lambda x: x**2,list1)))

Try some more examples by rewriting the normal codes using lambda functions, for example the following code -

def addFunc(a,b):

return a+b

print(addFunc(5,2))

Try to rewrite this using lambda function. Answer will be provided in the next post.

Read More

Scopes and namespace

Namespace is a container that has all the names (of variables/functions/classes) that you define. You can define same names in different namespaces. A name or a variable exists in a specific area of the code which defines its scope. The information regarding binding between the variables/objects is stored in the namespace. There are three types of namespaces or scopes.

1. Built-in Namespace: These are in-built functions that are available across all files or modules.

2. Global Namespace: The global namespace has all the variables, functions, and classes that are available in a single file.

3. Local Namespace: The local namespace are variables defined within a function.

The scopes are nested, which means that the local namespace is nested within a global namespace which is nested within built-in namespace. Each scope has its namespace.

Built-in namespace - Built-in namespace are available across all the files, and module in Python. All functions that you see below print(), tuple(), type() are all built-in function, and belong to this namespace and are available across all files and modules in Python.

>>> list1 = [1,2,3,4,5,6]

>>> tup1 = tuple(list1)

>>> type(tup1)

<class 'tuple'>

>>> print("Hi")

Hi

>>>

Global namespace - Look at the following code:

x = 20

x += y

print(x)

When we execute this code, it generates a Name Error:

Output:

Traceback (most recent call last):

File "F:\2020 - BPB\input.py", line 2, in <module>

x += y

NameError: name 'y' is not defined

>>>

This is because Python looks for the name y in the global namespace, and fails to find it. It then looks for it in the built in namespace, and does not find it again. Hence, an error is generated. The following code works fine and does not produce any error because the statement y = 5 created a global namespace:

x = 20

y = 5

x += y

print(x)

Output:

25

>>>

Local namespace - Now, let’s look at another example.

x = 20

def print_x():

x = 10

print('Local variable x is equal to ',x)

print('Global variable x is equal to ',x)

print_x()

Output:

Global variable x is equal to 20

Local variable x is equal to 10

>>>

When a call is made to the function, the Python interpreter tries to locate the local variable called x. If that is not available, it will look for x at global namespace.

Code:

x = 20

def print_x():

print('Local variable x is equal to ',x)

print('Global variable x is equal to ',x)

print_x()

Output:

Global variable x is equal to 20

Local variable x is equal to 20

>>>

Local variables, that is, the variables within a function are created when a call is made to that function. Whenever a call is made to a function, a new scope is created, and variables are assigned to that scope. Once the function has been executed, its scope is also gone. In the first example, when the function print_x() was called, it was able to find a local variable x = 10 within the local namespace, and used it up. This value of x existed within the function, and vanishes with the function after its execution is over.

Sometimes, when we want to use the global variable inside our function namespace, we should use global keyword for that variable to make it clear that we want to use the global variable only. Look at the following code:

x = 20

def print_x():

global x

x = 10

print('Local variable x is equal to ',x)

print('Global variable x is equal to ',x)

print_x()

print('Global variable x is equal to ',x)

The moment global keyword is used, the function print_x() comes to know that the global variable x will be used. In the next statement x = 10, the value 10 is assigned to x which is a global variable. Therefore, you will see in the output the value of global variable is 20 before the function is called, and 10 after the function is called. When you are using a global keyword with a variable name within a function, Python will not allow you to create another variable with the same name within the same function.

Read More

The return statement

The return keyword is used at the end of the function when there is a need to send back the result of the function back to the caller. Software programming is not about printing results all the time. There are some calculations that must be performed behind the scene, hidden from the users. These values are further used in calculations to get the final output that the users desire. The value obtained from the return statement can be assigned to a variable and used further for calculations.

Look at the code given in the following box. The function adding_numbers() adds three numbers and returns the result which is is assigned to variable x. The value of x is then displayed as output.

def adding_numbers(num1, num2, num3):

print('Have to add three numbers')

print('First Number = {}'.format(num1))

print('Second Number = {}'.format(num2))

print('Third Number = {}'.format(num3))

return num1 + num2 + num3

x = adding_numbers(10, 20, 30)

print('The function returned a value of {}'.format(x))

Output:

Have to add three numbers

First Number = 10

Second Number = 20

Third Number = 30

The function returned a value of 60

Thus we can say that:

1. A return statement exits the function. It is the last statement of a function and any statement coming after that will not be executed.

2. When a function is not returning a value explicitly that means that indirectly or implicitly it is returning a value of None.

3. If a function has to return more than one value, then all the values will be returned as a tuple.

As an exercise make some more programs and use the return statement in them. 

Read More

*args and **kwargs

*args is used when you don’t have any idea about how many arguments you will use. If you don’t know how many parameter you require, then * args is the way to go. Suppose you have decided to go shopping, but you don’t know how many items you are going to buy or how much you are going to spend. So, the expenditure is undecided. You have no idea about how many products you will buy so a function cannot be created with defined number of elements. When using *args, you are using a tuple with a potential of additional arguments. This tuple is initially empty, and no error is generated if no argument is provided.

def sum_func(a, *args):

s = a + sum(args)

print(s)

sum_func(10)

sum_func(10,20)

sum_func(10,20,30)

sum_func(10, 20, 30, 40)

Output:

10

30

60

100

>>>

**kwargs stands for keyworded arguments(of variable length), and is used when you don’t have any idea about how many keyword arguments you would be using. **kwargs builds a dictionary of key value pairs. These types of arguments are often used when working with different external modules and libraries. The double star ‘**’ in **kwargs allows any number of keyworded arguments to pass through. As the name suggests, in keyword argument a name is provided to the variable while passing it to the function similar to dictionary where keywords are associated with values.

def shopping(**kwargs):

print(kwargs)

if kwargs:

print('you bought', kwargs['dress'])

print('you bought', kwargs['food'])

print('you bought', kwargs['Shampoo'])

shopping(dress = 'Frock',Shampoo ='Dove',food = 'Pedigree Puppy')

Output:

{'dress': 'Frock', 'Shampoo': 'Dove', 'food': 'Pedigree Puppy'}

you bought Frock

you bought Pedigree Puppy

you bought Dove

It is important to note that since **kwargs is similar to dictionary, if you try to iterate over it then it may or may not print in the same order. As far as the name is concerned, you can use any name instead of args or kwargs. These names are recommended, but are not mandatory. However, it is important to use * for positional arguments and ** for keyword arguments. This is necessary.

Read More

Functional arguments

Python programmers often use parameters and arguments interchangeably as both are in a way quite similar but as a Python developer, you must understand the difference between the two. Parameters are declared in the function, and arguments are the values passed to a function when it is called. There are five types of functional arguments in Python.

Positional arguments

In positional arguments, that is, arguments are assigned to the parameters in the order in which they are passed or their position.

def sum_prod(num1,num2):

num_sum = num1 + num2

num_prod = num1 * num2

return num_sum,num_prod

x = int(input('Enter the first number :'))

y = int(input('Enter the second number :'))

print(sum_prod(x,y))

Output:

Enter the first number :10

Enter the second number :20

(30, 200)

>>>

The positional argument looks at the position of the parameter where it will be assigned. So, the value of x gets mapped to num1, and value of y gets mapped to num2 and these values are passed on to the function block code.

If you call a function with different number of parameters, an error will be generated.

Default arguments

You have the option of specifying default value of a parameter in the function definition. The positional argument for which a default value is defined, becomes optional and therefore known as default argument.

def sum_prod(num1,num2 =0):

num_sum = num1 + num2

num_prod = num1 * num2

return num_sum,num_prod

print(sum_prod(2,5))

print(sum_prod(2))

Output:

(7, 10)

(2, 0)

>>>

The function shown above can be called with one or two arguments. If you omit the second argument, the function definition will pass on its default value, which is 0.

In Python, a non-default argument cannot follow a default argument.

def sum_func(num1,num2 =0,num3):

return num1+num2+num3

The preceding function will throw an error because num3 which is a nondefault argument follows num2, which is a default argument. So, if you type sum_func(10,20), the interpreter will not understand whether to assign 20 to num2 or continue with the default value. The complexity will increase as the number of default arguments increase. In this scenario you will receive a Syntax Error: "non default argument follow default argument". The correct way of using default arguments is shown in the following code:

def sum_func(num1,num2 =30,num3=40):

return num1 + num2 + num3

print(sum_func(10))

print(sum_func(10,20))

print(sum_func(10,20,30))

Output:

80

70

60

Keyword arguments

Keyword arguments allow you to ignore the order in which the parameters are entered in a function or even skip them when calling a function.. The function with keyword arguments are defined the same way as the function with positional arguments, but the difference is in the way they are called. Have a look at the following code:

def sum_func(num1,num2 =30,num3=40):

print("num1 = ", num1)

print("num2 = ", num2)

print("num3 = ", num3)

return num1 + num2 + num3

print(sum_func(num3 = 10, num1 =20))

Output:

num1 = 20

num2 = 30

num3 = 10

60

As you can see, the arguments are not passed in the desired order, but while passing the arguments, it is specified which argument belongs to which parameter. Since the default value of num2 is zero, even if it is skipped, it does not matter. We wanted to use the default value of num2 therefore only the value of num1 and num3 were specified. If that is not done, the output will be incorrect. As you can see in the following code, the value 20 is assigned to num2 and default value of num3 is taken as a result of which the result is completely different.

def sum_func(num1,num2 =30,num3=40):

print("num1 = ", num1)

print("num2 = ", num2)

print("num3 = ", num3)

return num1 + num2 + num3

print(sum_func(10, 20))

Output:

num1 = 10

num2 = 20

num3 = 40

70

The remaining *args and **kwargs arguments we will see in the next post.

Read More

Functions and Recursion in Python

A function is block of reusable code that is defined to carry out one particular task. Once you understand the concept of function, you will learn about scenarios where a problem can be solved by a function making call to itself. This is known as Recursion.

Functions form the most important aspect of programming in Python and is the core topic for any object-oriented programming language. Functions are important because:

1. They provide better readability and modularity*.
2. Functions help save time and effort in designing and executing the code.
3. Functions reduces duplication of code.
4. They make code reusable.
5. They Make code easy to maintain.
6. With functions it becomes easier to understand how the code works.
7. Functions help in information hiding.

In Python programming, functions can be classified into two types:

Built-in functions

Built-in functions are functions that are provided by Python. They are readily available. All the functions that we have worked with till now are all built-in functions such as max(), min(), len(), and so on.

User-defined functions

This chapter is actually all about user-defined functions. These functions are not provided by Python, but are created by programmers to perform a particular task.

Now let us see how to create functions. We know that Functions make code reusable. If there is a block of code that you need to execute, again and again, you can place that block of code inside a function, and call that function whenever you need to execute that specific task. There is a software development practice by the acronym ‘DRY’ that stands for “Don’t Repeat Yourself”. Functions help in keeping the code DRY. This is opposite to another coding acronym called WET, which stands for “Write Everything Twice”. Now we will create our first function.

For creating a function, we will have to follow certain rules regarding:

How to define a function

1. The definition of a function starts with the ‘def’ keyword.

2. The def keyword is followed by the name of the function.

3. The name of the function is followed by parenthesis ().

4. After the parenthesis comes the colon : which marks the beginning of the function’s block of code.

def addNumbers():

The function body

The block of code that comes after the function definition should be indented one level to the right, which is four spaces as per PEP-8.

def addNumbers(a,b):

    c=a+b

    print(c)

Calling a function

You can call the function anytime by its name followed by parenthesis.

addNumbers(a,b):

The function does not execute as long it is called explicitly.

def addNumbers(a,b):

print('value after addition is')

addNumbers(10,20)

The function has only one print statement. There are some points, which are not mandatory but are best practices to follow:

1. Use lowercase letters for function name, words can be separated by underscore. Some also prefer to use camel case.

2. It is recommended to have a docstring as the first part of the function. The docstring must emphasize on what the function does, and not on how it does it.

3. Place the code after docstring.

The output of the code is as follows:

value after addition is 30

Read More

Exploring machine learning software

Before we start developing models, we need to few tools to help us. Regardless of whether you are using a Mac, PC, or Linux, almost everything we use is compatible with all platforms. There are three main items we need to install: a language to develop our models in, a database to store our data in, and a cloud computing space to deploy our models in. There is a fantastic technology stack ready to support these needs. We can use the Python programming language to develop our models, MySQL to store our data, and AWS to run our cloud computing processes. Let's take a closer look at these three items.

Python (programming language)

Python is one of the most commonly used programming languages and sought-after skills in the data science industry today. There are several ways you can install Python on your computer. You can install the language in its standalone form from Python.org. This will provide you with a Python interpreter in its most basic form where you can run commands and execute scripts. An alternative installation process that would install Python, pip (a package to help you install and manage Python libraries), and a collection of other useful libraries can be done by using Anaconda, which can be retrieved from anaconda.com. To have a working version of Python and its associated libraries on your computer as quickly as possible, using Anaconda is highly recommended. In addition to Python, we will need to install libraries to assist in a few areas. Think of libraries as nicely packaged portions of code that we can import and use as we see fit. Anaconda will, by default, install a few important libraries for us, but there will be others that we will need. We can install those on-the-go using pip.

MySQL (database)

When handling vast quantities of information, we will need a place to store and save all of our data throughout the analysis and preprocessing phases of our projects. For this, we will use MySQL, one of the most common relational databases used to store and retrieve data. We will take a closer look at the use of MySQL by using SQL. In addition to the MySQL relational database, we will also explore the use of DynamoDB, a non-relational and NoSQL database that has gained quite a bit of popularity in recent years.

AWS and GCP (Cloud Computing)

Finally, after developing our machine learning models in Python and training them using the data in our databases, we deploy our models to the cloud using both Amazon Web Services (AWS), and Google Cloud Platform (GCP). In addition to deploying our models, you can also explore a number of useful tools and resources such as Sagemaker, EC2, and AutoPilot (AWS), and Notebooks, App Engine, and AutoML (GCP).

Read More