The subprocess module

The sub-process module allows us to spawn processes, connect to their input/output/error pipes, and obtain their return codes. sub-process should be used for accessing system commands. With sub-process you can suppress the output, which is very handy when you want to run a system call but are not interested about the standard output. 

The subprocess.call method()

The subprocess.call method is an easy to way to invoke an external program. It works on all platforms.  It also gives you a way to cleanly integrate shell commands into your scripts while managing input/output in a standard way. We can use sub-process.call return codes to determine the success of the command. 

Every process will return an exit code and you can do something with your script based on that code. If the return code is anything else than zero, it means that an error occurred. Let's see an example which uses subprocess to launch a more powerful compressor, PAQ. A PAQ executable is available in downloadable archives on the Internet. We will use a PAQ8 implementation. The same command compresses, and expands, a file. See the code below:

import subprocess

exe = r"C:\fp8_v2.exe"

source = r"C:\profiles\file.bin"

subprocess.call(exe + " " + source)

When we run the program we get the following output:

Creating archive C:\profiles\file.bin.fp8 with 1 file(s)...

File list (18 bytes)

Compressed from 18 to 22 bytes.

1/1  Filename: C:/profiles/file.bin (3836648 bytes)

Block segmentation:

 0           | default   |     28016 bytes [0 - 28015]

 1           | jpeg      |      8917 bytes [28016 - 36932]

 2           | default   |   3799715 bytes [36933 - 3836647]

Compressed from 3836648 to 114930 bytes.

Total 3836648 bytes compressed to 114958 bytes.

Time 44.26 sec, used 180892539 bytes of memory

Close this window or press ENTER to continue...

The subprocess.Popen()

The sub process module enables you to start new applications from your Python program. We can start a process in Python using the Popen function call. To start a command prompt from a program we can pass the program’s filename to subprocess.Popen() as shown below:

import subprocess

subprocess.Popen('C:\Windows\System32\cmd.exe')

When we run this program the command prompt will be open in the ouput window as shown below:

------------------

(program exited with code: 0)

Press any key to continue . . . Microsoft Windows [Version 6.1.7601]

Copyright (c) 2009 Microsoft Corporation.  All rights reserved.

F:\Python_Code\examples>

The launched program is not run in the same thread as our Python program. The return value is a Popen object, which has two useful methods: poll() and wait().

The poll() method will return None if the process is still running at the time poll() is called. If the program has terminated, it will return the process’s integer exit code. An exit code is used to indicate whether the process terminated without errors (an exit code of 0) or whether an error caused the process to terminate (a nonzero exit code—generally 1, but it may vary depending on the program).

The wait() method will block until the launched process has terminated. This is helpful if we want our program to pause until the user finishes with the other program. The return value of wait() is the process’s integer exit code. See an example below:

import subprocess

cp = subprocess.Popen('C:\Windows\System32\dvdplay.exe')

print(cp.poll()==None)

print(cp.wait())

print(cp.poll())

Here we open a dvdplay process and while it is running we check if poll() returns none which it does as the process is still running. Next we close the dvdplay program and call wait() on the terminated

process. Both the wait() and poll() now return 0, indicating that the process terminated without errors. The output of the program is shown below:

True

0

0

------------------

(program exited with code: 0)

Press any key to continue . . .

The Python documentation recommends the use of Popen in advanced cases, when other methods such like subprocess.call cannot fulfill our needs.

We can pass command line arguments to processes you create with Popen(). To do so, you pass a list as the sole argument to Popen(). The first string in this list will be the executable filename of the program you want to launch; all the subsequent strings will be the command line arguments to pass to

the program when it starts. In effect, this list will be the value of sys.argv for the launched program.

Most applications with a graphical user interface (GUI) don’t use command line arguments as extensively as command line–based or terminal based programs do. But most GUI applications will accept a single argument for a file that the applications will immediately open when they start. Let's try to open a text file in our program directory using the Popen() which will not only launch Notepad application but also have it immediately open the desired text file. See the code below:

import subprocess

subprocess.Popen(['C:\\Windows\\notepad.exe', 'C:\\Python_Code\\examples\\cities.txt'])

When we run this program the cities.txt file opens in notepad.

It is also possible to launch a Python script from Python just like any other application. We just have to pass the python.exe executable to Popen() and the filename of the .py script we want to run as its argument. See the example below:

import subprocess

subprocess.Popen([r'C:\Users\Python\AppData\Local\Programs\Python\Python36\python.exe', 'complete_name.py'])

Pass Popen() a list containing a string of the Python executable’s path and a string of the script’s filename. Unlike importing the Python program as a module, when your Python program launches another Python program, the two are run in separate processes and will not be able to share each other’s variables. To handle separators in paths better, we can use raw string literals with the "r" prefix.

Each operating system has a program that performs the equivalent of double-clicking a document file to open it. On Windows, this is the start program. Python can also open files this way with Popen(). See the program below:

import subprocess

obj = open('cities.txt','w')

obj.write('Jabalpur')

subprocess.Popen(['start', 'cities.txt'], shell=True)

Here we write Jabalpur to a new cities.txt file. Then we call Popen(),passing it a list containing the program name (in this example, 'start' for Windows) and the filename. We also pass the shell=True keyword argument, which is needed only on Windows. The operating system knows all of the file associations and can figure out that it should launch, say, Notepad.exe to handle the cities.txt file. When we run this program a new new cities.txt file is created by the program and the program writes Jabalpur to the file and opens it in notepad.

With this I am ending today's post, Until we meet next keep practicing and learning Python as Python is easy to learn!

Read More

Multithreading (Introduction)

A thread is a sequence of instructions within a program that can be executed independently of other code. It is an entity within a process that can be scheduled for execution. Also, it is the smallest unit of processing that can be performed in an OS (Operating System). 

A thread contains all this information in a Thread Control Block (TCB):

Thread Identifier: Unique id (TID) is assigned to every new thread

Stack pointer: Points to thread’s stack in the process. Stack contains the local variables under thread’s scope.

Program counter: a register which stores the address of the instruction currently being executed by thread.

Thread state: can be running, ready, waiting, start or done.

Thread’s register set: registers assigned to thread for computations.

Parent process Pointer: A pointer to the Process control block (PCB) of the process that the thread lives on.

So far we have used a single thread programs and one such program is shown below:

import time

print('Start Thread1.')

time.sleep(5)

print('Sleeping time over, Thread1 stopped')

The output of this program is shown below:

Start Thread1.

Sleeping time over, Thread1 stopped

------------------

(program exited with code: 0)

Press any key to continue . . .

The above program prints Start Thread1, wait for 5 secs and then prints Sleeping time over, Thread1 stopped. This is because Python programs by default have a single thread of execution. A single threaded program has only one finger. But a multi threaded program has multiple fingers. Each finger still moves to the next line of code as defined by the flow  control statements, but the fingers can be at different places in the program, executing different lines of code at the same time.

Rather than having all of your code wait until the time.sleep() function finishes, you can execute the delayed or scheduled code in a separate thread using Python’s threading module. The separate thread will pause for the time.sleep calls. Meanwhile, your program can do other work in the original thread.

Let's modify our program as shown below:

import threading,time

print('Start Thread1.')

time.sleep(5)

print('Sleeping time over, Thread1 stopped')

print('Next part of program started')

def holdon():

  

    time.sleep(5)

    print('Thread which called function stopped')

  

obj = threading.Thread(target=holdon)

obj.start()

print('Next part of program ended')

The output of this program is shown below: 

Start Thread1.

Sleeping time over, Thread1 stopped

Next part of program started

Next part of program ended

Thread which called function stopped

------------------

(program exited with code: 0)

Press any key to continue . . .

To make a separate thread, you first need to make a Thread object by calling the threading.Thread() function. We define the holdon() function that we want to use in a new thread. To create a Thread object, we call threading.Thread() and pass it the keyword argument target=holdon which means the function we want to call in the new thread is holdon(). This is because you want to pass the holdon() function itself as the argument, not call holdon() and pass its return value. After we store the Thread object created by threading.Thread() in obj,we call obj.start() to create the new thread and start executing the target function in the new thread. 

As per our program Next part of program ended is the last print statement then it should be printed in the last in the output but instead it is printed second last. The reason Thread which called function stopped comes after it is that when obj.start() is called, the target function for obj is run in a new thread of execution. Think of it as a second finger appearing at the start of the holdon() function. The main thread continues to print('Next part of program ended.'). Meanwhile, the new thread that has been executing the time.sleep(5) call, pauses for 5 seconds. After it wakes from its 5-second wait, it prints 'Thread which called function stopped' and then returns from the holdon() function. Thus chronologically, 'Thread which called function stopped' is the last thing printed by the program. 

A Python program will not terminate until all its threads have terminated. When we ran our program, even though the original thread had terminated, the second thread was still executing the time.sleep(5) call hence the program which normally terminates when the last line of code in the file has run waited for the second thread to to terminate and then ended. 

Multiple threads can exist within one process where:

• Each thread contains its own register set and local variables (stored in stack).
• All thread of a process share global variables (stored in heap) and the program code.

Multiple threads can also cause problems called concurrency issues. These issues happen when threads read and write variables at the same time, causing the threads to trip over each other. Concurrency issues can be hard to reproduce consistently, making them hard to debug.

I've kept today's topic concise, we'll continue this topic in next posts, so until we meet keep practicing and learning Python as Python is easy to learn!

Read More