Multiprocessing and Parallel Programming In Python – POFTUT

Multiprocessing and Parallel Programming In Python

Multiprocessing is creating, running, ending multiple processes and threads from a single process for a specific purpose. Python programming language provides a lot of different features of multiprocessing.

Multiprocessing Features

Multiprocessing provides a lot of features to the program or applications developers. We will list some of the below.

  • Performance is the most popular feature of multiprocessing. A single process task can be divided into multiple ones where each process can complete faster. In the end, the complete task will be completed in less time.
  • Reliability is provided by creating simple tas pieces were even some piece corrupt we can create a little piece of task where other pieces to not interfere.
  • Efficiency is done by using all CPU sources like threads r multiple CPUs. Modern CPU’s provides multiple processing units and threads to run multiple processes at the same time. With multiprocessing, we can use these resources efficiently.

Import Multiprocessing Module/Library

Python provides multiprocessing functions and capabilities with the multiprocessing module. So in order to use multiprocessing functions and capabilities we have to import multiprocessing module like below.

Import Multiprocessing Module/Library

Import Multiprocessing Module/Library

We can also see that multiprocessing module provides process and similar functions.

Spawn Multiple Processes

We will start with a simple process spawn or create which will create a new process instance with the Process()function. Process() function basically needs two parameters where we need to specify the function name where the newly created function will run. Also, we can provide some parameters with the args set like below.

Spawn Multiple Processes

Spawn Multiple Processes

We can see that we have created 3 processes to spawn and with the start() function we started them. We can not start a single new process multiple times so we have to define a new process with the Process() function like above.

Set Name For Created Processes

Multiprocessing provides different useful features like setting a name for a new process. We will use the name option which will be provided to the Process() function. We can access this name from the newly created process with the multiprocessing.current_process().name attribute like below.

Set Name For Created Processes

Set Name For Created Processes

 

Create Daemon Processes

When a normally created or spawned process completes its execution its ended. This is the normal behavior of all of the normal processes. But some special cases we need to run a process like a daemon or service where they will not stop even they have no task to complete. Daemons mainly wait for a specific event like connection, trigger some event to run. Python multiprocessing provides the daemonoption which will turn given process into a daemon which will run forever normally. But if the main process finishes the daemons will be excited too.

Wait For Process To Complete

After the main process reached the end, the process will exit and all the child process will stop. But we can change this behavior where the main process will wait for the child process to finish and exit. We will use join() function where the specified child process exit will be waiting by the main process.

Terminate Process

After a new child process is created it will exit after the task is completed. But In some cases, we may need to terminate the given child process before it finishes. We can use terminate() function which will terminate the given process.

Check If Given Child Process Is Running or Alive

After creating a child process it will run according to its task or whether it is a daemon. We may need to this process current situation whether it is running or alive from the main process. We will use is_alive() function which will return boolean values True or False according to the child process situation.

Print Process Exit Status

The process executes, completes and exits. But the process will have some exit or finish reasons for different exit cases. For example, a process can be exit because it’s killed or it finishes its execution successfully.  We can get or print the child process exit status with the exitcode attribute.

Print Process Exit Status

Print Process Exit Status

There are meaning about the exit status.

  • If equal to the process is executed and finished successfully without any error.
  • If bigger than there is an error related to the code.
  • If lower than the process is killed with a signal.

Log Multiprocess Operations

Every created process has some logging mechanism. We can redirect these logs into a file or standard output with the log_to_stferr(logging.DEBUG) by specifying the level of the log verbosity which is DEBUG in this case.

Log Multiprocess Operations

Log Multiprocess Operations

Leave a Reply

Your email address will not be published. Required fields are marked *

Enjoy this blog? Please spread the word :)