Multiprocessing is creating, running, ending multiple processes and threads from a single process for a specific purpose. Python programming language provides a lot of different features of multiprocessing.
Multiprocessing Features
Multiprocessing provides a lot of features to the program or application developers. We will list some of the below.
Performance
is the most popular feature of multiprocessing. A single process task can be divided into multiple ones where each process can complete faster. In the end, the complete task will be completed in less time.Reliability
is provided by creating simple tas pieces were even some piece corrupt we can create a little piece of task where other pieces to not interfere.Efficiency
is done by using all CPU sources like threads r multiple CPUs. Modern CPU’s provides multiple processing units and threads to run multiple processes at the same time. With multiprocessing, we can use these resources efficiently.
Import Multiprocessing Module/Library
Python provides multiprocessing functions and capabilities with the multiprocessing
module. So in order to use multiprocessing functions and capabilities, we have to import multiprocessing
module like below.
import multiprocessing

We can also see that multiprocessing
module provides process
and similar functions.
Spawn Multiple Processes
We will start with a simple process spawn or create which will create a new process instance with the Process()
function. Process()
function basically needs two parameters where we need to specify the function name where the newly created function will run. Also, we can provide some parameters with the args
set like below.
import multiprocessing
def foo(message):
print(message)
if __name__ == '__main__':
p = multiprocessing.Process(target=foo, args=("This is a spawn",))
p.start()
p = multiprocessing.Process(target=foo, args=("This is a spawn 2",))
p.start()
p = multiprocessing.Process(target=foo, args=("This is a spawn 3",))
p.start()

We can see that we have created 3 processes to spawn and with the start()
function we started them. We can not start a single new process multiple times so we have to define a new process with the Process()
function like above.
Set Name For Created Processes
Multiprocessing provides different useful features like setting a name for a new process. We will use the name
option which will be provided to the Process()
function. We can access this name from the newly created process with the multiprocessing.current_process().name
attribute like below.
import multiprocessing
def daemon(message):
print(multiprocessing.current_process().name)
print(message)
if __name__ == '__main__':
p = multiprocessing.Process(name="This process name is daemon", target=daemon, args=("This is a spawn",))
p.start()

Create Daemon Processes
When a normally created or spawned process completes its execution its ended. This is the normal behavior of all of the normal processes. But some special cases we need to run a process like a daemon or service where they will not stop even they have no task to complete. Daemons mainly wait for a specific event like connection, trigger some event to run. Python multiprocessing provides the daemon
option which will turn given process into a daemon which will run forever normally. But if the main process finishes the daemons will be excited too.
import multiprocessing
def daemon(message):
print(message)
if __name__ == '__main__':
p = multiprocessing.Process(target=daemon, args=("Waiting For Tasks",))
p.daemon = True
p.start()
print("This will not printed bec")
Wait For Process To Complete
After the main process reached the end, the process will exit and all the child processes will stop. But we can change this behavior where the main process will wait for the child process to finish and exit. We will use join()
function where the specified child process exit will be waiting by the main process.
import multiprocessing
def daemon(message):
print(message)
if __name__ == '__main__':
p = multiprocessing.Process(target=daemon, args=("Waiting For Tasks",))
p.daemon = True
p.start()
p.join()
print("This will not printed bec")
Terminate Process
After a new child process is created it will exit after the task is completed. But In some cases, we may need to terminate the given child process before it finishes. We can use terminate()
function which will terminate the given process.
import multiprocessing
def daemon(message):
print(message)
if __name__ == '__main__':
p = multiprocessing.Process(target=daemon, args=("Waiting For Tasks",))
p.daemon = True
p.start()
p.terminate()
print("Process Is Terminated")
Check If Given Child Process Is Running or Alive
After creating a child process it will run according to its task or whether it is a daemon. We may need this process current situation whether it is running or alive from the main process. We will use is_alive()
function which will return boolean values True or False according to the child process situation.
import multiprocessing
def daemon(message):
print(message)
if __name__ == '__main__':
p = multiprocessing.Process(target=daemon, args=("Waiting For Tasks",))
p.daemon = True
p.start()
print(p.is_alive())
Print Process Exit Status
The process executes, completes, and exits. But the process will have some exit or finish reasons for different exit cases. For example, a process can be exit because it’s killed or it finishes its execution successfully. We can get or print the child process exit status with the exitcode
attribute.
import multiprocessing
import time
def foo(message):
print(message)
if __name__ == '__main__':
p1 = multiprocessing.Process(target=foo, args=("Waiting For Tasks",))
p2 = multiprocessing.Process(target=foo, args=("Waiting For Tasks",))
p1.start()
p2.start()
p2.terminate()
time.sleep(3)
print(p1.exitcode)
print(p2.exitcode)

There are meaning about the exit status.
- If equal to the 0 the process is executed and finished successfully without any error.
- If bigger than 0 there is an error related to the code.
- If lower than 0the process is killed with a signal.
Log Multiprocess Operations
Every created process has some logging mechanism. We can redirect these logs into a file or standard output with the log_to_stferr(logging.DEBUG)
by specifying the level of the log verbosity which is DEBUG
in this case.
def foo(message):
print(message)
if __name__ == '__main__':
multiprocessing.log_to_stderr(logging.DEBUG)
p1 = multiprocessing.Process(target=foo, args=("Waiting For Tasks",))
p2 = multiprocessing.Process(target=foo, args=("Waiting For Tasks",))
p1.start()
p2.start()
p2.terminate()
time.sleep(3)
print(p1.exitcode)
print(p2.exitcode)
