Python concurrency

From wikinotes

This section is dedicated to concurrency. Threading, Mulitprocessing, Queues, and any other tricks to do multiple things at once.


interface python version description
python thread everywhere
python threading 2.4+ Traditional class-based interface to threading (like java)
python multiprocessing 2.6+ Duplicate interface to threading, but for multiple processes.
python futures 3.2+
python coroutines 3.5+

Synchronization Primitives

interface python version description
python mutex deprecated since 2.6
python queue everywhere
python threading.Event 2.4+



os.pipe supplies you with two file descriptors, read and write. Anything written to the 'write' pipe is received by the 'read' pipe. You can incorporate pipes into loops, using them like sockets and periodically check them for new information.

(pipe_r, pipe_w) = os.pipe()



file-descriptors cannot be used with select on windows, only sockets. (but AF_UNIX sockets are available in win10-17603)

Select monitors multiple sockets or pipes(unix only) at once to check if they are:

  • ready to read
  • ready to write
  • an exceptional condition has occurred (see system-specific man-page)

read_fds = []
for filename in  (only sockets only sockets 'fileA.txt', 'fileB.txt', 'fileC.txt'):
    read_fds.append(open(filename), 'r')

(ready, write, err) =
    rlist=read_fds,     # wait until ready to read
    wlist=[],           # wait until ready to write
    xlist=read_fds,     # wait for exceptional-condition