Analysis Environment Implementation Issues: Python Threads

Python Threads Facts

From the python documentation python does use native threads. However, python uses a 'global interpreter lock' which means only one thread at a time may be executing interpreted python code. Also, unless specially written any C calls made from python are run 'atomically'. C functions can be specially written to release the 'global interpreter lock'. The native python I/O routines do just that.

This means that if we have two independent computationally demanding algorithms we need to run and these algorithms are written using purely python, it would not help to put them on two different python threads since they will just be serialized anyway. The only time this would be worthwhile is if both algorithms were producing a result (say a histogram) which the user is waiting for and if one of the algorithms is particularly shorter than the other one then the user would be able to respond to the results of the shorter one in a guaranteed shorter time (assuming we can not order them so that the short one is always run before the long one, but that only helps on average if the longer algorithm is at least twice as long as the short one).

GUIs

It makes good sense to have the thread controlling the GUI be different from the thread running the results of the execution engine. This keep the GUI 'alive' while processing goes on.

Th Rpy package does run its GUI on a separate thread. However, all calls to R functions are synchronized with the GUI thread (presumable this means R's GUI is not thread safe with respect to R's functions) which means the R function calls hold both the 'global python interpreter' lock and the R GUI lock.

Additional Threads

In principle, additional threads might be useful. E.g., say each time we update the priliminary version of the histogram we recalculate its binning before sending it off to plot. Such rebinning would not have to hold up the rest of the execution engines work nor slow down the GUI. Therefore if the rebinning could be run in parallel, then we'd save time. However, if this rebinning were done in straight python (or simple C) it would be serialized with the execution engine and we would not see a benefit (in fact the user would see that updates happening with a lower frequency). However, if the execution engine were reading data from an I/O device with released the 'global interpreter lock' then some speed gain could be seen. Unfortunately, I've checked and the PyROOT code does not release the 'global interpreter lock'.

-- ChrisDJones - 03 Nov 2006