Lately I’ve been doing some Python multi-threading to make the best use of some of our amazing server resources. As I was pondering the reasons why one of our 8-core servers reported 83% idle despite 8 threads banging away, I re-discovered the Global Interpreter Lock.
The GIL enforces Python’s requirement that only a single bytecode operation is executed at a time. My nicely coded multi-threaded app was only being executed serially!! Sadly, this seems unlikely to change, even in Python 3000. Last year Guido said:
“Just Say No to the combined evils of locking, deadlocks, lock granularity, livelocks, nondeterminism and race conditions.”
I was brought up to believe that threading was dirty and independent communicating processes were the way to go. But even I realize that this just isn’t practical in these days of GUIs, multi-core processors, and application servers.
Why does the Python community accept the GIL? Is it because most people only use Python as a scripting language? Are there simple workarounds (e.g. not forking, shared memory, or the like) that I’m missing?