Day 4 at Europython
The day after the Europython Social Event started very slow. I missed the keynote as well as the first talk. The theme of the day was the GIL and PyPy. (In the picture attached, you can see how much faster PyPySTM is than normal PyPy.)
GitFS
https://github.com/PressLabs/gitfs
Danci and Vlad presenting their solution on how to deal with people that can only upload via FTP and don’t know what git is.
They build a mountable representation of a git branch tree using python pygit2 and fuse via pyfuse.
Raspberry Pi - Weatherstation
A really neat project that is supposed to show kids how to interact and work with digital sensors. For example, the students had to calculate how much it rained based on a variaty of digital and analog sensoric data.
- naturebits (birdbox)
- AstroPi - fixed date for the rocket launch
- RaspPi - teaching
On a side note he shared with us an annectdote where he’s been testing the airquality and noticed that around 3pm on a weekday the quality
would significantly drop. Turns out, his office is right above the parking lot on the first floor and all the teachers would leave the building at 3pm.
The Infamous GIL
Larry Hastings presentation are as always an absolute event to watch.
Speaking about the GIL, he took us on a historic journey to the beginning of the GIL.
1992 multithreading became known.
Globals, had a pointer to the current executing frame
The GIL received some additional changes for Python 3 in 2009, the new GIL.
Guido only allows the GIL to be changed if single core performance doesn’t get slower.
There have been multiple tries to fix the GIL, starting with a branch of Python 1.4, however all implementations had the same issue.
If the GIL is a deal breaker, one can always use one of the other implementations of Python; Jython, IronPython which use garbage collection instead.
All major languages developed in the time after Python came out are using a garbage collection and don’t support C extensions.
He comes to the conclusion that to support a garbage collector, CPython needs to drop C-extension support.
PyPy - STM - the GIL is dead
PyPy is a just-in-time (JIT) compiler for python that started about 10 years ago. PyPy-STM is a forked version of PyPy that implements
software-transactional-memory.
This provides a huge speed benefit if you’re using multiple threads and shared data.
As example, he showed us a realtime 3D rendered view as comparrison, CPython managed to generate 0.8 FPS, PyPy did 17 FPS and PyPy-STM rendered 35 FPS. PyPy-STM can run N-number of threads concurrently; N equal the number of CPU cores. One important current limitation is that one needs to ensure that each thread is either CPU or I/O bound not both and that it doesn’t switch between them either since that is attached to a lot of overhead.
This sounds easy, however simply executing a log statement consitutes an I/O operation which will slow down a CPU bound thread significantly.
How do the STM transactions work
While analyzing the GIL, one notices that it actually behaves just like a database. PyPy-STM needs to simply check if another thread touched the same object in another transaction and then only if they conflict it’ll need to do some work to arrange a consensus.
Obviously I/O work will be committed first since we can’t turn back time and redo it.
To ensure that no automatic context switching happens he added a context manager “with atomic” which ensures that.
#pypy on irc.freenode.net
QnA
How does PyPy-STM know when an effect like I/O will happen?
— The simple answer is, whenever CPython would release the GIL.
What’s the speed difference between PyPy-STM and PyPy for single threaded applications?
— It’s about 20% to 30% slower.
Lightning talks
@nailor showed us a introspection library that allows to inspect running processes called python-manhole. http://python-manhole.readthedocs.com
Announced PyCon Pl on the 13th Oct 2015 - 190Euro incl Accom and Food.
