PyGrunn: gevent - Denis Bilenko¶

Tags: pygrunn, python

(One of the summaries of the one-day 2014 PyGrunn conference in Groningen in the Netherlands).

gevent gives you something that works pretty much like threads without many of the drawbacks. Denis Bilenko is the author of it. It works by using user-level event loops. Gevent mostly works at the level of IO, for instance for web connections, sockets, processes, etc.

Gevent tries to be very stdlib compatible. Gevent modules are mostly drop-in replacements for python standard lib ones: just change the import and you use the gevent version. This helps a lot with the learning curve (“just the generic stdlib way of working”) and it also helps by forcing backward compatibleness (“it has to work like the stdlib version”).

Note: gunicorn, used a lot for Django, uses gevent behind the scenes.

Gevent was written to avoid the complexity of event loops. Event loops mean you give up most of the control of your code to whatever is happening inside those loops. Regular python exception handling becomes almost impossible. And you have to give up context managers. And the common synchronous programming style. Giving all this up is not needed: use gevent.

Gevent uses “greenlets” behind the scenes: so-called “stackful co-routines”. Switching is just a function call. And you can switch to any target. And it saves the whole stack, like a thread. But… it is more effecient than threads. Threads take time to set up and take quite some memory. Greenlets/gevent uses less resources.

But… when he did some benchmarks, he saw that threads performed better than greenlets. he found out that apparently linux 2.6 improved the generic thread performance a lot. On-going work, obviously :-)

His main tip: migrating between gevent and threads is easy. Migrating between async and sync models is not. So use either gevent or threads. Don’t bother with async.