I had some problems in some of my Django websites:
What I hoped was that gunicorn would kill off the offending process after a while. That was one of the reasons to switch to gunicorn. But the reliable-killing-off didn’t seem to be happening reliably. I couldn’t really put my finger on it.
The one thing that stands out is that both problems are related to non-pure-python libraries. Libraries with some c-level code in them. Perhaps gunicorn cannot handle them right?
I tried out the
--preload option to gunicorn. This loads the application in memory
beforehand instead of when the individual worker starts. Perhaps this helps
with some global import-time state in one of those offending libraries? I
Hurray! If I use
--preload, the mysql problem goes away. “Away” in the
sense that newly started workers work reliably. At least in my current test.
… and hurray, with
--preload, gunicorn also reliably recovers from a
faulty shapefile (that would have it spinning 100% indefinitively otherwise).
I don’t fully understand the reason. My guess is that
better with not-pure-python libraries. If you have a similar problem, it might
be worth a try.
My name is Reinout van Rees and I work a lot with Python (programming language) and Django (website framework). I live in The Netherlands and I'm happily married to Annie van Rees-Kooiman.
Most of my website content is in my weblog. You can keep up to date by subscribing to the automatic feeds (for instance with Google reader):