All our websites go to one single apache machine that proxies out the requests to the various backend servers. (Btw, I think I’d like to replace apache with nginx, but this is what’s running now).
Our latest sites run with gunicorn behind nginx.
One URL took a long time to render. Calculating a year’s worth of data. Longer than 30 seconds. And after 30 seconds I’d get an nginx “502 Bad Gateway” error. Looking at the nginx error log I’d see a comment about an upstream gateway unexpectedly closing the connection.
Nginx has a proxy_read_timeout. Default is 60 seconds, not 30, but for good measure I set it to 90 seconds. Didn’t help.
Upstream? What is upstream, what is downstream? I assumed apache would be “up”, as that’s where the request is coming from. There was no relevant timeout setting there.
Turns out that upstream means where the response is coming from. So gunicorn. And yes, gunicorn by default has a timeout of 30 seconds.
Solution: call gunicorn with --timeout 90
.
Now that I’ve written it down I can google it in two years’ time when I’ve forgotten all about it :-) To close it off, here’s a picture of with lots and lots of lines. An apache/nginx/gunicorn problem is less complicated :-)
My name is Reinout van Rees and I program in Python, I live in the Netherlands, I cycle recumbent bikes and I have a model railway.
Most of my website content is in my weblog. You can keep up to date by subscribing to the automatic feeds (for instance with Google reader):