Python comes with batteries included, so it also includes a C profiler that works pretty well. But it doesn’t work nice for django because the output is so huge. If you use the GUI RunSnakeRun, it is more managable that way.
Additional problem with cProfile: it has about 100% overhead, so you can’t run it in production (which they need).
You can do more targeted profiling. For instance in Django. The way a web framework processes requests is normally always in the same way. You can use that during profiling.
There are two important stages: interrupt and inquire.
sys._current_frames()to get yourself the frames (“stack trace”) from the current thread. Here you can extract the filename, line number and so of the most interesting frames. He’s building a “call tree” structure, mapping how often a function is called, preserving the parent/child relation between functions. The tricky part was visualizing it :-) In the end they did it with d3.js.
There’s some 3% overhead in the thread approach, so that’s real good. They can switch it on for a type of request and it’ll profile the next 100 requests of that type.
My name is Reinout van Rees and I work a lot with Python (programming language) and Django (website framework). I live in The Netherlands and I'm happily married to Annie van Rees-Kooiman.
Most of my website content is in my weblog. You can keep up to date by subscribing to the automatic feeds (for instance with Google reader):