Mock your database - Marc Tamlyn¶

Tags: django, djangocon

Your database is slow. Especially for testing. Nobody likes waiting, so SPEED is essential. Speed of iteration. Faster unit tests mean more tests. No one likes waiting.

Once you create a couple of model objects in your tests, you’re going to hit your database. He did a quick test with some 8 database queries. 5 seconds for 1000 runs on postgresql. 4 seconds for in-memory sqlite, so that’s a bit better.

Why not use factory_boy? That could help, right? More efficient instantiation, perhaps, all in one go (as the test was with a bunch of related objects)? 6 seconds on postgres, sigh. Higher.

He tried a dirty dirty dirty crazy hack, using Django internals. (I looked to my right and the #1 Django database core committer was grinning from ear to ear…). No database queries. 1 second.

Better: mocking. So he did some still-quite-hacky mocking with the mock library. 1.2 seconds. Ok, but the code is a bit yucky.

Summary till now: avoiding the database is quicker. The difference is bigger than the difference between postgresql and in-memory sqlite. So… write your code in such a way to make avoiding the database easier.

But… is there a better way? He looked for inspiration in some mozilla project that mocked a Redis database: very quick and without a Redis dependency.

So he proposes https://github.com/mjtamlyn/django-test-db . It is not a database. It patches QuerySet to provide the same API but not to hit your database. Ideal case would be to use this locally and to run it against your real database on your continuous integration server.

Note: it doesn’t totally work yet, you still need to do some manual mocks. So it needs auto-patching. Oh, and some things don’t go through the QuerySet. But regular fields work, foreign keys work, etc.

He’d love feedback.