Djangocon: two talks, ecology and healthchecks¶

Django powered ecological data - Jakub Witold Bubnicki¶

Jakub Witold Bubnicki works on data-intensive science. For instance for biological sciences, more and more data is becoming available. Globally. In data intensive research, you often have to integrate and adapt multiple sources.

Smaller teams (let anone individual PhD students) can often not create the infrastructure needed.

He collects photos from his “camera traps” for two years now. He has to integrate this with geographical data and, for instance, remote temperature datasets. Often, the data includes both a space and a time component (location and date).

The collected data needs to be easily accessible, discoverable, shareable and reusable. This is often not the case. So he started designing a django architecture to fix this. He also uses lots of existing applications like metacat (catalogs), geoserver (geographical data) and rasdaman (rasters).

Django provides the authentication, filtering, browsing, searching, permissions. It connects to the external applications.

The django site itself also has an API so that others can connect to it.

Note: they’re still working on it, it is not finished. But the generic data browsing and a bunch of mapviews make the existing system already quite handy.

Health checks for django - Benoit Bryon¶

Benoit Bryon presents hospital, a tool to diagnose your python project (supervision, monitoring).

Your app normally constists of a lot of moving parts. Database, cache, mail, filesystem, etc. You could simply deploy it and wait for the bug reports. Or you could do some manual checks.

So… automatic checks would be nice, right? There are of course already existing technologies like tests, provisioning (states), logging and monitoring. But something is missing.

What is missing are specific health checks that assertions on the running application. Assertions about configuration, the environment, data, etc. It is done from within the application, written by the developers. Just like a real-life doctor asks you, the patient, what you feel and what is wrong. He won’t guess blindly.

The health check should become a part of the regular supervision toolkit.

The benefits:

• You can validate a deployment.

• You can monitor application status.

• It is easier to diagnose and triage issues. (“Is the database OK? Is the data available?”)

With the “hospital” tool, you can write health checks like tests. Yes: regular unittest syntax. You can run them with pytest/nose or with the hospital command line interface. But… also via the web as a WSGI application. It returns json with the status (which you can later integrate into the normal monitoring tools (munin, nagios) that you use).

Example checks include hospital.assert_ping(...) and hospital.assert_http_response(...). If you want more generic checks: make a pull request.

Note: django’s upcoming check framework is for static checks, not dynamic ones.

Everything you normally do manually to check a deployment, do that via a health check instead!