wednesday lightning talks

Tags: pycon, python, django

(One of my summaries of a talk at the 2017 conference).

Lightning talks, so I probably won’t get all the names right. If you have additions/corrections, mail me :-)

Automatic screenshots of your web app - Raphael Michel

Raphael develops a web application. A web app needs documentation. Documentation needs screenshots.

Making screenshots by hand is a lot of work. You have to do it over and over again for every new version.

Selenium is a tool for instrumenting/steering your browser. He uses chrome --headless. So a proper browser, but without the GUI. py.test handles the various testcases, including the fixtures. Django has a LiveTestServerTestcase that is also handy.

The screenshots are written like tests. Really simple and powerfull.

Note: 2018 will be in Heidelberg, Germany. 23-27 May.

Counter-intuitive optimizations - Michael Penkov

He had to make some counter-intuitive optimizations lately.

He gathers data about websites and stores the info in mongodb. 500 million domains, more or less. He wanted, from an existing list, to know which domains weren’t in the system yet.

Doing it in python and querying the DB was slow. In the end he exported everything to text files and did it with linux command line tools like sort. Much faster….

Easy releases with zest.releaser - Reinout van Rees (myself)

See :-) Easy releases of (python) programs: tagging, updating version number, new changelog header, pypi upload, etc.

Deploying your ML applications with a single command - Anand Chitipothu

When you deploy a machine learning application, you normally want to expose some function as an API for others to use.

They’ve written a tool called “firefly” to make this easier.

The big data cheat sheet - Jens Dittrich

There is no database performance problem anymore. Regular queries ought to be 1ms or 1ns: way below the human perception speed of 25ms.

Problems start when you’ve got Big Data. But: what is Big Data? How big is big?

Big data…. there are four basic solutions to it:

  • Horizontal partitioning/sharding.

  • Replication (including caching, view.

  • Hashing (may be persisted or not).

  • Differential files/LSM. Invented five times a year. Don’t re-invent it.

So: it is mostly solved already.


The things they don’t tell you in university - Sophia

She recently moved from university to industry…

  • You might not have as much data as they initially told you.

  • Watch out with ‘special’ values like -999. Those might actually mean ‘None’.

  • Essential data might be completely missing.

  • You sometimes need something to break. Only you’re not allowed to break it.

  • Boss: “I don’t care if it is impossible, I’ve already sold it to two customers”.

So: don’t trust anyone in the university if they tell you about perfect datasets without omissions and errors :-)

Open source days Kopenhagen

16+17 March 2018 there’ll be an open source conference in Kopenhagen. Anything open source is fine.

Photo explanation: simply a picture from my train trip (with a nice planned detour through the Eifel) from Utrecht (NL) to Karlsruhe (DE). The disused ironworks in Völklingen, a UNESCO heritage site. logo

About me

My name is Reinout van Rees and I work a lot with Python (programming language) and Django (website framework). I live in The Netherlands and I'm happily married to Annie van Rees-Kooiman.

Weblog feeds

Most of my website content is in my weblog. You can keep up to date by subscribing to the automatic feeds (for instance with Google reader):