PyGrunn: list-man, pragmatic system integration - Doeke Zanstra¶
(One of my summaries of the 2026 one-day PyGrunn conference in Groningen, NL).
When automating in a big company with many systems, you often end up with spaghetti: many systems connecting to a lot of the others… A common solution is to have a “bus architecture”. Generic existing “enterprise service bus” solutions were clearly overkill, so he proposed an alternative solution.
He made a couple of assumptions/choices. All data is tabular data. He wanted to store a copy of data in a database. SQL views to access the data. So: multiple sources that he wanted to import in a central database (which would function as a sort of “read-only enterprise service bus”). And a generic sql/view-based way of accessing the data.
He initially focused on read-only data. And he started real simple. Just a bash script that ran regularly that scraped data from other systems and injected it in the database.
In the second version of the system, for every system he wrote a target/command in a
Makefile. Every thing that needed to be scraped got its own table (called a “list”
in his system”). Lists could be compared. The first killer app was a comparison between
a telephone list and the list of employees so that differences could be consolidated.
For the third version, he started using more and more python. CSV file imports. Downloaders from REST APIs. All configurable so that he could use the same python script for many different sources.
He now had a simple sytem for which he could write views and exports.
Publishing data on the intranet via the “jekyll” static site generator. For instance a “mug book” of all employees.
And regularly exporting a list of names+emailaddresses in a format suitable for the multifunctional printer: to make it easy to select your email address when scanning on the printer.
An export to a google spreadsheet that combined the holiday spreadsheet with the data on part-time days.
Security was handled with a role-based system.
Unrelated photo: the “lac de Kruth-Wildenstein” reservoir during a family holiday in France in 2006.