Reinout van Rees’ weblog¶

Leiden python meetup: memory graph - Bas Terwijn¶

2025-09-04

Tags: pun, python

(One of my summaries of the fifth Python meetup in Leiden, NL).

Full title of the talk: memory graph: teaching tool and debugging aid in context of references, mutable data types, and shallow and deep copy.

memory_graph is a python debugging aid and teaching tool. It is a modern version of python tutor. (There is an online demo)

Python has two categories of types:

• Immutable types: bool, int, float, str, tuple, etcetera. They cannot be mutated, so when a value is changed, a copy is made. If you add an item to a tuple, you get a new tuple with the extra item.

• Mutable types: dicts, lists. You can change the values without the actual dict/list changing. You can add items to a list and you still have the same list object.

When you want an actual copy of a mutable type, you need to use import copy and copy.copy(your_list). And copy.deepcopy().

• list2 = list1 is an assignment.

• list2 = copy.copy(list1) gives you a second, separate, list object, but it points at the same values inside it as list1.

• list2 = copy.deepcopy(list1) gives you a second, separate, list object and separate copies of the values inside it.

Watch out with the list2 = list1 assignment. When you add an item to list2, it is also “added” to list1 as it is the same.

He had a couple of simple exercises for us, which were amusingly hard :-)

Apart from the web online demo, there are also integrations for jupyter notebooks and lots of IDEs. Here’s an animated gif from the github repo:

Leiden python meetup: HTMX - Jan Murre¶

2025-09-04

Tags: pun, python, django

(One of my summaries of the fifth Python meetup in Leiden, NL).

The person who invented htmx (Carson Gross) always begins with hypermedia. Hypermedia is a media that includes non-linear branching from one location in the media to another, via hyperlinks. HTML is hypermedia, the world’s most succesful hypertext.

Another important term: HATEOAS, hypermedia as the engine of application state. With hypermedia, state is on the server. So not in a javascript frontend. With traditional single page apps that you see nowadays, you only read some json and the frontend needs to know what to do with it. Lots of logic is on the client. And you get “javascript fatigue”.

With hypermedia, you have server-side rendering. Minimal javascript. Progressive enhancement. SEO friendly. And… accessible by default. The content you get includes the behaviour (like a link to delete an item).

HTMX extends HTML with modern interactivity using simple attributes. You can target specific elements on your page for an update, so you don’t need to get a full page refresh. And you can use any http verb (get/post/put/delete/patch). And you’re not limited to forms and links: any element can trigger a request.

Some attribute examples:

• hx-get issues a GET request to the server when you click on the element.

• hx-post, same with POST.

• hx-target, what you get back from your get/post, where do you want to place it?

• hx-swap: just replace part of the page.

• hx-trigger: when to do the request. Based on a click or based on a timer, for instance.

An advantage of HTMX is maintainability. Complexity is way lower than a single page app. Familiar patterns and regular server-side logic. Much simpler. Accessible (provided you put in some effort).

He showed a nice example with a generated list, a search field and a form. Nice: extra validation on one of the form fields via a hx-post and a hx-trigger="on-blur".

Nice trick for hx-target: you can give it the value of "closest .some-css-class", then it finds the closes enclosing element with that class.

Other niceties: hx-indicator enables a spinner upon a POST and disables it once the POST succeeds. <div hx-boost="true"> around your content tells HTMX to replace your whole page’s content with the new page, the result is the same as normally, only without the temporary flicker when loading the page.

HTMX is great for:

• CRUD applications.

• Content-heavy sites.

• Forms and validation.

• Server-side rendered apps.

• Progressive enhancement.

• Moderate interactivity.

You can read a book about HTMX here: https://hypermedia.systems/

In response to a question: the author considers htmx to be complete and finished. What works now should work in 20 years. So it will rarely change (unlike something like react that changes all the time).

Python Leiden meetup: Deploying python apps with django-simple-deploy - Michiel Beijen¶

2025-07-10

Tags: pun, python, django

(One of my summaries of the fourth Python meetup in Leiden, NL).

Michiel discovered django simple deploy via the django podcast.

Deploying? Previously, heroku was often used as an example in books, but they ditched their free tier and are pretty expensive now. So most examples nowadays just show you how to run django locally. Deploying is harder.

There are several hosting parties that provide relatively easy django hosting like heroku. https://fly.io, https://platform.sh and https://heroku.com . “Easy” is relative, as he ran into some problems with platform.sh and he didn’t have an ssh connection to fix it. And yesterday they nuked his account so he couldn’t re-try it for today’s demo.

Since recently there is also a generic “vps” plugin: just running on some random virtual server with ssh access that you can rent virtually anywhere. https://github.com/django-simple-deploy/dsd-vps . “Random” should be debian-like, btw.

He demoed it on a digitalocean VM (=”droplet”). With some environment variables he got it working within 15 minutes and some manual fixes. The longest wait was the “apt-get upgrade” call.

The VPS version of simple deploy has its drawbacks. It needs a root password, for instance, and at the moment it doesn’t accept ssh key authentication. He previously tried it on a transip.nl host, which doesn’t have a root user with a password: you get a regular user with sudo privileges. The VPS plugin doesn’t like that.

A second “manage.py deploy” (after a trivial template update) also did not work. One of the items it generated the first time was (of course) already created, so it failed the second time. Oh…

Anyway, the VPS version of django-simple-deploy doesn’t seem to work yet.

Python Leiden meetup: Handling crash reports with Bugsink - Klaas van Schelven¶

2025-07-10

Tags: pun, python, django

(One of my summaries of the fourth Python meetup in Leiden, NL).

Bugsink is a tool/website for collecting error messages from (for instance) your websites. You get an email or other notification and you can visit the error page. You’ll see the exact error and a traceback and the environment variables and the library versions.

But… isn’t that just like sentry? Yes. Mostly.

But… why bugsink? It is self-hostable. Originally, 20 years ago, sentry was also self-hostable, but it isn’t anymore.

There’s a bit of workflow, too. You can mark issues as “fixed in the next release”. Bugsink will then ignore new occurrences of the error until it notices you made a release.

It is not just in the same market as sentry, it is even sentry-compatible. If you already have sentry set up, moving to bugsink only means changing the SENTRY_DSN setting. The API is the same.

Self-hosting? You can run a docker. He showed that pip install bugsink also works. It is just a plain regular django site with migrate and runserver. A single instance can easily handle 1.5 million errors per day.

Some generic tricks (whether sentry or bugsink):

• Just use your error tracker in development, too! The nice thing is that it preserves history. When you notice an unrelated error while working on some feature, you can continue working on your feature and go back to the error later.

  Using a local bugsink instance for this can help prevent filling up your production

• Do a pip install dsnrun. (See github). Run a failing python script with dsnrun and any traceback will be posted to your error tracker.

While building bugsink, he made a couple of surprising architectural decisions:

• Some tasks need to be run outside the main loop. With django, you often use celery. But that means installing and running a second program, which was a bit at odds with his aim of making bugsink easily self-hostable.

  So… he wrote his own library for that: snappea.

• Sqlite is the default database. Though often said to be only for small use cases, others say it can handle much bigger tasks. He uses sqlite and it can handle 1.5 million issues per day just fine.

Some answers/questions afterwards:

• License and business model? The license is https://polyformproject.org/licenses/shield/1.0.0/ , so it isn’t Stallman-compatible open source, but quite open.

• How did you make the frontend? Mostly plain html generated by django templates. He’s one developer competing with 150 sentry devs and didn’t have the bandwidth to build a fully javascript frontend. And… it isn’t needed. And it is much easier on self-hosters as they also don’t have to upgrade a big ball of javascript.

  He looked at htmx, but didn’t need it yet.

My article in the German “ADJ” magazine¶

2025-06-19

Tags: eifelburgenbahn, trains

I’ve got a realistic German model railway in my attic and I tend to take that realism seriously. So I spend a lot of time researching things like proper signalling and operation. For mainline operations, you can find a lot of information on signalling. Everything is by the book. But for simple single track branch lines, things get harder to figure out.

How do things work when there are no electrical block systems between the stations and all you have is a telephone line? And what if the station isn’t even manned? No signals? Anyway, it has been fun trying to figure it out. I’ve made a 45 minute video explaining it in the context of one of my two stations.

And… I’ve re-worked the video into a proper article for the German Anlagen Design Journal, the #17 issue.

On five pages, I’m allowed to explain the German theory and the (mostly cheap) practical solutions I used for my “Eifelburgenbahn” layout.

Note: I also really like the rest of this issue, it is one of the best in the series. If you can read German and like model railways, you might want to snag a copy for just €8 as an “Einzelheftbestellung” directly from the small publisher (Tip, #6 has my other big article: a good summary of my layout.)

iSAQB meetup: software architecture decision making in practice¶

2025-06-13

Tags: python

I attended a meetup of the Dutch iSAQB community in Utrecht (NL). The location was in the old industrial buildings of the former werkspoor train manufacturer, something I personally like :-)

(At least three people asked me during dinner whether there were any Dutch python meetups, so I’ll post the three active ones that I know of here for ease of finding them: Utrecht, Leiden and Amsterdam. And don’t forget the two one-day conferences, PyGrunn (Groningen) and pycon NL (Utrecht).)

Pygrunn keynote: Homo ludens, python and play - Daniele Procida¶

2025-05-16

Tags: pygrunn, python

(One of my summaries of the 2025 pygrunn conference in Groningen, NL).

Organizing a conference is a weird mix between it-is-not-my-job and it-is-a-lot-of-real-work.

When Daniele Procida learned python 16 years ago, he was often told that learning python is fun! And it is easy! Often when people say it is fun and easy, that you should be suspicious.

It actually wasn’t fun. There was lots of satisfaction, though. Solving problems is nice. He didn’t want to have fun and he didn’t want to be a great programmer: He just wanted to solve problems!

Fun, playful… Python early on had a bit of playfulness. Named after Monty Python, an old BBC comedic television series. Nowadays most people haven’t seen Monty Python on television and actually discover it through the programming language. How does an Indonesian teenager react to the unknown Monty Python jokes?

Same with python-the-snakes. Lots of book with snakes on the cover. Some playful, some aggressive. Some with dripping venom (that doesn’t fit with a constricting type of snake…). But try talking about “python” in African countries where Pythons actually are a menace…

His definition of humor: the surprising violation of an expectation of congruity. A momentary re-ordering of the world. It gives us a surprising delight.

Python itself is full of expectations that can be violated… He was originally baffled by the term “pythonic”: he was just wondering whether his code worked or not, he didn’t expect his code to be judged on some fit-in-with-the-locals quality like “pythonic”.

An exception means you need a rule. A holiday is something different than regular days. A “holiday from the rule”. The special depends on the normal. Fun is a “holiday from seriousness”. Then: how can “fun” be made the expectation? A constant low-level fun is not funny.

Johan Huizinga wrote “homo ludens” in 1938. Homo ludens, the playing man. See wikipedia. He says that play is the primary pre-condition of culture. But how? According to Daniele’s examples, the holiday can’t come before the work. You need to have an exception when there’s a normal. God rested on the seventh day, but that only could be rest because he actually worked for six days.

Culture? Players on a stage. Which has rules. A courtroom can be a sort of playground. A kids’ playground has some boundaries. Could there be something to Huizinga’s argument?

At work, you can receive clear outcomes: recognition, rewards. The same system offers penalties. Work is a relatively clear system. A framework of rewards, value, etc. Daniele likes his work (he works for canonical/ubuntu). He’s allowed to collaborate a lot.

What can compete with that? In the weekend or the evenings? Often what you do is unclearer. When is it finished? When did you do a good job? To do something at home, he has to get tools out of the garage and he has to clean up afterwards: just grabbing a laptop at work is much easier. Grabbing his guitar takes more work. And nobody thanks him for playing on it. It can be too much work even to take a holiday from work. To play.

Easy-to-start-with work with good feedback is always easily available…

There’s an asymmetry of performance, failure and accountability. At work, you can get negative feedback. At home, what are they going to do? Sack him? No. Why are people talking about gamifying work? They should be talking about “workifying”! That’s more attractive!

Open source: is it work or a labour of love? What about people pestering an open source project with unwanted contributions? The dance between maintainer and contributor. What when a state actor manages to get a malicious patch into a central piece of open source software (like what happened last year)?

Does this have to do with the difference between work and play? Could open source software benefit by some explicit rules, some explicit expectations? Social expectations?

Photo explanation: picture from our Harz (DE) holiday in 2023

Pygrunn: python on a tractor - Wieneke Keller, Sebastian Lenartowicz¶

2025-05-16

Tags: pygrunn, python

(One of my summaries of the 2025 pygrunn conference in Groningen, NL).

How to become an apple farmer in 5 minutes:

• You want mid-size apples, as they fetch the best price.

• You want blossom on your tree, but not too much. Otherwise the tree has to divide its water and nourishment over more apples, which makes them smaller…

They work at aurea imaging, maker of “treescout”. The treescout is a device/camera on top of a tractor that drives along the apple trees in an orchard. The device detects the tree, blossoms and other attributes of the tree. The first use case: blossom thinning to aim at the right size of apples. Blossom thinning happens with a sprayer. The blossom blossoms for just two or three weeks.

The first season they tried their project was educational. Lots or problems :-) GPS tracks that were not straight. Detected trees were not in the right location. Etcetera.

Some of the challenges:

• It is on a vehicle, so limited power.

• It is on a farm, so very limited or no connectivity.

• Agricultural standards are actually from the maritime industry. GPS that is accurate within a few meters is fine, there. But not detailed enough for locating trees…

• They were a software company, so they had to use off-the-shelf components.

The first blossom season, their architecture looked like this:

• Python monolith with multiprocessing.

• Restarting upon failure was hard due to them using unix pipes for inter-process communication.

• Poor separation of responsibilities.

The second blossom season they changed several things.

• They used k3s (a lightweight kubernetes).

• A single-node k3s cluster sitting on top of a tractor.

• K3s is responsible for the runtime environment and workload management.

• Rabbitmq for inter-process communication.

• A Kubernetes cluster really helps with rolling out quick updates.

A problem they noticed is python’s concurrency problem. There are some central components that are a bottleneck.

What they’re working on:

• ArgoCD, open cluster management, kairos (“in-place os updates”), embedded linux. They’re close to fully-automatic remote upgrades.

• More flexible hardware setup.

• Machine learning and insights on tree level. BIG amount of data for an eight-man dev team…

• Increasing the number of customers.

Photo explanation: picture from our Harz (DE) holiday in 2023

Pygrunn: team alignment, enterprise design power - Edzo A. Botjes¶

2025-05-16

Tags: pygrunn, python

(One of my summaries of the 2025 pygrunn conference in Groningen, NL).

He helps startups to design their business. He’s got more info at https://www.edzob.com/page/enterprise/ , an “enterprise design cheat sheet”. He was a consultant for a long time and started noticing patterns. He’s now in education/research and he’s focused on culture.

According to Osterwalder, success = value proposition + uniqueness + business model + timing + team alignment. In Edzo’s experience, the team alignment is often a core problem.

As a person, you have a skill set (behaviour and capabilities). As a team, you have a collective toolset: (structure, processes, data and tech). Those two are the tangible stuff. Intangible is your mindset (attitude and motivation) as a person, and the culture as a team.

UX is a social contract between an app and the user. There’s a social contract behind the interactions within the company. A culture. How do you want to collaborate? How you collaborate defines what you’re going to be building. Conway’s law. (He mentioned his talk of last year about this subject).

His wife did a PhD about the meaning behind fairy tales. For him, as a technical person, the idea of having multiple meanings of the same physical text was initially hard. What do words mean? What is their purpose? Having good conversations about the terms and words used in, for instance, your business model/motivition canvas is a good idea. Communication is key.

There was a lot more in his presentation, as it was intended as a fast-paced talk. Lots of way to look at your business or product. But in the end, team alignment could well be key. Optimizing your team members. Organizing them well. Collaboration? Facilitating? Key: have conversations. “Invest in beer and bitterballen, not in courses”.

And: you should always be looking at what can I destroy. What in your company can be stopped, removed, destroyed? You need that to change and improve.

His slides are online with lots of extra links and background information.

Photo explanation: picture from our Harz (DE) holiday in 2023

Pygrunn: how to solve a python mystery - Aivars Kalvāns¶

2025-05-16

Tags: pygrunn, python

(One of my summaries of the 2025 pygrunn conference in Groningen, NL).

Aivars pointed at https://www.brendangregg.com/linuxperf.html as a good overview of linux tools

A good start is the /proc filesystem, you can use it to gather information on processes, for instance to grab the environment used by a process:

$ cat /proc/1234455/environ || tr '\0' '\n'

The files/sockets used by a specific process:

$ ls /proc/12345/fd/*

You might have an unfindable file that takes up lots of space (like a logfile that has been deleted from a directory, but that is still open in some program). The command above will have (deleted) next to deleted files, so you can search for that string in the output to find the process that still has such a big file open.

Another handy tool: strace, it traces linux system kernel calls. You don’t even need root access if you just want to trace your own processes. An example command:

$ strace -f -ttt -o output.txt -s 1024 -p <PID>
$ strace -f -ttt -o output.txt -s 1024 -p your-new-process.sh

If your code does a system call (“read something from somewhere”), strace prints both the start and the end of the call. So you can find out exactly where something is blocking in case of an error. He mentioned https://filippo.io/linux-syscall-table/ as a good overview of the available system calls you might see in the output.

Disk IO problems? Try iostat -x to see where the IO is happening. When testing disk throughput, don’t just test huge blobs of data, but make sure to use the actual block size (often 4k or 8k).

When debugging network access, you often use ping or traceroute. But both use protocols (ICMP and UDP) that are often blocked by network admins. He suggests tcptraceroute which uses TCP and often gives a better view of reality.

With network problems, TCP_NODELAY is a possible cause. See https://brooker.co.za/blog/2024/05/09/nagle.html for more information. Read this especially when you see the magic number 40ms in your logs, or only get 25 transactions per second.

Tip: set timeouts for everything. The defaults are often a cause for hanging.

Photo explanation: picture from our Harz (DE) holiday in 2023

Overview by year¶

Statistics: charts of posts per year and per month.