Reinout van Rees’ weblog¶

Tombi, pre-commit, prek and uv.lock¶

2026-03-18

Tags: python, django

In almost all my Python projects, I’m using pre-commit to handle/check formatting and linting. The advantage: pre-commit is the only tool you need to install. Pre-commit itself reads its config file and installs the formatters and linters you defined in there.

Here’s a typical .pre-commit-config.yaml:

default_language_version:
  python: python3

repos:
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v6.0.0
    hooks:
      - id: trailing-whitespace
      - id: end-of-file-fixer
      - id: check-yaml
        args: [--allow-multiple-documents]
      - id: check-toml
      - id: check-added-large-files
  - repo: https://github.com/astral-sh/ruff-pre-commit
    # Ruff version.
    rev: v0.15.6
    hooks:
      # Run the linter.
      - id: ruff
        args: ["--fix"]
      # Run the formatter.
      - id: ruff-format
  - repo: https://github.com/tombi-toml/tombi-pre-commit
    rev: v0.9.6
    hooks:
      - id: tombi-format
        args: ["--offline"]
      - id: tombi-lint
        args: ["--offline"]

The “tombi” at the end might be a bit curious. There’s already the build-in “check-toml” toml syntax checker, right? Well, tombi also does formatting and schema validation. And in a recent project, I handled configuration through toml files.

It was for a Django website where several geographical maps were shown, each with its own title, description, legend yes/no, etcetera. I made up a .toml configuration format so that a colleague could configure all those maps without needing to deal with the python code. I created a json schema as format specification (yes, json is funnily used for that purpose). With tombi, I could make sure the config files were valid.

Oh, and tombi has an LSP plugin, so my colleague got autocomplete and syntax help out of the box. nice.

I’m also using uv a lot. That generates an uv.lock file, in .toml format, with all the version pins. It is a toml file, but without the .toml extension. So pre-commit ignored it. Until suddenly it started complaining about the indentation. But only in a github action, not locally.

Note: the complaint about the indentation is probably correct, as there’s an issue in the uv bugtracker about changing the indentation from 4 to 2 in the lockfile.

The weird thing for me was that I pin the the versions of the plugins. So the behaviour locally and on github should be the same. Some observations:

• Running tombi from the commandline on uv.lock resulted in re-formatting to two spaces, whatever the tombi version.

• Pre-commit locally did not re-format the file, but pre-commit on the server did.

• I tried it with the new rust-based alternative for pre-commit, prek (see https://github.com/j178/prek), which did re-format uv.lock.

Some further debugging showed that pre-commit was actually skipping the uv.lock file. But apparently not on github. I did some searching in pre-commit’s source code and tombi’s pre-commit hook definition. The only relevant part there was types: [toml]. So somehow pre-commit has a definition of what a toml file is. But I couldn’t find anything.

Until I spotted that pre-commit uses identify as the means to detect file types. (Looks like a handy library, btw!). And that project had a change a couple of weeks ago that identifies uv.lock as a toml file!

• My colleague updated his pre-commit installation and yes: uv.lock was getting re-formatted.

• So: github actions had a newer version than we had.

• Weird, as I just updated my python tool install this morning. Ah: I installed it with homebrew instead of uv tool, that’s why it is still older.

Anyway: small mystery solved.

Write the docs meetup: digital sovereignty for writers - Olufunke Moronfolu¶

2026-03-05

Tags: writethedocs, python

(One of my summaries of the Amsterdam *write the docs* meetup).

Full title: digital sovereignty for writers: your data, your decisions. Olufunke Moronfolu has her website at https://writerwhocodes.com/ .

“Digital sovereignty is the ability to have control over your own digital destiny: the data, hardware and software that you rely on and create” (quote from the World Economic Forum).

What do writers want? Mostly: to be read. For this you could for instance start looking for (commercial) blogging platforms, searching for the best one. And after a while you start looking for a different one. On and on. You can run into problems. Substack might ban your newsletter. A google workspace domain being blocked. A Medium story getting deleted without feedback.

Tim Berners-Lee intended for the web to be universal and open. But now it is mostly a collection of isolated silos.

There are some questions you can ask yourself to test your sovereignty. If your current platform deletes your account, is your content completely lost? Second question: can you export your work in some portable format (like markdown).

If you are a technical writer, you have to do the test twice. Once for your own content and once for your company’s documentation.

Own your content. Most sovereign for your own website/blog would be hugo/jekyll or other static generators. In the middle are (self-hosted?) wordpress sites. Least sovereign is something like linkedin/medium/substack. For company content, confluence/notion would be least sovereign. Wiki.js/bookstack middle. The best is docs as code like some markdown in git.

So: review the platform’s policy. What is the ease of export? Do you have control? What’s the stability? Do you have an identity there? Perhaps even a domain?

Own your identity. Having your own domain is best. If you’re some-platform.com/name, your identity goes away if the site disappears.

Decide how to share. Sovereign would be an email list, an RSS feed or something like the POSSE approach (Publish (on your) Own Site, Syndicate Elsewhere).

Build for the future: build something. Start. It doesn’t have to be perfect. Your own domain name and a single static page is already much more sovereign than a million followers on a site that could vanish tomorrow.

If you want to do more, join the “independent web” (indieweb, https://indieweb.org) movement.

Personal note: I’ve got my own domain. This is a blog entry that ends up in an RSS/atom feed. The site is .rst files in a git repo. Statically generated with Sphinx. So: yeah, pretty sovereign :-)

Write the docs meetup: developers documentation, your hidden strength - Frédéric Harper¶

2026-03-05

Tags: writethedocs, python

(One of my summaries of the Amsterdam *write the docs* meetup).

If you have a product, you need good developer documentation. “It is an integral part of your product: one cannot exist without the other”. You might have the best product, but if people don’t know how to use it, it doesn’t matter.

What he tells developers: good documentation reduces support tickets and angry customers. You should be able to “sell” good documentation to your company: it saves money and results in more sales.

Some notes on documentation contents:

• You need a search function. The first thing you need to add.

• Think about John Snow (game of thrones): “you know nothing, John Snow”. Be detailed in your instructions, they’ll need it. Start with the assumption that the user knows nothing about your program. Advanced users can easily skip those parts.

• Have a proper architecture/structure. Simply having a “home” link to get back to the start already helps. Add a “getting started” section with step-by-step instructions to get something simple running. And detailed how-to guides where you go into depth.

• Show a table of contents of the current page.

• Keep the docs of previous versions available.

• Take great screenshots. Docs should have great quality and it especially shows in the screenshots.

• Don’t show off your language skills too much. Keep the language simple. Not everyone will have your documentation’s language as their native language.

• Test the code in your documentation! There’s nothing more irritating than errors in example code. And keep it up to date. Especially watch out when the software gets updated. Do you give your documentation time to get updated?

Some extra notes:

• Make your docs accessible for people with disabilities.

• Are your docs fast? Load times help you get ranked higher in search engines.

• Some people read your documentation on their phones: does it work there?

• Try to make your docs open source. You might get an occasional fix. And perhaps more feedback.

Python Leiden meetup: PR vs ROC curves, which to use - Sultan K. Imangaliyev¶

2026-01-22

Tags: python, pun

(One of my summaries of the Python Leiden meetup in Leiden, NL).

Precision-recall (PR) versus Receiver Operating Characteristics (ROC) curves: which one to use if data is imbalanced?

Imbalanced data: for instance when you’re investigating rare diseases. “Rare” means few people have them. So if you have data, most of the data will be of healthy people, there’s a huge imbalance in the data.

Sensitivity versus specificity: sensitive means you find most of the sick people, specificity means you want as few false negatives and false positives as possible. Sensitivity/specificity looks a bit like precision/recall.

• Sensitivity: true positive rate.

• Specificity: false positive rate

If you classify, you can classify immediately into healthy/sick, but you can also use a probabilistic classifier which returns a chance (percentage) that someone can be classified as sick. You can then tweak which threshold you want to use: how sensitive and/or specific do you want to be?

PR and ROC curves (curve = graph showing the sensitivity/specificity relation on two axis) are two ways of measuring/visualising the sensitivity/specificity relation. He showed some data: if the data is imbalanced, PR is much better at evaluating your model. He compared balanced and imbalanced data with ROC and there was hardly a change in the curve.

He used scikit-learn for his data evaluations and demos.

Python Leiden meetup: PostgreSQL + Python in 2026 – Aleksandr Dinu¶

2026-01-22

Tags: python, pun

(One of my summaries of the Python Leiden meetup in Leiden, NL).

He’s going to revisit common gotchas of Python ORM usage. Plus some Postgresql-specific tricks.

ORM (object relational mappers) define tables, columns etc using Python concepts: classes, attributes and methods. In your software, you work with objects instead of rows. They can help with database schema management (migrations and so). It looks like this:

class Question(models.Model):
    question = models.Charfield(...)
    answer = models.Charfield(...)

You often have Python “context managers” for database sessions.

ORMs are handy, but you must be beware of what you’re fetching:

# Bad, grabs all objects and then takes the length using python:
questions_count = len(Question.objects.all())
# Good: let the database do it,
# the code does the equivalent of "SELECT COUNT(*)":
questions_count = Question.objects.all().count()

Relational databases allow 1:M and N:M relations. You use them with JOIN in SQL. If you use an ORM, make sure you use the database to follow the relations. If you first grab the first set of objects and then grab the second kind of objects with python, your code will be much slower.

“Migrations” generated by your ORM to move from one version of your schema to the next are real handy. But not all SQL concepts can be expressed in an ORM. Custom types, stored procedures. You have to handle them yourselves. You can get undesired behaviour as specific database versions can take a long time rebuilding after a change.

Migrations are nice, but they can lead to other problems from a database maintainer’s point of view, like the performance suddenly dropping. And optimising is hard as often you don’t know which server is connecting how much and also you don’t know what is queried. Some solutions for postgresql:

• log_line_prefix = '%a %u %d" to show who is connecting to which database.

• log_min_duration_statement = 1000 logs every query taking more than 1000ms.

• log_lock_waits = on for feedback on blocking operations (like migrations).

• Handy: feedback on the number of queries being done, as simple programming errors can translate into lots of small queries instead of one faster bigger one.

If you’ve found a slow query, run that query with EXPLAIN (ANALYZE, BUFFERS) the-query. BUFFERS tells you how many pages of 8k the server uses for your query (and whether those were memory or disk pages). This is so useful that they made it the default in postgresql 18.

Some tools:

• RegreSQL: performance regression testing. You feed it a list of queries that you worry about. It will store how those queries are executed and compare it with the new version of your code and warn you when one of those queries suddenly takes a lot more time.

• Squawk: tells you (in CI, like github actions) which migrations are backward-incompatible or that might take a long time.

• You can look at one of the branching tools: aimed at getting access to production databases for testing. Like running your migration against a “branch”/copy of production. There are several tricks that are used, like filesystem layers. “pg_branch” and “pgcow” are examples. Several DB-as-a-service products also provide it (Databricks Lakebase, Neon, Heroku, Postgres.ai).

Ansible-lint pre-commit problem + “fix”¶

2025-11-19

Tags: python, django

I’m used to running pre-commit autoupdate regularly to update the versions of the linters/formatters that I use. Especially when there’s some error.

For example, a couple of months ago, there was some problem with ansible-lint. You have an ansible-lint, ansible and ansible-core package and one of them needed an upgrade. I’d get an error like this:

ModuleNotFoundError: No module named 'ansible.parsing.yaml.constructor'

The solution: pre-commit autoupdate, which grabbed a new ansible-lint version that solved the problem. Upgrading is good.

But… little over a month ago, ansible-lint pinned python to 3.13 in the pre-commit hook. So when you update, you suddenly need to have 3.13 on your machine. I have that locally, but on the often-used “ubuntu latest” (24.04) github action runner, only 3.12 is installed by default. Then you’d get this:

[INFO] Installing environment for https://github.com/pre-commit/pre-commit-hooks.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
[INFO] Installing environment for https://github.com/astral-sh/ruff-pre-commit.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
[INFO] Installing environment for https://github.com/ansible-community/ansible-lint.git.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
An unexpected error has occurred: CalledProcessError: command:
  ('/opt/hostedtoolcache/Python/3.12.12/x64/bin/python', '-mvirtualenv',
  '/home/runner/.cache/pre-commit/repomm4m0yuo/py_env-python3.13', '-p', 'python3.13')
return code: 1
stdout:
    RuntimeError: failed to find interpreter for Builtin discover of python_spec='python3.13'
stderr: (none)
Check the log at /home/runner/.cache/pre-commit/pre-commit.log
Error: Process completed with exit code 3.

Ansible-lint’s pre-commit hook needs 3.10+ or so, but won’t accept anything except 3.13. Here’s the change: https://github.com/ansible/ansible-lint/pull/4796 (including some comments that it is not ideal, including the github action problem).

The change apparently gives a good error message to people running too-old python versions, but it punishes those that do regular updates (and have perfectly fine non-3.13 python versions). A similar pin was done in “black” and later reverted (see the comments on this issue) as it caused too many problems.

Note: this comment gives some of the reasons for hardcoding 3.13. Pre-commit itself doesn’t have a way to specify a minimum Python version. Apparently old Python version cans lead to weird install errors, though I haven’t found a good ticket about that in the issue tracker. The number of issues in the tracker is impressively high, so I can imagine such a hardcoded version helping a bit.

Now on to the “fix”. Override the language_version like this:

- repo: https://github.com/ansible-community/ansible-lint.git
  hooks:
    - id: ansible-lint
      language_version: python3  # or python3.12 or so

If you use ansible-lint a lot (like I do), you’ll have to add that line to all your (django) project repositories when you update your pre-commit config…

I personally think this pinning is a bad idea. After some discussion in issue 4821 I created a sub-optimal proposal to at least setting the default to 3.12, but that issue was closed&locked because I apparently “didn’t search the issue tracker”.

Anyway, this blog post hopefully helps people adjust their many pre-commit configs.

Python Leiden (NL) meetup summaries¶

2025-11-13

Tags: python, pun

My summaries from the sixth Python meetup in Leiden (NL).

PyUtrecht (NL) meetup: the future of Python typing - Victorien Plot¶

2025-11-04

Tags: python, pun

(One of my summaries of the PyUtrecht meetup in Utrecht, NL).

Note: Victorien is currently the number one person maintaining Pydantic. Pydantic is basically “dataclasses with validation”.

There was a show of hands: about 70% uses type hints. Type hints has been around since python 3.5. There have been improvements during the years like str|None instead of Union[str, None] in 3.10, for instance.

Something I didn’t know: you can always introspect type hints when running your python code: typing.get_type_hints(my_func).

Getting typing-related changes into Python takes a lot of work. You need to implemeent the changes in CPython. You have to update the spec. And get it supported by the major type checkers. That’s really a difference to typescript, as typing is built-in from the start, there.

Something that helps typing in the future is 3.15’s lazy from xxx import yyy import.

There’s an upcoming PEP 764, “inline typed dictionaries”:

def get_movie() -> {"name": str, "year": int}:
    # At least something like this ^^^, I can't type that quickly :-)
    ...

He has some suggestions for a new syntax, using something like <{ .... }>, but getting a syntax change into Python takes a lot of talking and a really solid proposal.

PyUtrecht (NL) meetup: streaming telemetry (network monitoring with gRPC and gNMI) - Maurice Stoof¶

2025-11-04

Tags: python, pun

(One of my summaries of the PyUtrecht meetup in Utrecht, NL).

“From SNMP to gRPC”. Maurice is working on network automation. (The link goes to his github account, the presentation’s demo code is there).

SNMP, the Simple Network Monitoring Protocol, has been the standard for network monitoring since 1980. But its age is showing. It is polling-pased, which is wasteful. The mechanism will continually poll the endpoints. It is like checking for new messages on your phone every minute instead of relying on push messaging.

The better way is streaming telemetry, the push model. He uses gRPC, “A high performance, open source universal RPC framework” and gNMI, “gRPC Network Management Interface”.

You can ask for capabilities: used in the discovery phase. Get is a simple one-time request for a specific value. With set you can do a bit of configuring. The magic is in subscribe: it creates a persistent connection, allowing the device to continuously stream data back to the client (according to the settings done with “set”).

(For the demo, he use pyGMNI, a handy python library for gNMI.)

When to use streaming?

• With high-frequency monitoring. If you need data more frequent than once every 10 seconds.

• When you need real-time alerting.

• Large-scale deployments. With lots of devices, polling efficiency starts to pay off.

SNMP is still fine when you have small setup and hign frequency isn’t really needed.

Pycon NL: don’t panic, a developer’s guide to security - Sebastiaan Zeeff¶

2025-10-16

Tags: python, pycon

(One of my summaries of the Pycon NL one-day conference in Utrecht, NL).

He showed a drawing of Cornelis “wooden leg” Jol, a pirate from the 17th century from Sebastiaan’s hometown. Why is he a pirate? He dresses like one, has a wooden leg, murders people like pirate and even has a parrot, so he’s probably a pirate. For python programmers used to duck typing, this is familiar.

The 17th century, the Netherlands were economically wealthy. And had a big sea-faring empire. But they wanted a way to expand their might without paying for it. So… privatization to the rescue. You give pirates a vrijbrief, a government letter saying they’ve got some kind of “permission” from the Dutch government to rob and pillage and kill everybody as long it aren’t Dutch people and ships. A privateer.So it looks like a pirate and behaves like a pirate, but it isn’t technically a real pirate.

Now on to today. There are a lot of cyber threats. Often state-sponsored. You might have a false sense of security in working for a relatively small company instead of for a juicy government target. But… privateers are back! Lots of hacking companies have coverage of governments - as long as they hack other countries. And hacking small companies can also be profitable.

“I care about security”. Do you really? What do real security people think? They think developers don’t really pay much attention to it. Eye-roll at best, disinterest at worst. Basically, “it is somebody else’s problem”.

What you need is a security culture. A buy-in at every level. You can draw an analogy with safety culture at physically dangerous companies like petrochemical. So: you as developer, should argue for security with your boss. You are a developer, so you have a duty to speak up. Just like a generic employee at a chemical plant has the duty to speak when seeing something risky.

You don’t have to become a security export (on top of everything else), but you do have to pay attention. Here are some pointers:

• “Shift left”. A term meaning you have to do it earlier rather than later. Don’t try to secure your app just before shipping, but take it into account from the beginning. Defense in depth.

• “Swiss cheese model”. You have multiple layers in your setup. Every layer only needs one hole for the total to be penetrated.

• Learn secure design principles. “Deny by default”, “fail securely”, “avoid security by obscurity”, “minimize your attack surface”, etc. Deny by default is a problem in the python world. We’re beginner-friendly, so often everything is open…

• Adopt mature security practices. Ignore ISO 27001, that’s too hard to understand. Look at OWASP instead. OWASP DevSecOps maturity model (“pin your artifacts”, for instance).

• Know common vulnerabilities. Look at the popular “top 10” lists. Today, SQL injection still makes victims…

Unrelated photo from our 2025 holiday in Austria: center of Neufelden, nicely restored and beautifully painted.

Overview by year¶

Statistics: charts of posts per year and per month.