Pycon NL: kedro, lessons from maintaining an open source framework - Merel Theisen¶

Tags: python, pycon

(One of my summaries of the Pycon NL one-day conference in Utrecht, NL).

Full title: leading kedro: lessons from maintaining an open source python framework.

Merel is the tech lead of the python open source framework kedro.

What is open source? Ok, the source code is publicly available for anyone to use, modify and share. But it is also a concept of sharing. Developing together. “Peer production”. It also means sharing of technical information and documentation. In the 1990s the actual term “open source” was coined. Also, an important milestone: Github was launched in 2008, greatly easing open source development.

Kedro is a python toolbox that applies software engineering principles to data science code, making it easier to go from prototype to production. Started in 2017, it was open sourced in 2019. (Note: Kedro has now been donated to the Linux foundation). This made it much easier to collaborate with others outside the original company (Quantumblack).

Open source also means maintenance challenges. It is not just code. Code is the simple part. How to attract contributors? How to get good quality contributions? What to accept/reject? How to balance quick wins with the long term vision of the project? How to make contributors come back?

What lessons did they learn?

• Importance of contributor guidance. They themselves had high standards with good programming practices. How much can you ask from new contributors? They noticed they needed to improve their documentation a lot. And they had to improve their tooling. If you want well-formatted code, you need easy tools to do the formatting, for instance. And you need to actually document your formatting guidelines :-)

• Response time is important. Response time for issues, pull requests and support. If you don’t get a timely answer, you’ll lose interest as contributor. Also: tickets need to be polished and made clearer so that new contributors can help fixing them.

• Sharing pain points is a contribution, too. More contributors and users automatically mean more feature requests. But you don’t want your project to become a Frankenstein monster… A configuration file, for instance, can quickly become too cluttered because of all the options. Sometimes you need to evolve the architecture to deal with common problems. Users will tell you what they want, but perhaps it can be solved differently.

• The importance of finding contribution models that fit. Perhaps a plugin mechanism for new functionality? Perhaps a section of the code marked “community” without the regular project’s guarantees about maintenance and longevity?

• Be patient and kind. “Open source” means “people”. Code is the easy part, people add complexity. Maintainers can be defensive and contributors can be demanding.

Unrelated photo from our 2025 holiday in Austria: Neufelden has a dam+reservoir, the water travels downstream by underground pipe to the hydropower plant. At this point the pipe comes to the surface and crosses the river on a concrete construction. Nearby, the highest road bridge in this region also crosses.