Reinout van Rees’ weblog

Automation for better behaviour

2015-09-02

Tags: django, python, nelenschuurmans

Now... that’s a provocative title! In a sense, it is intended that way. Some behaviour is better than other behaviour. A value judgment! In the Netherlands, where I live, value judgments are suspect. If you have a comment on someone’s behaviour, a common question is whether you’re “better” than them. If you have a judgment, you apparently automatically think you’ve got a higher social status or a higher moral standing. And that’s bad, apparently.

Well, I despise such thinking :-)

Absolute values

I think there are absolutes you can refer to, that you can compare to. Lofty goals you can try to accomplish. Obvious truths (which can theoretically be wrong...) that are recognized by many.

Nihilism is fine, btw. If you’re a pure nihilist: I can respect that. It is an internally-logical viewpoint. Only you shouldn’t complain if some other nihilist cuts you to pieces if that suits his purely-individual nihilistic purposes.

So for practical purposes I’m going to assume there’s some higher goal/law/purpose/whatever that we should attain.

Take programming in python. PEP 8, python’s official style guide is recognized by most of the python programmers as the style guide they should adhere to. At least, nobody in my company complained if I adjusted/fixed their code to comply with PEP 8. And the addition of bin/pep8 in all of our software projects to make it easy to check for compliance didn’t raise any protests. Pyflakes is even clearer, as it often points at real errors of obvious omissions.

For django projects, possible good things include:

  • Sentry integration for nicely-accessible error logging.
  • Using a recent and supported django version. So those 1.4 instances we still have at my workplace should go the way of the dodo.
  • Using proper releases instead of using the latest master git checkout.
  • Using migrations.
  • Tests.

Automation is central to good behaviour

My take on good behaviour is that you should either make it easy to do the good thing or you should make non-good behaviour visible.

As an example, take python releases. As a manager you can say “thou shalt make good releases”. Oh wow. An impressive display of power. It reminds me of a certain SF comic where, to teach them a lesson, an entire political assembly was threathened with obliteration from orbit. Needless to say, the strong words didn’t have a measurable effect.

You can say the same words at a programmer meeting, of course. “Let’s agree to make proper releases”. Yes. Right.

What do you have to do for a proper release?

  • Adjust the version in setup.py from 1.2.dev.0 to 1.2.
  • Record the release date in the changelog.
  • Tag the release.
  • Update the version number in setup.py to 1.3.dev.0.
  • Add a new header for 1.3 in the changelog.

Now... That’s quite an amount of work. If I’m honest, I trust about 40% of my colleagues to make that effort every time they release a package.

There is a better way. Those very same colleagues can be relied on to make perfect releases all the time if all they have to do is to call bin/fullrelease and press ENTER a few times to do all of the above automatically. Thanks to zest.releaser.

Zest.releaser makes it easier and quicker to make good releases than it is to make bad/quick/sloppy releases by hand.

Further examples

Now... here are some further examples to get you thinking.

All of our projects are started with “nensskel”, a tool to create a skeleton for a new project (python lib, django app, django site). It uses “paste script”; many people now use “cookie cutter”, which serves the same purpose.

  • For all projects, a proper test setup is included. You can always run bin/test and your test case will run. You only have to fill it in.

  • bin/fullrelease, bin/pep8, bin/pyflakes: if you haven’t yet installed those programs globally, how easy can I make it for you to use them???

  • If you want to add documentation, sphinx is all set up for you. The docs/source/ directory is there and sphinx is automatically run every time you run buildout.

  • The README.rst has some easy do-this-do-that comments in there for when you’ve just started your project. Simple quick things like “add your name in the setup.py author field”. And “add a one-line summary to the setup.py and add that same one to the github.com description”.

    I cannot make it much easier, right?

    Now... quite some projects still have this TODO list in their README.

Conclusion: you need automation to enable policy

You need automation to enable policy, but even that isn’t enough. I cannot possibly automatically write a one-line summary for a just-generated project. So I have to make do with a TODO note in the README and in the setup.py. Which gets disregarded.

If even such simple things get disregarded, bigger things like “add a test” and “provide documentation” and “make sure there is a proper release script” will be hard to get right. I must admit to not always adding tests for functionality.

I’ll hereby torture myself with a quote. “Unit testing is for programmers what washing your hands is for doctors before an operation”. It is an essential part of your profession. If you go to the hospital, you don’t expect to have to ask your doctor to disinfect the hands before the operation. That’s expected. Likewise, you shouldn’t expect your clients to explicitly ask you for software tests: those should be there by default!

Again, I admit to not always adding tests. That’s bad. As a professional software developer I should make sure that at least 90% test coverage is considered normal at my company. In the cases where we measure it, coverage is probably around 50%. Which means “bad”. Which also means “you’re not measuring it all the time”. 90% should also be normal for my own code and I also don’t always attain that.

Our company-wide policy should be to get our test coverage at least to 90%. Whether or not if that’s our policy, we’ll never make 90% if we don’t measure it.

And that is the point I want to make. You need tools. You need automation. If you don’t measure your test coverage, any developer or management policy statement will be effectively meaningless. If you have a jenkins instance that’s in serious neglect (70% of the projects are red), you don’t effectively have meaningful tests. Without a functioning jenkins instance (or travis-ci.org), you cannot properly say you’re delivering quality software.

Without tooling and automation to prove your policy, your policy statements are effectively worthless. And that’s quite a strong value statement :-)

Lessons learned from buildout a django site with a reactjs front-end

2015-08-31

Tags: django, python, nelenschuurmans

My colleague Gijs Nijholt just posted his blog entry lessons learned from building a larger app with React.js, which is about the javascript/reactjs side of a django website we both (plus another colleague) recently worked on.

Simplified a bit, the origin is a big pile of measurement data, imported from csv and xml files. Just a huge list of measurements, each with a pointer to a location, parameter, unit, named area, and so on. A relatively simple data model.

The core purpose of the site is threefold:

  • Import, store and export the data. Csv/xml, basically.
  • Select a subset of the data.
  • Show the subset in a table, on a map or visualized as graphs.

The whole import, store, export is where Django shines. The model layer with its friendly and powerful ORM works fine for this kind of relational data. With a bit of work, the admin can be configured so that you can view and edit the data.

Mostly “view” as the data is generally imported automatically. Which means you discover possible errors like “why isn’t this data shown” and “why is it shown in the wrong location”. With the right search configuration and filters, you can drill down to the offending data and check what’s wrong.

Import/export works well with custom django management commands, admin actions and celery tasks.

Now on to the front-end. With the basis being “select a subset” and then “view the subset”, I advocated a simple interface with a sidebar. All selection would happen in the sidebar, the main content area would be for viewing. And perhaps some view-customization like sorting/filtering in a table column or adjusting graph colors. This is the mockup I made of the table screen:

http://reinout.vanrees.org/images/2015/efcis_tabel_mockup.png

In the sidebar you can select a period, locations/location groups and parameters. The main area is for one big table. (Or a map or a graph component).

To quickly get a first working demo, I initially threw together three django views, each with a template that extended one base template. Storing the state (=your selection) as a dict in the django session on the server side. A bit of bootstrap css and you’ve got a reasonable layout. Enough, as Gijs said in his blog entry, to sell the prototype to the customer and get the functional design nailed down.

Expanding the system. The table? That means javascript. And in the end, reactjs was handy to manage the table, the sorting, the data loading and so on. And suddenly state started spreading. Who manages the state? The front-end or the back-end? If it is half-half, how do you coordinate it?

Within a week, we switched the way the site worked. The state is now all on the client side. Reactjs handles the pages and the state and the table and the graph and the map. Everything on one side (whether client-side or server-side) is handiest.

Here’s the current table page for comparison with the mockup shown above:

http://reinout.vanrees.org/images/2015/efcis_tabel_website.png

Cooperation is simple this way. The front-end is self-contained and simply talks to a (django-rest-framework) REST django backend. State is on the client (local storage) and the relevant parameters (=the selection) are passed to the server.

Django rest framework’s class based views came in handy. Almost all requests, whether for the map, the table or the graph, are basically a filter on the same data, only rendered/serialized in a different way. So we made one base view that grabs all the GET/POST parameters and uses them for a big django query. All the methods of the subclassing views can then use those query results.

A big hurray for class based views that make it easy to put functionality like this in just one place. Less errors that way.

Some extra comments/tips:

  • Even with a javascript front-end, it is still handy to generate the homepage with a Django template. That way, you can generate the URLs to the various API calls as data attributes on a specific element. This prevents hard-coding in the javascript code:

    <body>
      <!-- React app renders itself into this div -->
      <div id="efcis-app"></div>
    
      <script>
        // An object filled with dynamic urls from Django (for XHR data retrieval in React app)
        var config = {};
        config.locationsUrl = '{% url 'efcis-locaties-list' %}';
        config.meetnetTreeUrl = '{% url 'efcis-meetnet-tree' %}';
        config.parameterGroupTreeUrl = '{% url 'efcis-parametergroep-tree' %}';
        config.mapUrl = '{% url 'efcis-map' %}';
        window.config = config;
      </script>
      ...
    </body>
  • Likewise, with django staticfiles and staticfiles’ ManifestStaticFilesStorage, you get guaranteed unique filenames so that you can cache your static files forever for great performance.

Lessons learned?

  • Splitting up the work is easy and enjoyable when there’s a REST back-end and a javascript front-end and if the state is firmly handled by the front-end. Responsibility is clearly divided that way and adding new functionality is often a matter of looking where to implement it. Something that’s hard in javascript is sometimes just a few lines of code in python (where you have numpy to do the calculation for you).

    Similarly, the user interface can boil down complex issues to just a single extra parameter send to the REST API, making life easier for the python side of things.

  • When you split state, things get hard. And in practice that, in my experience, means the javascript front-end wins. It takes over the application and the django website is “reduced” to an ORM + admin + REST framework.

    This isn’t intended as a positive/negative value statement, just as an observation. Though a javascript framework like reactjs can be used to just manage individual page elements, often after a while everything simply works better if the framework manages everything, including most/all of the state.

Easy maintainance: script that prints out repair steps

2015-08-25

Tags: python, django, nelenschuurmans

At my work we have quite a number of different sites/apps. Sometimes it is just a regular django website. Sometimes django + celery. Sometimes it also has extra django management commands, running from cronjobs. Sometimes Redis is used. Sometimes there are a couple of servers working together....

Anyway, life is interesting if you’re the one that people go to when something is (inexplicably) broken :-) What are the moving parts? What do you need to check? Running top to see if there’s a stuck process running at 100% CPU. Or if something eats up all the memory. df -h to check for a disk that’s full. Or looking at performance graphs in Zabbix. Checking our “sentry” instance for error messages. And so on.

You can solve the common problems that way. Restart a stuck server, clean up some files. But what about a website that depends on background jobs, run periodically from celery? If there are 10 similar processes stuck? Can you kill them all? Will they restart?

I had just such a problem a while ago. So I sat down with the developer. Three things came out of it.

  • I was told I could just kill the smaller processes. They can be re-run later. This means it is a good, loosely-coupled design: fine :-)

  • The README now has a section called “troubleshooting” with a couple of command line examples. For instance the specific celery command to purge a specific queue that’s often troublesome.

    This is essential! I’m not going to remember that. There are too many different sites/apps to keep all those troubleshooting commands in my head.

  • A handy script (bin/repair) that prints out the commands that need to be executed to get everything right again. Re-running previously-killed jobs, for instance.

The script grew out of the joint debugging session. My colleague was telling me about the various types of jobs and celery/redis queues. And showing me redis commands that told me which jobs still needed executing. “Ok, so how do I then run those jobs? What should I type in?”

And I could check serveral directories to see which files were missing. Plus commands to re-create them. “So how am I going to remember this?”

In the end, I asked him if he could write a small program that did all the work we just did manually. Looking at the directories, looking at the redis queue, printing out the relevant commands?

Yes, that was possible. So a week ago, when the site broke down and the colleague was away on holiday, I could kill a few stuck processes, restart celery and run bin/repair. And copy/paste the suggested commands and execute them. Hurray!

So... make your sysadmin/devops/whatever happy and...

  • Provide a good README with troubleshooting info. Stuff like “you can always run bin/supervisorctl restart all without everything breaking. Or warnings not to do that but to instead do xyz.
  • Provide a script that prints out what needs doing to get everything OK again.

Runs on python 3: checkoutmanager

2015-08-24

Tags: python, django

Checkoutmanager is a five-year old tool that I still use daily. The idea? A simple ~/.checkoutmanager.cfg ini file that lists your checkouts/clones. Like this (only much longer):

[local]
vcs = git
basedir = ~/local/
checkouts =
    git@github.com:nens/syseggrecipe.git
    git@github.com:buildout/buildout.git
    git@github.com:reinout/reinout-media.git
    git@github.com:rvanlaar/djangorecipe.git

[svn]
vcs = svn
basedir = ~/svn/
checkouts =
    svn+ssh://reinout@svn.zope.org/repos/main/z3c.recipe.usercrontab/trunk

In the morning, I’ll normally do a checkoutmanager up and it’ll go through the list and do svn up, git pull, hg pull -u, depending on the version control system. Much better than going though a number of them by hand!

Regularly, I’ll do checkoutmanager st to see if I’ve got something I still need to commit. If you just work on one project, no problem. But if you need to do quick fixes on several projects and perhaps also store your laptop’s configuration in git... it is easy to forget something:

$ checkoutmanager st
/Users/reinout/vm/veertien/efcis-site
 M README.rst

And did you ever commit something but forgot to push it to the server? checkoutmanager out tells you if you did :-)

Porting to python 3. The repo was originally on bitbucket, but nowadays I keep having to look all over my screen, looking for buttons, to get anything done there. I’m just too used to github, it seems. So after merging a pull request I finally got down to moving it to github.

I also copied over the issues and added one that told me to make sure it runs on python 3, too. Why? Well, it is the good thing to do. And... we had a work meeting last week where we said that ideally we’d want to run everything on python 3.

Two years ago I started a django site with python 3. No real problems there. I had to fix two buildout recipes myself. And the python LDAP package didn’t work, but I could work around it. And supervisord didn’t run so I had to use the apt-get-installed global one. For the rest: fine.

Recently I got zest.releaser to work on python 3 (that is: someone else did most of the hard work, I helped getting the pull request properly merged :-) ). For that, several test dependencies needed to be fixed for python 3 (which, again, someone else did). Checkoutmanager had the same test dependencies, so getting the test machinery to run was just a matter of updating dependencies.

What had to be done?

  • print 'something' is now a function: print('something'). Boring work, but easy.

  • Some __future__ imports, mostly for the print function and unicode characters.

  • Oh, and setting up travis-ci.org testing. Very easy to get both python 2.7 and 3.4 testing your software that way. Otherwise you keep on switching back/forth between versions yourself.

    (There’s also ‘tox’ you can use for local multi-python-version testing in case you really really need that all the time, I don’t use it myself though.)

  • Some from six.moves import xyz to work around changed imports between 2 and 3. Easy peasy, just look at the list in the documentation.

  • It is now try... except SomeError as e instead of try... except SomeError, e. The new syntax already works in 2.7, so there’s no problem there.

  • The one tricky part was that checkoutmanager uses doctests instead of “regular” tests. And getting string comparison/printing right on both python 2 and 3 is a pain. You need an ugly change like this one to get it working. Bah.

    But: most people don’t use doctests, so they won’t have this problem :-)

  • The full list of changes is in this pull request: https://github.com/reinout/checkoutmanager/pull/9 .

  • A handy resource is http://python3porting.com/problems.html . Many common problems are mentioned there. Including solution.

    Django’s porting tips at https://docs.djangoproject.com/en/1.8/topics/python3/ are what I recommended to my colleagues as a useful initial guide on what to do. Sane, short advice.

Anyway... Another python 3 package! (And if I’ve written something that’s still used but that hasn’t been ported yet: feel free to bug me or to send a pull request!)

Buildout and Django: djangorecipe updated for gunicorn support

2015-06-16

Tags: django

Most people in the Django world probably use pip to install everything. I (and the company were I work, Nelen & Schuurmans) use buildout instead. If there are any other buildout users left outside of zope/plone, I’d love to hear it :-)

First the news about the new update, after that I’ll add a quick note about what’s good about buildout, ok?

Djangorecipe 2.1.1 is out. The two main improvements:

  • Lots of old unused functionality has been removed. Project generation, for instance. Django’s own startproject is good enough right now. And you can also look at cookiecutter. Options like projectegg and wsgilog are gone as they’re not needed anymore.

  • The latest gunicorn releases didn’t come with django support anymore. You used to have a bin/django run_gunicorn (or python manage.py run_gunicorn) management command, but now you just have to run bin/gunicorn yourproject.wsgi. And pass along an environment variable that points at your django settings.

    With the latest djangorecipe, you can add a scripts-with-settings = gunicorn option and it’ll create a bin/gunicorn-with-settings script for you that sets the environment variable automatically. Handy!

Advantage of buildout. To me, the advantage of buildout is threefold:

  • Buildout is more fool-proof. With pip/virtualenv you should remember to activate the virtualenv. With buildout, the scripts themselves make sure the correct sys.path is set.

    With pip install something you shouldn’t forget the -r requirements.txt option. With buildout, the requirement restrictions (“versions”) are applied automatically.

    With pip, you need to set the django settings environment variable in production and staging. With buildout, it is just bin/django like in development: it includes the correct reference to the correct settings file automatically.

    There just isn’t anything you can forget!

  • Buildout is extensible. You can extend it with “recipes”. Like a django recipe that helps with the settings and so. Or a template recipe that generates an ngnix config based on a template with the django port and hostname already filled in from the buildout config file. Or a sysegg recipe that selectively injects system packages (=hard to compile things like numpy, scipy, netcdf4).

  • Buildout “composes” your entire site, as far as possible. Pip “just” grabs your python packages. Buildout can also build NPM and run grunt to grab your javascript and can automatically run bin/django collectstatic -y when you install it in production. And generate an nginx/apache file based on your config’s gunicorn port. And generate a supervisord config with the correct gunicorn call with the same port number.

Of course there are drawbacks:

  • The documentation is definitively not up to the standards of django itself. Actually, I don’t really want to point at the effectively unmaintained main documentation site at http://www.buildout.org/.... You need some experience with buildout to be able to get and keep it working.
  • Most people use pip.

Why do I still use it?

  • The level of automation you can get with buildout (“composability”) is great.
  • It is fool-proof. One bin/buildout and everything is set up correctly. Do you trust every colleague (including yourself) to remember 5 different commands to set up a full environment?
  • If you don’t use buildout, you have to use pip and virtualenv. And a makefile or something like that to collect all the various parts. Or you need ansible even to set up a local environment.
  • Syseggrecipe makes it easy to include system packages like numpy, scipy, mapnik and so on. Most pip-using web developers only need a handful of pure python packages. We’re deep into GIS and numpy/gdal territory. You don’t want to compile all that stuff by hand. You don’t want to have to keep track of all the development header file packages!

So... hurray for buildout and for the updated djangorecipe functionality! If you still use it, please give me some feedback at reinout@vanrees.org or in the comments below. I’ve removed quite some old functionality and I might have broken some usecases. And buildout/django ideas and thoughts are always welcome.

Formatting python log messages

2015-06-05

Tags: python

I see three conflicting styles of log formatting in most of the code I come across. Basically:

import logging

logging = logging.getLogger(__name__)

logger.info("%s went %s wrong", 42, 'very')
logger.info("{} went {} wrong".format(42, 'very'))
logger.info("%s went %s wrong" % (42, 'very'))

I looked at the official PEP 282 and at the official docs.

With the help of someone’s stackoverflow question I think I understand it now.

  • Use the first version of the three examples. So:

    • The actual log message with the old, well-known %s (and %d, %f, etc) string formatting indicators.
    • As many extra arguments as you have %s-thingies in your string.
  • Don’t use the second and third example, as both of them format the string before it gets passed to the logger. So even if the log message doesn’t need to be actually logged somewhere, the full string gets created.

    The first example only gets passed a string and some extra arguments and only turns it into a real full string for in your logfile if it actually gets logged. So if you only display WARN level and higher, your DEBUG messages don’t need to be calculated.

  • There is no easy way to use the first example with {} instead of %s.

So: use the logger.info("%s went %s wrong", 42, 'very') form.

(Unless someone corrects me, of course)

Djangocon sprint: zest.releaser 5.0 runs on python 3

2015-06-05

Tags: djangocon, python

Good news! zest.releaser supports python 3.3+ now!

Now... what is zest.releaser? zest.releaser takes care of the boring release tasks for you. So: make easy, quick and neat releases of your Python packages. The boring bit?

  • Change the version number (1.2.dev0 -> 1.2 before releasing and 1.2 -> 1.3.dev0 after releasing).
  • Add a new heading in your changelog. Preferrably record the release date in the changelog, too.
  • svn/git/bzr/hg tag your project.
  • Perhaps upload it to pypi.

Zest.releaser takes care of this for you! Look at the docs on readthedocs.org for details. The short page on our assumptions about a releasable python project might be a good starting point.

Now on to the python 3.3 support. It was on our wish list for quite some time and now Richard Mitchell added it, kindly sponsored by isotoma! We got a big pull request last week and I thought it would be a good idea to try and merge it at the djangocon sprint. Reason? There are people there who can help me with python 3.

Pull request 101 was the important one. So if you want to get an idea of what needs to be done on a python package that was first released in 2008, look at that one :-)

Long-time zest.releaser users: read Maurits’ list of improvements in 4.0, released just two weeks ago.

Running on python 3 is enough of a change to warrant a 5.0 release, though. Apart from python 3, the biggest change is that we now test the long description (what ends up on your pypi page) with the same readme library that pypi itself uses. So no more unformatted restructured text on your pypi page just because there was a small error somewhere. Handy. Run longtest to check it. If it renders OK, it’ll be opened in your webbrowser, always handy when you’re working on your readme.

Djangocon: React.js workshop - Maik Hoepfel

2015-06-04

Tags: djangocon

He compares react.js to angular. Angular basically want to take over your whole page. Multiple apps on one page, mixed with some jquery, isn’t really possible. You also have to learn a lot of terminology. It is a full stack framework. And angular 2.0 won’t have a migration path from angular 1.x...

React.js only does one thing and does it well: the view. It doesn’t do data fetching, url routing and so.

React.js is quite pythonic. You can start adding little bits at a time to your code. You cannot really do anything wrong with it.

A core idea is that the html is in one corner. That’s the output. And the state is in the other corner. They’re strictly seperated, which makes it easier to reason about.

It is build by facebook. They take page load time seriously. You can even run react.js partially on the server side (“react native”, inside node.js), taking down rendering time, especially on mobile. Facebook is pretty deeply invested in it, so the chances of it disappearing are low. (As opposed to angular: google is barely using it itself).

When you start using react.js on a page, you’ll have to start thinking about components. An “unordered list of results”, a “search box”. Those kinds of components.

  • Component: basically a piece of html, handled by react. If react renders it, html comes out. It looks at its state and properties to render it (“it calls the .render() method”).

  • State: the current state of the component. The component can change this.

    If you can do without state: do it. You’re better off calculating it from something else. And if it makes sense to store the state on a higher-level component: do it.

    If a component’s state changes, you can assume that the render method gets called (it does some magic so that it only updates html if there are changes).

  • Properties: what you pass a component from the outside. Think parameters, like a list of products. The component cannot change these.

React.js code is quite simple to reason about as there’s a simple path through the code. If some method changes the state, react handles the rendering. The rendering might set up click handlers or other things. If such a click handler changes the state, react renders everything and so on and so on.

Integration with existing jquery plugins is generally possible, but you’d rather not do it. React.js tries to store state nicely in a corner and jquery plugins normally store state on the DOM itself. So watch out a bit. You might have to build some more custom html than you’d have to do otherwise as you’d just download one of the existing jquery plugins.

A note about facebook’s flux. “A flux” is just a structure for building large applications. There are multiple ones. He hopes you don’t have to build such a large application yet, as the flux you’re choosing might not be the one that wins out in the end.

If you need a good example of django and react.js, look at django-mediacat.

Question: “How do you integrate with django? Does django still render forms?” Answer: normally, you’d decouple the front-end and back-end, at least for the bits you let react.js handle. So perhaps generate some javascript variables in the django template. And grab the rest via an REST API.

Some clarification to the previous paragraph: you can choose yourself how much of the page you let react.js handle. He himself is using react.js to handle a couple of interactive elements on an otherwise-static django-rendered page.

Question: “Are you happy with react.js?”. Answer: “YES!”. It is quite small in scope. It doesn’t get in your way. Easy to get started with. You can’t do an awfully lot wrong (he has seen terrible examples with angular: it gives you much more rope to hang yourself with). You can progressively add interactivity to your page.

Tip: keep your react.js components small. Only let it handle spitting out html. Do any calculations you might have to do on the server side, behind your REST API.

Djangocon: Lightning talks

2015-06-03

Tags: django, djangocon

(One of the summaries of a talk at the 2015 Djangocon EU conference).

Pycon Namibia - Daniele Procida

In february 2015, the python conference in Africa actually took place! In Namibia! Daniele Procida talked about the plans at many conference (like at djangocon.eu 2014).

It was really good. They formed valuable relationships and learned a lot. Several of the attendees are here at the conference. They’re going to organize another conference later this year. So if you want to escape a dreadful European winter...

I’m new to django but why should I use it? - Ben Sharif

Why should you use it? You’re new, so why Django? He’s in that situation.

He’s a medical student. He’s not a doctor yet. It also means he isn’t a professional programmer. He wants to quickly prototype ideas for medical web apps. He wants something that lets him develop things fast: Django fits the bill.

He started learning python in may 2014. He tried the django tutorial, but that didn’t really work. He got pointed at web2py. Easy to get started. Almost no configuration. Through the web development.

His demo is at http://medboard.co.uk/

So... what is he missing? The django community is absolutely massive. So... tell him how he can get involved with Django with a bit more success!

Speech to text typists

(Note: The whole conference, there were screens on which a live transcription/subtitling of the talks was provided for those that needed it (non-native speakers of those with hearing problems). (So if you think that I’m typing fast by typing in all these summaries: I don’t stand a chance against those two ladies!)

They travel around the country to support deaf people. She had a top 3 of hard conferences. Now she has a top 4.

They work with machine shorthand. It works with a system of phonetics. It is a special keyboard. You use it like a piano with chords: you press multiple keys together to form words and parts of the words. The software matches the keys they’re pressing to a dictionary.

They can adjust the dictionary. That was why halfway the conference the term “wiskey” got turned into “WSGI”.

With a regular keyboard, you get a 120 word per minute if you’re really really good. With a keyboard like this, you can get to 250. That’s enough to keep up with most people (except Russell) :-)

(Note: They got a thunderous applause!)

Holiday picture from the Eifel region

DjangoBeer Florence

Two years ago he attended the djangocon in the Warsaw circus tent. Why not do something similar in Florence? They now rebooted the event and turned it into a monthly meeting about all things web.

They like (craft) beer, but they love to enjoy it responsibly. No drunkenness. The code of conduct is strictly enforced!

The even had beer tasting lessons.

http://djangobeer.com

Systers - Ana Balica

There are a lot of organizations that want to increase the number of women in IT. Djangogirls (lots of applause!), Pyladies (lots of applause!) and lots more.

Systers is a community for women involved in STEM areas (Science, Technology, Mathematics).

I wrote my first line of code 1.25 years ago - Geraint Palmer

He’s a PhD student. His first line of code was in VBA. He could do a lot with it: a for loop and a condition...

In October 2014 he started using python.

He learned enough of it to quickly start teaching python. That’s the best way to learn the language yourself. Recommended.

Google a lot. Once you don’t know the answer, google it.

Then he found a new nice text editor. It revolutionized his programming!

Afterwards he installed his first library.

Then pair programming! He learned structuring his code. He learned to use version control. This helped him a lot to grow up as a programmer.

He participated in Pycon Namibia and discovered there was a huge python community all around the world. Thank you!

Do I need to update my django project? - Mounier Messelmeni

He uses django since three years.

Why do updates matter?

  • Security and performance. Old unsupported releases don’t get security updates!
  • Bugfixes.
  • New features: every new version gets nice new features.

The good news: the current 1.8 version is a long term support version.

Python versions? If you still use python 2.6, you really should upgrade to 2.7

For every upgrade step, read the release notes and especially the added/removed section.

Make sure you have good backups of your database and your uploaded files.

It would be great if you could also look at using python 3. Look at https://caniusepython3.com/ or install caniusepython3 from pypi.

The missing method murder mistery

Great visual talk about tabs and spaces. Impossible to write down, watch the video when it comes out. Great.

Moral of the story: friends don’t let friends mix tabs and spaces.

Beter still: friends tell friends to use python 3 (that has better tab/space warnings).

Sprinting - Russell Keith-Magee

A sprint is a collection of people that turn up in a space to hack on code.

There is a sprint the next two days. What needs to be done? There are bugs. You could solve them, you could triage them, you could check patches, you can provide tests for patches or to prove bugs. You could help with the documentation.

Not being an english speaker isn’t a problem. Just get the first draft done, then other people can work on the spelling.

If you’re new to django: working on the tutorial would be great! Point out problem areas.

You can work on django core. On libraries. On programs that help django. On the django project website.

So: pick an area that interests you!

If you have questions: just ask a core developer! Or better still, ask the person next to you. The core team aren’t superheroes, we’re all in the same boat.

Djangocon Europe 2016

Djangocon Europe 2016 will be in... Budapest in Hungary!

Holiday picture from the Eifel region

Djangocon: Conformity and you - a question of style - Greg Chapple

2015-06-03

Tags: django, djangocon, python

(One of the summaries of a talk at the 2015 Djangocon EU conference).

Greg Chapple works half his time on Django projects, half on Rust projects.

Some things you’ll probably never hear: “Can’t wait to fix some pep8 violations today”. Style conventions probably aren’t the sort of thing you take into account when picking a language or a framework.

Every line of code follows some sort of style convention. Intentional or not. The style of our code can identify us. Same as with this talk: Greg speaks with a slight Dublin accent, by default.

Something to ask yourself: if you look at a piece of code inside your company, could you identify the person that wrote it?

Let’s get back to Greg’s accent and the way he speaks. How did he come by this? Where he grew up, of course. The people he’s around regularly. You can change it, though. He spend two weeks in the USA and when he came back he got funny looks for some of the US pronouncement that krept into his vocabulary.

The same with your code style! You’re influenced by the code you’re surrounded by. If you want to make changes, it takes quite some effort to do it. Learning to use list comprehension instead of for loops, for instance.

Your style should be consistent. Using single quotes in one place and double quotes on the next line, that looks bad. Spaces after a comma in some places (=pep8) or not.

If you have to switch from one project to another, you’re helped a lot if both projects use the same style. That’s where style guides come in. PEP8, jshint, flake8. Corporate guides. Look for support for those in your text editor to make it easy for you to use them. If you look at the download numbers for these text editor extensions, they’re very popular (and rightfully so).

Interesting fact: people take comments about their coding style much better if it comes from a tool as opposed to from a human being :-) Having a tool in place to tell you about code style violations is very valuable as it removes lots of possible unfriendly interactions.

A nice quote from pep8: One of Guido’s key insights is that code is read much more often than it is written. As PEP 20 says, “Readability counts”.

Greg looked at the django source code, hoping to find some nice glaring inconsistencies, but the django code is pretty good! :-) He did find some small issues and showed them as examples.

Sometimes you should allow some slight breakage to actually improve readability.

If someone else will be working on it, make sure it looks good.

If the code looks dirty and non-consistent, it doesn’t help to make you trust the actual work the code does. Doesn’t the lack of attention to how it looks mean that the actual code is not well-designed? Maybe, maybe not. But the lack of attention could easily have been there in some of the algorithms or the security declarations, right?

Your code speaks about you. You, as a professional developer. What does it tell about you?

Holiday picture from the Eifel region
 
vanrees.org logo

About me

My name is Reinout van Rees and I work a lot with Python (programming language) and Django (website framework). I live in The Netherlands and I'm happily married to Annie van Rees-Kooiman.

Weblog feeds

Most of my website content is in my weblog. You can keep up to date by subscribing to the automatic feeds (for instance with Google reader):