Python usergroup the Netherlands (PUN) meeting in Rotterdam¶

Normally I've got a summary ready in no time. This time around I'm a couple of days late in posting. I'm already glad I was able to type in a summary as I'm slowly (but succesfully) curing from severe back/shoulder/arm pain.

Jelmer Vernooij - how samba and openchange use python

Samba is basically a free implementation of microsoft's network and file sharing protocols. Traditionally written in C and it's extremely portable. Originally they had to resort to network analysis, but thanks to the European Union, microsoft has been forced to document it. Actually, samba testified against microsoft as virtually the only one as most companies were bought out by microsoft...

Openchange is a free implementation of the MAPI protocol that microsoft uses for outlook and exchange. Openchange builds on samba 4 (3 is the current stable version, 4 is the hopefully-released-this-year version). The next gnome version ships with a plugin for evolution.

Samba saw a need for a scripting language (read: something easier than C). They went with... javascript. Which wasn't successful (no debugger, no nice language, no exceptions, etc.).

Two years ago python was proposed. The biggest hurdle was that it wasn't sanely bundleable. It would be samba's first ever mandatory build dependency other than libc. They did it anyway, so python bindings exist for samba 4. Mainly for the administrative tools, but also some core items to make sure that the bindings are always up-to-date.

To create bindings, they looked at SWIG. In theory: very nice. portable code, bindings for multiple languages, etc. But the python bindings are very C-like and thus Not Nice.

Pyrex/Cython, a binding alternative, is very python-like. But it is an extra build dependency. And it doesn't support certain use cases.

So they did it by hand. Hand-coding it isn't too bad after all. And it is much more flexible than autogenerated bindings.

The python bindings are surprisingly popular amongst samba programmers for writing automated tests. Those are much handier and shorter in python.

Erik Groeneveld (cq2.nl) - generators and program decomposition

weightless.io is part of what he talks about. (.io is the domain of the British Indian ocean territories (someone asked)). They use it a lot for their meresco components that do a lot with networking. There's a lot of slack inside common networking components that you can take out by being intelligent about it. He was surprised about the number of requests he got out of a 6 year old laptop with it compared to apache.

An important thing: don't use threads. Weightless doesn't use threads. With bigger applications, you have relationships between the components and you have to configure them. Weightless helps you manage those relationships and the configuration of them. The configuration uses the observer pattern, something that's really easy and comfortable in python.

You can call self.any, self.all or self.do to tell your observers to do something. self.all() uses a generator.

Erik explained Jackson pipelines. A 30 year old problem about reading data from tape (that didn't fit in memory), doing something with it and writing it back to another tape. This is exactly the problem we can have today with web frameworks and loads of data. Interesting stuff.

Generalized generators, the needed basis for so-called coroutines, was introduced in python 2.5 (see pep 342). "yield" is used in generators to return data incrementally. With the 2.5 functionality, you can also pass data back to the generator when calling the next item by passing the data in your next() call. What Erik did builds on that: it means that you can yield a second iterator from inside the first iterator. Erik wrote a helper method to deal with that which "flattens" the nested yields. yield something_that_also_yields() returns not the second set of generated elements, but only the generator object. Erik's method (compose()) makes it yield the elements instead of the generator.

At the end he explained a traceback trick that he did by using the (apparently abandoned) "tbtools" packages. Neat.

They use it in live production code. The cool thing is that your entire component becomes one big generator that you can connect to a (network) socket on both ends. All without the headaches of threads.

Lightning talks

Christoper Butcher, Speech recognition extension library. Speech recognition: you say something and the computer does something. Dragonfly is a python app that takes care of creating grammars for exiting speech recognition programs and executing commands based on their output. So steer your programs with speech by defining everything in python.

Guido Wesdorp, functional testing of AJAX webapps. He's working on a quite complex xml editor written in javascript. Javascript is notoriously difficult to test. He wanted to do functional testing of javascript from the commandline. Part of what he used was the python-spidermonkey library. He builds a "fake" browser environment (window, document, dom) in javascript. Stuff that cannot be done in javascript is shipped to python for processing and then returned to javascript. It is not complete yet. He hopes to be able to support/mimick different browser versions. That's a lot of work, though. He needs to get a community in place. The demonstration worked.

Remco Wendt, 1 minute python tricks. Recently introduced any() and all() to return True if any or all elements in a list match a condition. 'from collections import defaultdict': defaultdict allows you to create a dictionary with one specific default value for non-existing keys.

Matthijs Kadijk, experiences with Google App engine and Django. Google app engine is easy to get started. You can develop an application and google will host it. They'll scale it up or down own on demand, so it works even if you're slashdotted. The app engine uses django by default. A tip: restart your application if strange errors occur. Advantages of google app engine: it is free, it is python (though he is a ruby on rails developer), it is google. Disadvantages: it is free (they can't pay for extra bandwidth), it is python (...), it doesn't use sql (but some google-internal datastore). For the slides, see http://tng.blogspot.com .

Wichert Akkerman, WSGI and repoze. Zope2 is a big framework. Zope3 is a complete refactoring, but it is still a big pile of libraries. In the meantime, a lot of smaller python web frameworks sprang up. WSGI is a standard way to talk to most of them and to tie bits and pieces of them together. WSGI allows you to tie pipelines to an incoming web request (sets of components making up the pipeline get to process the request one after the other, all the way up to the actual end-application and then the request goes all the way back). Repoze makes zope3's components available to your WSGI pipelines. All the good stuff that zope3 provides is now available in handy bits and pieces for your custom WSGI applications. For instance: zope's transaction machinery to do safe database transactions on the web request's boundaries. So: the power of zope for your own small web framework!