Metaprogramming and decorators (Bruce Eckel)¶

Tags: europython2009, europython

Basically, metaprogramming is code that modifies other code. Several languages have something like that. C++’s template metaprogramming (which is too difficult for most people). Java has aspect oriented programming and annotations. Ruby can add methods to metaclass objects. Python first had metaclasses and moved on to the much simpler decorators afterwards.

You use metaprogramming when you start to repeat yourself over and over again but you cannot filter out a common method or base class.

Built-in decorators in python are @property, @classmethod, @contextmanager. Twisted, turbogears, django and zope all have their own decorators for common tasks. Someone came up with an @accepts(int, string) argument checker for checking the type of method arguments. And a @memoize decorator can store a method’s result in a cache so that the method only has to calculate everything once and that the cached value is returned afterwards, all without having to deal with it in the method itself as the decorator handles it in a generic manner.

An example:

def addstring(function):
    function.mystring = 'added string'
    return function

@addstring
def f():
    return 1

This results in:

>>> f()
1
>>> f.mystring
'added string'

So the addstring decorator gets passed the item it applies to and returns it again. But modifying it in-between. The way it works is extremely simple, but the effect is big and very useful. It gets a bit more evolved when you want to pass an argument to the decorator (@addstring('hello')), but that’s what examples are for. See for instance the functools standard library.

You can also use decorators on classes. They work on a whole class instead of on a function. Typical use is to register the class somewhere or to augment it. For example (see activestate recipe) adding the rest of the ordering methods give one of __lt__, __gt__, etc.

Metaclasses are still part of the python language. A big problem is that a metaclass is more or less hidden; decorators are visible right in front of your class, on the line above. In plain sight. Metaclasses are much harder to understand and should only be used in special cases.

In case you think you need metaclasses: have you looked at __new__() in addition to __init__()? The output of __new__() is what gets fed into __init__(), so if you need to do some fiddling with the inheritance tree or other specialized weird thing, you can just use __new__() instead of using metaclasses.

Again, if you only want to add some attributes/methods or register the class or so: just use a simple decorator.

For an (according to bruce excellent) presentation on “class decorators: radically simple”, see http://pycon.blip.tv/file/1949345 .

Summary: decorators are easy and very useful. Don’t forget to use functools.wraps. Metaclasses are hardly ever needed and when you need it you’ll know that you need it.