Acceptance testing with RobotFramework (Pekka Klarck)¶

Tags: europython2009, europython

Pekka’s one-man company concentrates on test automation and Agile testing, so he does this sort of testing stuff all day. He’s also the lead developer of RobotFramework.

(Sorry, there’s supposed to be an umlaut on “Klarck”, I’ve got a unicode bug in my website software stack…)

RobotFramework is generic. It does not specialize in, for instance, web testing. It uses the keyword-driven testing approach. Interesting: it fully supports ironpython (dot net) and jython (java). RobotFramework is used by a big project inside Nokia. There are several contributors and the community is growing.

RobotFramework sits in-between the test library and the test data. An advantage of such an extra layer: you can use multiple test libraries in case you need a specialized test library for your database and another one for web testing. The test data, as it uses the libraries through RobotFramework, is always in the same format so users don’t need to learn a different syntax.

The test cases are supposed to be not too technical. More user-oriented. Tests are specified in a table, see the user guide for examples. The second table column indicates the kind of test that needs to be run and the columns following that are keyword arguments (like password, username). See the examples in the user guide.

Test case files are put into directories (which you can nest) and RobotFramework makes one test suite out of them. Test case files are html (restructured text is also possible: it is converted automatically to html before running it).

A RobotFramework library is a class with methods named after the action you put in that second column, like attempt_to_login(). This method gets passed the keyword column data (like username='reinout', password='secret'). There are several libraries built-in, for instance for creating files, telnet, etc.

Non-programmers can use a table at the start of a test file to create their own “macros”: methods defined just in that test that take keyword arguments and that call “real” library methods. Handy if you have to do a task multiple times. Yes, as a programmer you can put the method as such directly in the library python code. But it is very useful if your test creators can do it inside the test files, too (assumed not every test creator can program).

Combination with for instance doctests or unittests: he would run at least the unittests separately and afterwards run the RobotFramework tests. You can, with some programming, integrate RobotFramework tests and other frameworks, but that depends on your own needs and situation.

A handy feature: assign tags to tests. They don’t have any influence on the tests, but they’re used in reports. Handy for grouping. “20 weird_edge_case testcases failed”. Simple, yet powerful, feature.

You can extend existing test systems, especially if they have python APIs. He showed a demo where RobotFramework steered selenium tests.

Question: what is the sweet spot for RobotFramework, when do you recommend to use it? If everybody in the team is a programmer, you can of course do it all in python directly. But as soon as there are non-programmers in your team, RobotFramework helps a lot. The test cases really explain themselves.

Question: how do you create the test case files? He sometimes uses restructuredtext (which is converted to html). Often it is a graphical html editor. Modern word versions also result in clean-enough html. A recent project is to make a custom editor for RobotFramework.