Domain specific languages for functional testing (Mark Fink)¶

Tags: europython2009, europython

He used the Hudson continuous integration tool (so: something like buildbot) to set up a local demo. Compared to buildbot (the thing that I know), hudson has a web interface for adding and configuring projecs. Buildbot uses configuration files. Hudson is programmed in java.

The actual demo test was adding a job to hudson via that web interface. So browser-based tests. The tests he made are selenium tests, created with the firefox selenium IDE plugin.

After recording the test with selenium IDE, he saved the selenium test from the IDE to a python module. Wow, I didn’t know that was possible :-) That’s some nice integration. Anyway, this uses the “capture-and-replay” method of GUI testing. Maintenance of such a test suite is hard as you need to re-record or tweak the test after code changes. Lack of structure. Limited re-usability. But it is used a lot as it is so easy to do initially. “Capture and replay is not test automation” according to a book by Mark Fewster and Dorothy Graham

A domain specific language (DSL) is a “little language” used to address small specific problems. Domain specific as opposed to general purpose languages like python or java.

You can start out with a feature diagram, for figuring out the features your DSL should have. For DSL, you can choose between an “internal” or “external” DSL. An internal language uses the features of a host language (like python). An external language is completely of your own making, which means more work but it probably also means a better language.

For his DSL he made commands like “new job”, “remove job”, “for xyz configure SCM file:somewhere” for steering selenium tests. The tests themselves run a couple of selenium actions, which he did by copy-pasting the original selenium IDE export. The DSL implementation used @dsl('some-regular-expression') decorators to make registering the tests with the “nose” test runner easy.

Question: What is the application domain? Well, he uses it mostly for web testing, but it should work just fine with desktop applications.