Hack for fixing invisible character differences in doctests¶

Tags: python

Before my holiday I was plagued by invisible differences in doctest output, something like

    Set up zope.testing.testrunner.layer.UnitTests in N.NNN seconds.
-   Ran 2 tests with 0 failures and 0 errors in N.NNN seconds.
+   Ran 2 tests with 0 failures and 0 errors in N.NNN seconds.
? ++++++++
    Tear down zope.testing.testrunner.layer.UnitTests in N.NNN seconds.

After reading some bug reports, I think the following is an accurate summary:

• Your operating system’s readline library sometimes emits a ^[[?1034h escape code to switch on 8bit character display when initialized.

• One of the things needed for the problem to occur: your TERM environment variable must be set to xterm. Either linux or vt100 is fine. I don’t know why. This is the point at which operating systems can have a different setting, btw.

• An import readline somewhere in your python code triggers the output of the escape code. The escape code is no problem, unless you’re catching stdout and doing string comparison on it…

• The doctest whitespace normalizer doesn’t filter out this escape code.

My hack to work around it is to set the environment variable to something not-xterm-like right smack at the start of my python file that sets up the tests. So something like:

import os
# Start dirty hack.
os.environ['TERM'] = 'linux'
# End of hack.

Not nice. But at least I can continue working for now on my real task. Should doctest’s whitespace normalizer be appended? Is there a real bug somewhere in python’s readline implementation (one bug report mentioned that python before 2.3 didn’t have this problem)? Enlightened opinion is welcome at reinout@vanrees.org, of course.

Comment by Jean-Paul Ladage: if you use buildout, you can set such an environment variable your the zc.recipe.egg part:

environment-vars =
    TERM linux

This way you don’t have to hack in the code.