Tempfile path normalisation in doctests¶

Tempfile doctest problem¶

In doctests, I often need a temporary directory for the duration of the test. In the test setup/teardown code, I create it and clean it up afterwards. So the test setup/teardown methods looks something like:

import tempfile
import shutil

def setup(test):
    test.tempdir = tempfile.mkdtemp()
    # other stuff
    test.globs.update({'tempdir': test.tempdir})

def teardown(test):
    shutil.rmtree(test.tempdir)

In the doctests, I can now use the tempdir variable that got injected into the test globs (=globals):

The test setup created a temp directory for us:

    >>> print tempdir
    /tmp/kdf34klsdj3200880f/

This directory is placed inside the default
temporary directory on our os, which is:

    >>> import tempfile
    >>> tempfile.gettempdir()
    '/tmp'

Well, not quite. The name of the tempdir is different all the time. And /tmp can be /var/tmp. And on osx it is something like /var/folders/qC/qC6d69l0EDe-sx-yyKchqU+++TI/-Tmp- and sometimes /private/var/folders/qC/qC6d69l0EDe-sx-yyKchqU+++TI/-Tmp-.

So you can opt to use ..., the doctest standard placeholder for “something”. But that can hide something:

The ... matches a lot:

    >>> delete_files_but_not_too_much()
    deleted ...
    deleted ...

So we think we just deleted two files, but the ... also matches
complete sets of lines, so the following test could also pass:

    >>> delete_files_but_not_too_much()
    deleted ...
    deleted ...
    deleted ...
    deleted ...

Oops.

Partial solution: normalize tempdir¶

You can pass a doctest output normalizer to your testrunner. For instance:

from zope.testing import renormalizing

checker = renormalizing.RENormalizing([
    # tempfile.gettempdir() is the OS's base tempdir
    (re.compile(re.escape(tempfile.gettempdir())),
     'TMPDIR')])
# Don't forget to pass the checker to the testrunner.

This grabs the default base temporary directory where tempfile creates its items. So tempfile.mkdtemp() creates a temporary directory inside the tempfile.gettempdir() folder. Your doctests can be more explicit this way:

TMPDIR is the base temporary directory

>>> delete_files_but_not_too_much()
deleted TMPDIR/...
deleted TMPDIR/...

Full solution: normalization with temp prefixes¶

You can do one better by also normalizing the actual created temporary directory. The trick here is to call mkdtemp() with a prefix option:

...
def setup(test):
    test.tempdir = tempfile.mkdtemp(prefix='mytest')
...

This creates directories like /tmp/mytest888d987f3uewer/. We cannot directly use the created tempdir’s name in a normalizer regex as the tempdir is normally created in the setup method and the normalizer method is outside it. The prefix helps us however in doing it anyway:

checker = renormalizing.RENormalizing([
    # Normalize tempdirs.  For this to work reliably, we need
    # to use a prefix in all tempfile.mkdtemp() calls.  We
    # look for the base tempdir, followed by the prefix,
    # followed by several non-slash characters.
    (re.compile(
        '%s/mytest[^/]+' % re.escape(tempfile.gettempdir())),
     'MYTEST'),
    # We probably also still need TMPDIR.  Place it after
    # the more specific checks otherwise it matches first...
    (re.compile(re.escape(tempfile.gettempdir())),
     'TMPDIR')])

Tadah, the doctest can now be really specific with no chance of erroneous hiding of lines:

MYTEST is the tempdir the setup prepared for us

    >>> delete_files_but_not_too_much()
    deleted MYTEST/file1.txt
    deleted MYTEST/file4.txt

OSX comment¶

In rare cases, osx “prepends” /private to the normal /var... base tempdir. Mostly when you call the python executable from within your doctest. So I had to add an extra normalizer to also detect that variant…