Geodjango with oracle on windows: surprises and fixes¶

Actual Django project: well-deployable¶

I did not have to worry about the actual Django project we were shipping. We’re using buildout to make our builds and deployments repeatable. On a regular ubunbu box, deploying a new site is a non-event: easy as pie and faultless, mostly. You know which “aptitude install” commands to run to get the external dependencies. And buildout grabs all the regular python packages you specified and prepares the Django instance, apache config and all the rest.

Buildout works the same on windows: it grabs all the stuff and configures everything I told it to configure. And as I created the project from our internal skeleton/code template, it came out of the box with that at-least-working apache config for windows I distilled from the last windows project. Happy.

Problem: no internet access on the server. Intranet. Well-protected. So I need to bring a CD with all the code. Easy to do with buildout if you use zc.sourcerelease. The documentation isn’t clear, but once you installed zc.sourcerelease somewhere, you can call bin/buildout-source-release https://url/to/your/project/tags/1.7 buildout.cfg. So the url is your svn url. That works best in my experience. And the second parameter is the buildout config (in our case deploy.cfg as that’s the production config). And give it a -n project-1.7 parameter to get a better name than 1.7.

The result is: project-1.7.tgz. Your complete buildout including all the eggs and downloads it would otherwise have to grab from the internet. Extract it and run python install.py (instead of the normal python bootstrap.py and bin/buildout calls). Ready. Wow.

Installing, installing¶

Installing on windows starts easy. I looked at the geodjango install page and ask a colleague for the set of .exe and .zip files from his last windows deployment. Ok: python-2.6.4.exe and double-click. Same for apache. And matplotlib. And numpy. And an oracle connector.

And lxml: ouch, the latest version doesn’t have a python2.6 installer! I found a slightly older one that did. Another problem solved.

Some stuff is different. Some oracle “instantclient” (a set of DLLs) must be placed in some directory that you then add to the system path. Same with mapnik.

And the apache mod_wsgi module is shipped as a .so file that you have to copy somewhere deep in apache’s directory structure. Manually.

Ok, fine so far. Let’s fire up apache. Boom. Ok, let’s try django itself on the terminal. Boom. Uh oh…

cx_Oracle DLL problem¶

When firing up django, it would complain that it couldn’t load the oracle database connector. Some dll had a hickup. Dependencywalker is a great, simple tool that inspects a DLL for you and tells you which DLLs that it depends on are missing. I don’t remember whether it was linkinfo.dll or msvcr90.dll or msvcr71.dll anymore. The last two are redistributable files from microsoft visual studio. You can actually grab a .exe installer for them. Only, which of them? The 2008 or the 2009 or 2010? It doesn’t say which one includes which number…

I’m actually not sure anymore what solved this issue. I installed at least two of those redistributable file installers. And I swapped around between an oracle 10 and 11 connector and matching “instantclient” libraries. I think the 11 one worked in the end. I sure hope it goes well on the client’s server!

Oracle/geodjango GEOS DLL problem¶

Geodjango depends on the GEOS library. Also in combination with oracle. When trying to connect to the spatial database, I’d get an ImproperlyConfigured: Could not import user-defined GEOMETRY_BACKEND “geos”. error. Well, that’s just another library to install, right?

Problem: there’s no windows installer for geos. Only compile instructions. What? Can’t be true. So I asked around on two mailinglists and on irc and got great help. What the situation is?

• Use the geodjango windows installer. It installs django, but you can ignore that (just zap the django.pth from your site-packages directory). It also installs the Proj projection files (ok, I already installed those by hand, but that’s fine anyway). It installs gdal (good thing as I almost forgot that one). The gdal install also included some geos .so. Unfortunately, that’s not enough.

• Incredible when you want to use a remote oracle database: you need to install postgresql and postgis. Say again? Yes, to get the geos libraries you need to install postgis. You can probably copy/paste some DLLs around, but I thought it safer to just install the whole thing cleanly.

Matplotlib DLL problem¶

Next DLL problem on the list: matplotlib. After finally getting over the oracle + geo database hurdle, I could really start up Django. Only to have it die almost instantly once a graph had to be shown. Without a clear error message:

from matplotlib._path import affine_transform
ImportError: DLL load failed: The specified module could not be found

Well, which DLL?!? I had to go into the source code to get it to print a better error message. The ImportError exception was silently swallowed and a different import tried which also failed with a different error… The real error message gave me more information about which DLL was missing.

Dependencywalker to the rescue again. This DLL was missing a linkinfo.dll. Another one of those microsoft DLLs. Not available, though. And googling it gives a metric ton of virus links and warnings.

Some more googling showed that more recent matplotlib releases were compiled in a more reliable way with ms visual studio instead of the mingw compiler. So I grabbed a 1.0.0 release which even worked.

PIL problems¶

The python imaging library isn’t clear on how it is supposed to be imported. “import Image”, “import PIL.Image”, “from PIL import Image”. You see all of them. But that’s apparently a problem on windows!

Python (and even apache plus mod_wsgi) completely died with a AccessInit: hash collision: 3 for both 1 and 1 error message. It turned out to be PIL. “Image” and “PIL.Image” are the same object, but with a different name. I have no idea why this would be different between windows and unix, but whatever.

The solution is to put this hack in your django’s settings.py (see the comments on that article linked above:

import sys
import PIL.Image
sys.modules['Image'] = PIL.Image

Conclusion¶

• Even when starting with a brand new windows install, you can still run into DLL hell.

• Expect unexpected weirdness.

• Reliably compiling for windows is hard, especially when you depend on a lot of things.

• Don’t complain toooooooo hard. Get to work. It needs doing, sadly.

• Buildout and zc.sourcerelease work just fine. They’re a island of tranquility in a sea of muck.