2012-01-26
All those UFO sightings? Just an unholy mix between a flying saucer and a German tank. You know, lots of nazis escaped to the moon (the dark side of it) in 1945. Now they’re coming back.
The film is partially volunteer-made and partially volunteer-funded. There’s no big Hollywood studio behind it. Made in Finland. Shot in Germany and Australia. Lots of input and ideas and designs from all over the world (“crowd-sourced”).
And... they made it to the Berlinale (Berlin film festival) program and they’ll apparently show in Berlin’s biggest cinema. Wow.
Here’s a trailer where they announce it. And there’s a lot of gorgeous footage at the start. This will be an awesome movie!
2012-01-26
I just had a funny autocorrect happening on my iPhone. It was still set to English and I entered the Dutch word “slagroom”, (“whipped cream” in English).
Autocorrect suggested “slag room”. It reminded me of slag that room, now from an older Schlock Mercenary comic. That was not what I had in mind when preparing my shopping list :-)
2012-01-22
I’ve made a couple of changes to my website (and thus weblog):
I’m getting used to git at the office. And my website was in svn. The svn is on me and my brother‘s server. And we’re going to re-build that server in the near future. And not having to set up svn there saves quite some time.
So I moved part of the code to github and the actual content to a simple directory in our own server. Git works just fine with a repository URL like ssh://vanrees.org/~/git/websitecontent. Just init a repo in ~/git/websitecontent on the server and off you go.
My site is static html now for two years or so. Generated with http://sphinx.pocoo.org/. No Plone (which it was when I worked with Plone) and no Django (which I’d like, as I work with it, but my current text-only setup works real fine and Sphinx is an essential Django documentation ingredient, too).
I also make my weblog with Sphinx. I need a couple of custom restructuredtext tags for that (like ..tags:: for handling tags). I basically point a script at a sphinx directory that contains my entries and it generates index.txt files for all the directories with a list of entries. And it generates tags/TAGNAME.txt files for the tags. Afterwards, I just run Sphinx and everything gets generated OK.
I got some questions in the past about my code, but never followed up by publishing my terribly Reinout-specific code. It is still Reinout-specific, but I’ve published it now. It might provide some ideas. The full code is at https://github.com/reinout/reinout.vanrees.org . Again: this is not something you can pip install. Best you can do is borrow and copy at the moment. (Threat the Python code in there as public domain).
Most of interest are probably https://github.com/reinout/reinout.vanrees.org/blob/master/rvo/rst.py which provides the custom restructuredtext (and thus sphinx) tags. And https://github.com/reinout/reinout.vanrees.org/blob/master/rvo/weblog.py which creates all the index and tags files.
It is not well-documented and quite specific to my case. But there are some docstrings and the code is nicely PEP8 compliant :-) If you need any info, feel free to contact me.
I have to re-visit the layout of the websites we make at my work (Nelen & Schuurmans). One of the things we want is to have a more attractive layout. And after some browsing, twitter bootstrap struck me as a good basis.
And, as I do often, I first experiment on myself. So I set out to change the layout of my own website by using twitter bootstrap. I’m not even using half of bootstrap’s possibilities, but I’m quite happy with the result. The basic typography is fine. There are some handy helpers for common things like labels (which I used to liven up the tags below the weblog headers on my weblog). And I quite like the simple header at the top of the page: just a simple bar.
And there’s another good thing about twitter bootstrap: less. See below.
What coffeescript is for javascript, less is for css. A simpler syntax that helps you write better and less duplicated css.
I like it! Take for example the headings of this entry. Hover over one and you’ll see a paragraph token at the end. Sphinx adds this so you can link to individual sections in a bigger file. Sphinx also has some css that hides the paragraph token normally and only shows it when you hover over the heading:
a.headerlink {
visibility: hidden;
}
h1:hover > a.headerlink,
h2:hover > a.headerlink,
h3:hover > a.headerlink,
h4:hover > a.headerlink,
h5:hover > a.headerlink,
h6:hover > a.headerlink,
dt:hover > a.headerlink {
visibility: visible;
}
I restricted that to:
.headerlink {
visibility: hidden;
}
h1, h2, h3, h4, h5, h6, dt {
&:hover {
.headerlink {
visibility: visible;
}
}
}
It looks just like css, as you see in that first .headerlink rule. The second one has the special less stuff. The ampersand means an extra condition, so the &:hover effectively adds :hover to each of the h1..``dt``. This really shines if you have a couple of other extra conditions on the same set of elements.
Anyway, head over to http://lesscss.org/ where they have a better explanation.
I’m happy with the change. One extra change that I didn’t mention yet: I’ve removed deliverance from the mix. Deliverance is a tool that I used to change the layout of a couple of inputs (sphinx, docbook2html, latex2html) into my vanrees.org layout. It did that on the fly. But the versions kept changing and the versions I had were getting old and hard to update. And I had to run a wsgi process to run it, even though the actual website was all plain html.
I’m back to plain html now. I’ve implemented the layout in a Sphinx template. Simple to do, as it is just jinja2, which has the same syntax as Django. And the docbook and latex html exports are just that. I’ll leave them that way. It isn’t that important to have my logo on those pages :-)
One extra advantage: the site ought to be faster!
2012-01-18
(Talk at the 2012-01-18 Dutch Django meeting)
Security is about three things:
A well-known bad case is DigiNotar. Malware on most of their computers, including the most secure once. Their digital certificates were compromised and the company went bankrupt in three weeks time.
One thing that we have to take care of regarding Django: cookies. Cookies are stored in your browser and are linked to a website. All requests to that website get the cookies in every request. In Django, your session ID is stored in your cookie. If you get hold of the session ID, you get hold of the user’s session. Django stores the actual session’s data in the database, so the actual data is safe.
In Django 1.4 there are cookie-based sessions, but they’re a bad idea: never use them. He doesn’t know why they even build it in. UPDATE: they’re not that bad, see the comments.
CSRF: cross site request forging. Tricking the browser into posting requests to other websites. http://your-router/set_password?pwd=hamster . Django protects you by adding a secred csrf_token to every POST form and checking for it. So you’re safe.
XSS injection. Cross site scripting. Happens when you don’t filter/escape user submitted content. So you could send a <script> tag with a script that for instance steals every cookie on your server for stealing session IDs. Common is tricking a single user into running the script. A more powerful, but rarer, way is to get a script on some website and trick everyone visiting the site...
Django is mostly safe as it escapes template tag output by default. There is a rare vulnerability if you don’t put double quotes around html attributes. Everything after a space in var in <p class={{ var }}> ends up as a separate attibute...
Cookie security is pretty good in Django. 1.4 sets SESSION_COOKIE_HTTPONLY to true. You ought to do this always. And if you always use https, set CSRF_COOKIE_SECURE and SESSION_COOKIE_SECURE to true.
And... don’t trust in a regex that filters dirty tags out of your user-submitted code. There are SO many tricks and ways to circumvent any regex. Use markdown or something like that.
SQL injection is common, but Django has no know exploits. It is rock solid. But there are other ways. For instance LDAP injection. Don’t put an unfiltered username into your LDAP query as you can hack the query. Or direct user input into filenames. And what about shell injection? some_filename.txt;rm -rf... Shell calls are always very risky, but Python’s subprocess() helps a lot.
There’s a difference between validation and escaping. Validation is done as early as possible. Escaping is done as late as possible. Both serve a different purpose. As always: whitelisting is better than blacklisting.
A new option is clickjacking that overlays a transparent website over another one, stealing your form fields and clicks. It works with iframes. Django 1.4 has a new X_FRAME_OPTIONS='DENY' that works with newer browsers.
And... Don’t trust browsers. If you disable something in javascript, a user can work around it, for instance. The speaker could always book rooms in school before anyone else could as the only-two-weeks-in-advance limitation was implemented in javascript, not in the backend...
Regarding tests: check login-protected pages as anonymous first. Check actions for the admin user first as anonymous, then as a regular user. Check it.
And if you run some test site that’s (temporarily) on an external IP address, you can be sure someone finds it. So watch out.
If you run backups, do you also run restores? Do you keep backups with a different provider? Once a provider was hacked and its servers were wiped. Including their backups. All customer data lost.
Django: don’t ever run it in DEBUG mode in production. It gives away way too much information. Another tip: name the admin url something different than /admin. it is a dead give-away that it is a Django website.
Watch out for other apps on the same server. Your Django server can be compromised by running some faulty-configured PHP code on the same server.
Always restrict database access per app.
Summary. Django helps out with CSRF, SQL injection and XSS; you need to take care of all other injection attacs (LDAP, filesystem, etc). Use the build-in Django security features. And never trust what the browser sends. And secure the web server itself as well. Make backups.
2012-01-18
I sometimes commit sacrilege.
I make notes in my books. I scrawl in the margins. I underline important sentences. Sacrilege! It horrifies some people that someone can do that to a book.
Why do I do it?
You know what’s real sacrilege? Making notes in your bible :-) One of my bible study group members shuddered at the thought of not keeping it as tidy as possible. Well, mine is missing part of the cover, has notes in it and has lots of marks from been carried across town in many different kinds of bags... Sacrilege or usage? Both attitudes are fine, I think.
The kinds of notes that I make:
Maybe it is time for you to start making notes in your books?
2012-01-18
(Talk at the 2012-01-18 Dutch Django meeting)
Jeroen actually has zero Django experience, but he does nice things :-) That website of his is actually made with SVG.
SVG is more than 10 years old, Adobe published the first version in 1999. IE6 supported it with an adobe plugin, most other browsers started adopting it in 2003. IE9 fully supports it, btw. Nice: since 2011 (corrected, I said 2001...) it is even part of the html5 specification.
Scalable vector images are, well, scalable. Despite the screen size, you won’t see the pixels as in an upscaled pixel image.
SVG as a format is just XML. You can draw lines, arcs, circles, boxes and so on on a canvas. The output can be styled with css, just like html. It are DOM elements. And you can manipulate the drawn SVG elements with javascript, too.
What you use often is pen movements. M or m for move, L or l to draw a line. The lower case is relative to the current location, the upper case for absolute location. M 10 10 l 10 20 l 20 20 l 20 10 l 10 10 would draw a rectangle. For more elaborate forms you can use inkscape, an open source SVG editor.
Nice: events on SVG objects can be processed by Javascript. You can also do animations with SVG. There are three ways:
As browser compatibility is still a bit of a problem, you’re probably best off by using libraries like raphaeljs if you want to use SVG. (Personal note: I used raphaeljs with success in a small Django project last week).
He showed a couple of demos. What was particularly good was the easy and on-the-fly integration between forms and the picture (like for a custom table that can hover over the couch). Put in a different height of the couch and the table would adjust. Good javascript integration.
He uses it for making designs for feeding them into CNC milling machines like they have at the fab lab in The Waag in Amsterdam.
2012-01-18
(Short 5 minute talks at the 2012-01-18 Dutch Django meeting)
Coffeescript is not really a language, but it is syntactic sugar for javascript. A nicer syntax. It compiles to javascript. It uses indentation (like Python) to indicate code blocks instead of using curly braces, for instance. And the way to declare functions is easier and less verbose.
Coffeescript also allows classes, including constructors, and inheritance.
Nice: on the coffeescript site you can type in some code and get it transformed in to javascript. Handy for trying it out.
The language isn’t difficult. You can learn it within the hour (he said ‘a bit more than 5 minutes’). You can compile it into javascript on the server with node.js. You can also include the coffeescript.js script in your page and let the browser handle the compiling automatically (at a performance penalty, but it is easier for development).
Django-coffeescript integrates the compiling into Django. And you can even put inline coffeescript in your template. But normally you’ll just point at external files.
Bonus tip: django-less. Less is css + variables + nesting + inheritance. Integrated in the same way as django-coffeescript.
One drawback of coffeescript: it makes debugging harder as you have to debug the javascript that comes out of the compiler instead of the coffeescript code you just wrote.
“Django-fluent-dashboard - a fresh dashboard for your admin interface” was the title.
Django is a bit technical. His thought: how to make the Django admin more usable as a CMS that’s friendly? Another core thing in his thinkwork was the idea of reusable apps.
If you look at the current admin, you directly see the huge number of apps. Not nice.
Somebody made Django admin tools. Nice. A good solution. It gives the admin a nicer layout and even a menu bar that helps you find things.
The core idea: django-fluent-dashboard only modifies the first page of the admin: instead of the big list of apps you get friendly, grouped, icons.
He made a modification (or plugin, or newer version, I don’t know) that looks even nicer. More dashboard-based. See the example on the documentation site.
In the end, he hopes to turn it into a full CMS. There’s already a django-fluent-contents and django-fluent-pages on github. But they’re separate projects. django-fluent-dashboard works just fine with Django Fiber, for instance.
A small teaser for a bigger talk at next month’s Dutch python meeting.
Logbook is nicer logging for python than the standard library’s default logging module.
And for Django: whatever logging module you use, use django-sentry to get a website that collects your various site’s logs. He made a version that works especially nice with logbook, as logbooks gives you all logs of a web request if something goes wrong: it doesn’t only give you just the last traceback. So use debug logging liberally, it will help you debug.
http://30loops.net/ is a Python and Django hosting platform, located in Amsterdam (and a bit in Berlin since he moved recently).
They have years of experience in sysadmin and hosting, so they’re building yet another django/python/wsgi hosting platform. What you normally get at such a “platform as a service:
They differ on two major points:
They actually abstracted away quite a lot, so you can run your apps on 30loops’ own hardware, but also on other cloud providers. Handy if you have to deploy to multiple physical locations.
You can access all logs and metrics with REST: you own all your data.
They’re also on github: https://github.com/30loops
You can sign up for private beta now, btw.
Asking around gave the usual list of “good tools”, “passion” and “emacs”. He mentioned:
Why all these questions? Reason: there is a big need for professional software developers. You see lots of mails on the mailinglist asking for personell.
Would it be a good idea to start a “django academy”? Certification? Learning about acquiring clients? Create a network for supply and demand of professional developers?
There was quite some discussion afterwards (and I didn’t agree with some of the points, for the record).
Django fiber is a simple, user friendly CMS for your Django projects. It is getting bigger and bigger. 11k downloads since April 7, 2011. That’s about 1250 downloads per month. That’s only counted on pypi, though. Django CMS has 60k downloads since the beginning of time, as a comparison. So it is turning out to be pretty big!
What did we do since early 2011?
And... they build a lot of sites and apps with it (he said to me they build some 60 sites!)
Near-future plans:
They are realizing that they need to build a community around it if they want to get it off the ground further:
Nice: a couple of days ago, Luke Plant mentioned some big db optimizations for Django. They will look at that tomorrow, too.
(I wrote about an earlier 30 minute talk he gave on Django Fiber for if you want more information.)
2012-01-18
Despite my back, I’m back.
Since the summer my right leg is acting up. We moved to a new house and I probably did too much hard work. My right leg muscles started contracting the whole time, making me limp a bit, giving me muscle pain. Sometimes it felt like my leg was partially on fire.
I went to the doctor and she told me it was probably a lower back problem. A lower back problem can squeeze out another muscle or nerve by constricting it or limiting oxygen supply or so. The problem then radiates from the lower back into other body parts: for instance, a leg. Of course, limping a bit on that leg isn’t too good for your posture, potentially leading to other back problems. The verdict: nothing major, just irritating. She suggested me to go visit a physiotherapist.
Four days later I bend over the wrong way while standing in the bath. It took me five minutes to crawl out of the bath: I severely wrecked my back muscles. I had to lie down in bed or on the sofa for four days. On my left hand side. Sitting or lying or standing or walking were painful. Going to the toilet was a major undertaking.
After three days I started taking stronger-than-aspirine painkillers (ibuproven), which helped getting the pain a bit under control and allowed me to move again. Moving sore muscles is necessary for getting them back in order again.
At the end of the week I managed to sit down (yes! accomplishment!) at my parents’ 40 year wedding anniversery dinner.
I stayed home for more than two weeks. First I couldn’t move or sit. Later I could actually sit down on the couch, but not in a regular office chair. And soaking your muscles in a nice hot bath halfway the day, that’s not something I’m going to attempt at the office :-)
Luckily I’m back at work again. The going is slow, as I tire at the end of the day, even when I take regular walks through the office instead of sitting. So I go home earlier.
And I come into the office later, as the internal body repair functions apparently take a lot of energy, so I need to sleep more. And I still prefer to soak in a hot bath in the morning to get my muscles going.
One thing that really suffered during those weeks was my Django book. I wanted to finish three chapters by the end of the month, but that schedule was obviously shot. I’m still occasionally tired (and most book work is in the evening), but I’ve gotten back on track.
Good news: today and tomorrow I’ve taken a day off to work full-time on my book. Well, full-time apart from this evening as there’s a Django meeting in Amsterdam tonight which I’ll attend.
I went to the physiotherapist a couple of years ago. And I’m back. With my back. The lower back this time: the last time it was more in my shoulder area.
He helps, as I can do much more than a few weeks ago. I still limp and walking is a chore, but I can at least move about again.
Back to the medieval times of torture... Ouch, it hurts to have a physiotherapist massage a leg muscle that’s been rock solid hard for a couple of months... But it helped!
I haven’t blogged in about a month; no wonder, seeing the above. I’m starting again. That Django meeting in Amsterdam tonight? Great time to make summaries again; the program looks great.
Last year was a good blogging year. 165 entries in one year: my second best year. I somehow managed 182 in 2004. I like statistics, so I have two graphs with statistics on my weblog.
Well, I’m back to writing my book now. Bye!
2011-12-13
In an early chapter of my upcoming “solid Django” book, I have to explain Django. How it works, what it is, how it does things. I use several ways. One of them is the well-known model/view/controller (MVC) architecture.
Most web frameworks explain themselves in that way and every web framework sees MVC a bit differently. For Django, for instance, you also see “model/view/template” as an explanation. I myself think that MVC offers a pretty helpful way to look at Django. Only, it is pretty hard to explain right. And I have to leave out quite some details: it is an early chapter. So I’m leaving out middleware, forms, template tags and so on :-)
Here’s the core of my explanation. If you’ve got any suggestion/gripe/complaint/praise, please share it as a comment on this article. That way I can improve my chapter! I’m well aware that you can explain Django in lots of ways so I want to make sure I don’t foul it up.
I’ve also made a video where I’m explaining it this way. I’m experimenting to find a good setup for making such a video, so it isn’t perfect yet. Especially the sound can be better, I think.
2011-12-10
Properly tagged and released: that, all our websites are. Most of them are subversion projects, but a handful are now in git. When switching to a new version of a site, the subversion process is clear:
$> svn switch https://our.svn.server/project/tags/1.2
$> bin/buildout
$> ...
And the next time:
$> svn switch https://our.svn.server/project/tags/1.3
$> bin/buildout
$> ...
What I did with our first git project (after an initial “clone” of course):
$> git pull
$> git checkout 1.2
... Dire warning about "headless state" as I checked out a tag ...
$> bin/buildout
$> ...
And the second time:
$> git pull
... polite notice to bugger off as we're in a headless state ...
... no, we're not going to do a pull! ...
Ok, what now? I asked around on twitter and got the luckily simple answer: use git fetch instead of git pull. Fetch fetches the last bunch of commits from the server. Pull does the same and additionally updates the checkout to the latest relevant version. Which it refuses to do if the current checkout is a tag.
A tag as such is not really connected to the main tree of development that git knows about. It is more of a light-weight pointer. You get a certain clearly identified version of the code, but that version you have is not really hooked up in to the git tree. It has no parent to point to, so it also doesn’t have a relation to some pointer to the latest version on this branch of the tree: the head. That’s where the “headless state” comes from.
git fetch just grabs the latest state from the server. And a subsequent git checkout 1.3 checks out the 1.3 tag without a complaint:
$> git fetch
$> git checkout 1.3
$> bin/buildout
$> ...
Statistics: charts of posts per year and per month.
My name is Reinout van Rees and I work a lot with Python (programming language) and Django (website framework). I live in The Netherlands and I'm happily married to Annie van Rees-Kooiman.
Most of my website content is in my weblog. You can keep up to date by subscribing to the automatic feeds (for instance with Google reader):