Redis in practice - Pieter Noordhuis (PyGrunn conference)¶

Tags: python, pygrunn, django

Pieter works on the core of Redis. The core is just two people, but still :-) So it was fun to have him give a talk here at this conference.

Redis is “an advanced key/value store”. You probably have heard of memcached. It is also a key/value store. Redis is something similar, but it has more data types. Memcached only stores blobs, redis also has lists, sets, etc.

Redis stores everything in memory, so it is blindingly fast. Single thread, so just fire up multiple intsances if you’ve got more cores. Unlike memcached, you can persist it to disk. Replication is build-in. And it has client libraries for some 20 languages.

Redis has build-in expiry. Just explicitly set the expiration time, no need to keep track of expiration yourself.

Regarding lists, redis shows why it is more advanced than other non-datatype key/value stores: it supports pushing and popping at the begin and end of lists. You can use this for instance for showing recent events on some dashboard. Or a twitter-like timeline. To prevent these lists from becoming too big, you can tell redis to limit the length to a certain number. Now multiple clients can add items to the list with impunity without making the list too big.

Lists are also a natural fit for job queues. You add new jobs with a “right push” and workers grab jobs with a “left pop”. FIFO queue.

A different datatype: sets. Just unordered sets of items. There are no duplicates, though, like is normal for sets. The good thing about sets is constant-time membership tests. And cheap intersections and unions.

A really powerful datatype is the sorted set. An ordered set of unique values. Redis orders them in-memory by score. Technically, behind the scenes, it is a so-called “skiplist”. The extra sorting takes extra time. Inserting isn’t constant-time anymore, but log(N).

Every command in redis is atomic. But what if we want to group multiple commands in a transaction? For this, redis can queue up commands until you tell it to execute them all in one go. So there’s no rollback, but it can executed at the same time. The way redis works, this is enough.

On a regular machine, you can get 100k GET/SET per second. Very performant.

Persistance is handled with snapshotting. You can save every x seconds or every x changes. You an also write all writes to a log that you can replay.