Djangocon: it’s not a bug, it’s a bias - Anna-Livia Gomart¶

Tags: djangocon, django, python

(One of my summaries of a talk at the 2018 european djangocon.)

If you have a bad experience with a program, is it a bug of can there be a bias in the software? Apple’s Siri, at first, had answers for questions like “where do I hide a body?”, but it didn’t have an answer for “where can I get an abortion?”…

Are algorithms neutral or can they be biased? You can have biased programs, biased users and biased data.

Programs are made by programmers. Programmers are humans. Humans have experiences. Experience is often colored…

We, as programmers, can read the algorithms. So it is our role (or our duty) to critique them. Look at computer games. Often there is “procedural rhetoric” in it: some worldview or agenda that is embedded in the game’s enviroment or rules.

Facebook has a definition of “friend”. So in fact they’re re-defining “friend”…

A problem you often have: normalcy. As designer/programmer/manager, you have certain assumptions. These might not be true for the user. “Everyone lives in a EU city and has 4G”. “No-one has a last name called Null”. “Everybody reads English perfectly”.

An important bias: the negativity bias. Negative reviews are notice much more than positive reviews. One bad review can ruin a small business.

Machines can be biased, too. Say, you notice that you yourself are biased when hiring people. That’s not good. So you let the computer make the decision based on machine learning. You feed it the list of current employees….. And you have a biased algorithm.

To make it worse… Imaginge you turning it into a website that is used by many companies…. Suddenly there’s an algorithm trained on biased data that makes or breaks people.

How do you discover biases? Catch alienating experiences, for instance. Look in support emails. “My last name really is one character long”.

What helps? Have a diverse team. Various backgrounds. Various ages. Various life experiences. And check your normalcy. Check if your assumptions are true. (There apparently is a list on github about “things developers get wrong”, but I can’t find the right one).

Photo explanation: I’m making interiors, too, so I need to paint figures. 1:87