Category: Women in tech

Machine-Learnt Algorithms and Unconscious Bias

Machine-Learnt Algorithms and Unconscious Bias

I attended a web foundation lunch a couple of weeks ago, around many issues related to data and ethics. I have to confess a lot of it went over my head. Since then I attended a talk by Cathy O’Neill which made some things a lot clearer to me. I’ve bought her book Weapons of Math Destruction, so hopefully I’ll have some more cogent notes after I’ve read that.

But in the meantime, here’s one small thought I had about algorithms and patterns:

As human beings we’re very good at identifying patterns, without thinking about it. So, if my experience is that the vast majority of women I see in technical meetings are admin staff, then whenever I encounter a woman in that context, I will guess that she is non-technical.

In a similar way, the machine-learnt algorithms are making assumptions and developing decision-making processes based on patterns identified in existing data. As human beings, we can train ourselves and each other not to make assumptions based on previous experience, and to be open to new possibilities. Is this also something which is / should be / can be built into machine-learnt algorithms?

Unconscious Bias vs Cognitive Bias

Unconscious Bias vs Cognitive Bias

Question:

What is the connection between unconscious bias and cognitive bias?

Answer:

Unconscious bias is a term used to describe people making assumptions about sub-groups within society. They are not automatically conscious of those assumptions, or the impact they have on their own behaviour and reasoning.

So, for instance: Society – and my experience – has encouraged me to believe that women are not likely to hold senior technical roles. Therefore if I walk into a meeting of senior technologists and there is one woman present, I might assume she is there in some other capacity – eg some kind of administrative role. If you ask me whether I believe women are less capable than men of being senior technologists, I will say no, but my unconscious bias still leads me to draw unhelpful conclusions, and may cause me to treat this woman in a way I would not deliberately choose.

Cognitive bias is the phenomenon where people used flawed judgement to assess data. Those flaws are caused by the brain’s tendency to be biased according to various factors.

So, for instance, I may listen to two political arguments and assign greater validity to one than the other. I will believe that my assessment is based on rigorous logic whereas in fact, I am simply drawn towards the argument that confirms beliefs I already hold (this is called “confirmation bias”).

Another example is the gambler’s fallacy: If someone tosses a coin ten times and it comes down Heads every time, they might believe that it is more likely to land Tails on the next toss. In fact the likelihood is still 50:50, just as it always was.

Cognitive biases are a very heavily theorised & researched concept within psychology, whereas “unconscious bias” is just a description rather than a clearly defined technical term.

The two terms are related, because cognitive biases may well inform some of our unconscious biases, but they are not generally used in quite the same context.

References:
Cognitive bias: http://io9.gizmodo.com/5974468/the-most-common-cognitive-biases-that-prevent-you-from-being-rational
Unconscious bias: https://www.brainpickings.org/2014/04/09/the-hidden-brain-shankar-vedantam/