Category: machine learning

Machine-Learnt Algorithms and Unconscious Bias

Machine-Learnt Algorithms and Unconscious Bias

I attended a web foundation lunch a couple of weeks ago, around many issues related to data and ethics. I have to confess a lot of it went over my head. Since then I attended a talk by Cathy O’Neill which made some things a lot clearer to me. I’ve bought her book Weapons of Math Destruction, so hopefully I’ll have some more cogent notes after I’ve read that.

But in the meantime, here’s one small thought I had about algorithms and patterns:

As human beings we’re very good at identifying patterns, without thinking about it. So, if my experience is that the vast majority of women I see in technical meetings are admin staff, then whenever I encounter a woman in that context, I will guess that she is non-technical.

In a similar way, the machine-learnt algorithms are making assumptions and developing decision-making processes based on patterns identified in existing data. As human beings, we can train ourselves and each other not to make assumptions based on previous experience, and to be open to new possibilities. Is this also something which is / should be / can be built into machine-learnt algorithms?