Machine-Learnt Algorithms and Unconscious Bias

I attended a web foundation lunch a couple of weeks ago, around many issues related to data and ethics. I have to confess a lot of it went over my head. Since then I attended a talk by Cathy O’Neill which made some things a lot clearer to me. I’ve bought her book Weapons of Math Destruction, so hopefully I’ll have some more cogent notes after I’ve read that.

But in the meantime, here’s one small thought I had about algorithms and patterns:

As human beings we’re very good at identifying patterns, without thinking about it. So, if my experience is that the vast majority of women I see in technical meetings are admin staff, then whenever I encounter a woman in that context, I will guess that she is non-technical.

In a similar way, the machine-learnt algorithms are making assumptions and developing decision-making processes based on patterns identified in existing data. As human beings, we can train ourselves and each other not to make assumptions based on previous experience, and to be open to new possibilities. Is this also something which is / should be / can be built into machine-learnt algorithms?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s