Category: Coding skills

Another Interesting Conference Numbers Problem

Another Interesting Conference Numbers Problem

The other day I posted this problem.

In the process of chatting about it, an associated mathematics problem came up, about permutations and probabilities.

To remind you of the basic problem: We have 450 conference attendees. We have one day of workshops. There are five sessions throughout the day, and five workshops to attend. We want every attendee to attend every workshop. The workshops are being repeated throughout the day: In every session, there are 10 rooms available. There are five workshops, and each workshop is duplicated. So for instance, workshop 1 will be run in rooms A and B, workshop 2 in rooms C and D, etc. They are repeated throughout the day, so one attendee might do them in the order 15423, and another might do 42315.

There will be 45 people in each workshop. The nub of the problem is that the organisers want as much cross-pollenation as possible. People do not get to choose what order they do the workshops: they are told which room to be in at what time. The organisers would like each attendee to meet as many new people as possible in every single session.

So here’s the maths question: What is the minimum average number of repeat people that attendees have to meet in each session? By “repeat people” I mean “people they have already attended a previous session with.”

There is one extra constraint: There is one group of 30 people (labelled “FRIENDS” in the spreadsheet below) that have to be kept together in every session. So they only get 15 people in every session that they may not have met before, ie for them the min value of Repeat People is 30 in every session. This also has an impact on all the other people in the other sessions.

However this is still an interesting problem even without the extra 30-people constraint.

So in session one, Repeat People = 0. In every one of the workshops in session 1, attendees are in a brand new group of people that they have never attended this conference with before.

So what is the minimum average value of Repeat People for session 2? I’m specifying average, because it would be possible to keep things pure for at least one attendee, where they meet 44 new people in every single session. But that would have an adverse effect on everyone else. So I’m aiming for everyone to have roughly the same number of Repeat People per workshop.

I have two theories, one of which I can prove:

  1. Apart from in Session 1, the average Repeat People is definitely greater than zero. You can’t avoid meeting attendees more than once (if you’re spreading the load evenly) (I can prove this one).
  2. The average number of Repeat People will get gradually higher throughout the day. The first solution I came up with maintains a constant number for Repeat People after session 1, but it’s non-optimal (I think).

Here is a way of visualising the problem:

Gina-workshop-empty-2

I’ll be very interested to hear any solutions!

A Solution to the Interesting Conference Numbers Problem

A Solution to the Interesting Conference Numbers Problem

The problem is described here. You could approach the problem as a coding kata, which I’d love to do at some point – I didn’t have the time on this occasion. I ended up just solving it using pen and paper.

My solution is below. It won’t be the only solution, there are probably better ones out there that rely more on randomisation and less on patterns (or just use better patterns).

Don’t scroll down if you want to have a go at it yourself first!

 

 

 

So, I found a solution which works pretty well. I’m pretty pleased with it, and it’s nice and simple and neat.

The participants are split into groups of 15. The FRIENDS are in two groups of 15, both marked “FRIENDS” in the spreadsheet below. All the other groups are non-FRIENDS (well they might be friends, who knows? But they still don’t get to stay together – we’re mean like that).

It does mean that the people move in groups of 15, and they will stay in those groups. Not that they will necessarily know it. Because of the numbers, no matter what you do, they will keep encountering some people repeatedly (the maths of this is a separate problem which I’ve blogged about here). If you shuffle the list and assign the non-FRIENDS to 28 groups of 15, hopefully they will already be mixed up with people they don’t know, so they needn’t be aware that they have been grouped. I’ve labelled the groups B1 to O1 and B2 to O2.

What it does mean is that each group of 15 never encounters another group of 15 twice – they meet a different two groups in every session. So everybody meets 30 new people in every session (I haven’t actually proved this but I’m pretty confident).

(Since I came up with this, I’ve realised I can improve it quite a lot so that people don’t have to stay with exactly the same 15 people – scroll down to the bottom to see this improvement).

So, there are three groups in every session, like this (this is my actual solution):

Gina-Workshop-2

Here’s how I did it:

First, split the non-FRIENDS people into two halves, 225 in each half. The first half has groups B1 to O1, and the second group has B2 to O2.

For now, we will also split the FRIENDS into two groups, A1 and A2 (this was my “Aha” moment, all made possible because of the fact that you have duplicate workshops in each session).

So for each half, we have letters A to O. For the first session, we just spread them across the 5 workshops: ABC in the workshop 1, DEF in workshop 2… etc.

For the second session, the first group in each triplet just shuffles along to the next workshop. So A is now in workshop 2, D is in workshop 3, etc.

We keep doing this all the way down the sessions, for the first group in each triplet.

For the second group in each triplet (B, E, H, K and N), instead of shuffling them on by 1, we shuffle them 2 workshops along. So, B was in workshop 1 in session 1, then we add 2 so they are workshop 3 for session 2, then add 2 again so they are in workshop 5 for session 3, keep going (wrapping around) and they are in workshop 2 for session 4 and workshop 4 for session 5. Do this for all the second groups (B, E, H, K and N).

For the third group in each triplet (C, F, I, L, O), add 3 on each time. So group C has the following workshops: 1, then 4, then 2, then 5, then 3.

Do this for all of those third groups.

This is what you get:

Gina-workshop-diag

Now, at this point you have a problem: Your FRIENDS group have been split into two groups of 15. But ooh! Look! You did the same thing for A1 to O1 as you did for A2 to O2!

A1 and A2 are always in the same workshops at the same time. So you can move A2 into the same workshop as A1, and swap another group out into the spot left blank by A2.

I then renamed both A1 and A2 to “FRIENDS”, and that’s how I arrived at the spreadsheet pasted above.

POST SCRIPT (also see separate maths problem here):

Since I came up with this, I’ve realised I can improve it quite a lot so that people don’t have to stay with exactly the same 15 people:

The “FRIENDS” group mess it up a bit, but in most cases you’ll have parallel groups moving independently in duplicate workshops, eg when group C1 is doing workshop 3, group C2 will be doing the other duplicate of workshop 3.

Well… If each group of 15 was split into two sub groups of 7 and 8, then they could shuffle around and meet each other. Also this could be done dynamically.

So, for instance, you have C1a (7 people), C1b (8 people), C2a (7 people) and C2b (8 people).

In session 2, swap C1b and C2b so now only 7 (or 8) people stay together. Do the same with all groups (except those affected by the “FRIENDS” group).

In session 3, put C1a with C2a and C2b with C2b. This adds up to 14 and 16, so somebody will have to switch sub groups. In sessions 4 and 5 you could split them again, but this time randomly. As long as all the Cs are doing the same workshops, you can split them how you like. Ultimately they will all meet each other so they’ll still have repetition with 15 people overall, but it won’t be the same 15 people in every workshop. And they’ll still get 30 brand new people in every workshop.

A Really Interesting Conference Numbers Problem

A Really Interesting Conference Numbers Problem

I think this would make a great code kata, although I found a solution using pen and paper. It is a real problem, which my friend is genuinely trying to solve.

She is organising a conference. She has been given the following very interesting, and non-negotiable, requirements:

There will be one day of workshops. There are 5 sessions and 5 workshops. There will be 45 people in each workshop. There are 450 conference attendees, so each workshop is duplicated in each session. That is to say, during each session there will be ten actual workshops, but only five distinct workshops. So workshop 1 is happening simultaneously in two different rooms, as are the rest of them.

The aim is for participants to meet as many new people as possible. So for each attendee, we want to minimise the number of people in each workshop that they have met in a previous workshop.

We also want every attendee to attend every workshop.

And there is one more special requirement: There is a group of 30 attendees, we’ll call them FRIENDS, that must be kept together at all times. So in their workshops, there will be a rotating number of 15 extra people. They must meet a different 15 people in every workshop.

I have a solution for this, but I’ll post it separately in case you want to try it for yourself.

Here is a diagram that might help you to visualise the problem:

Gina-workshop-empty-2

Here is my solution to the problem.

There is an associated maths problem about the permutations and probabilities involved, here.

 

Idempotence: What it is and what it isn’t (What is an idempotent function?)

Idempotence: What it is and what it isn’t (What is an idempotent function?)

tl;dr: The mathematical definition of an idempotent function is subtly different to the definition used in software engineering. In software engineering an idempotent function is one that has the same impact on state, no matter how many times it is run. In mathematics, an idempotent function is one where f(x) =f(f(x)).

The concept of idempotence came up recently at work, in the context of infrastructure. A statement along the lines of “When treating infrastructure as code, it’s often important to ensure that your functionality is idempotent.”

I asked what “idempotent” meant, and I was given the (incorrect in all contexts) answer, “for the same input, you will always get the same output.” I was also told (correct for maths, but not for software) that the following function is not idempotent:

f(x) = x + 1

…but the following function is idempotent:

f(x) = x * 1

This instantly got me asking questions, because the first two definitions I was given were at odds with one another. The first definition describes a deterministic function. For instance: Both f(x) = x + 1 and f(x) = x * 1 are deterministic, ie given the same input, you’ll always get the same output (3 + 1 will always be equal to 4, and 3 * 1 will always equal 3).

But then I was given a better example, more relevant to the original conversation: If you have a function that adds an entry to a hosts file, you want to know that no matter how many times you execute that function, you will only ever add one entry to the hosts file. You don’t want to add more than one entry.

For instance, we start with a hosts file that looks like this:

127.0.0.1 localhost

We run our function, and now it looks like this:

127.0.0.1 localhost
255.255.255.255 broadcasthost

We run our function again, and nothing changes. The broadcasthost entry has already been added. Our function has nothing to do.

And then I found a true (in mathematics) definition of idempotence:

A function f(x) is idempotent if f(x) = f(f(x)).

To reiterate: The originator made a mistake: It is NOT true that idempotence is defined as “the same input always gives the same output”.

That is to say, if a function is idempotent and you apply that function to x, then you apply the function again to the return value, you still get the same result. Keep taking the result of each pass and sending it back into the function, and you still get the same result.

At this point I didn’t know about the difference between mathematical idempotence and software idempotence, and I was happy with my new definition: In our hosts file example, if our function takes the file content as an input and outputs the transformed result as an output, you can keep reapplying the function and you will keep getting the same result.

Using our new definition, we can easily see how f(x) = x + 1 is not idempotent, but f(x) = x * 1 is idempotent.

So far so good. But then I found this article claiming that pure functions are always idempotent, and my head exploded.

A pure function is one that has no side effects and no hidden state. The example given was this one:

f(a,b) = a + b

My confusion stemmed from two sources: Firstly, how can I apply my definition above – f(x) = f(f(x)) to this new example? It takes two parameters, but only returns one result! But secondly, how can it possibly be idempotent? It’s deterministic, yes, but any way you can find of repeatedly applying the same function to a new output will surely produce a different result? And what on earth does its pureness have to do with anything??

Well. I asked a bunch of clever people, and discovered that I had been dealing with the mathematical definition of idempotence, which is subtly different to the software engineering definition.

In software engineering, it’s all about state. My hosts file example was flawed because I had assumed that the hosts file content was being passed in as an input and then returned as an output. In fact, we are talking about a function that acts on the hosts file. This function’s input may be the hosts file path. Its output may be some kind of success code. It is not a pure function, because it will have the side effect (sometimes) of altering the state of the hosts file.

BUT our idempotent hosts-file-editing function can be run several times, and its effect on state will always be the same. No matter how many times we run this function, the hosts file will always be impacted in the same way.

The article that confused me so much was in fact making a very simple point: Pure functions are idempotent because they do not alter state. Therefore state is always impacted in the same way by multiple calls to a pure function, because it is simply not impacted. So in reality, most idempotent functions are not pure, but all pure functions are idempotent.

One common example of idempotence in software engineering is the HTTP specification – which states that GET, PUT and DELETE requests should all be idempotent, but POST should not.

At this point I will quote my colleague Mouad (and the Stormpath blog), who between them say this:

“The HTTP RFC have a better definition which goes:

A request method is considered “idempotent” if the intended effect on the server of multiple identical requests with that method is the same as the effect for a single such request.

The ‘intended effect’ as defined above is not the same thing as the returned value, example: calling PUT two times may return a different result in the second call (e.g. 409 conflict), but a PUT is still idempotent if the state and effect didn’t change by the second call, in other words, ‘HTTP idempotency only applies to server state – not client state’ ref. https://stormpath.com/blog/put-or-post.”

Hopefully your head has not exploded. Or if it did, I manage to unexplode it and return it to its former state. Hopefully also, no matter how many more times you read this post, your head will remain unexploded. And we have ourselves an idempotent blog post. Voila!

Let’s Stop Making People Feel Stupid.

Let’s Stop Making People Feel Stupid.

Intro here.

There’s been a lot of interest in this talk, which I’m doing in various locations this year (details here). People have asked me to share slides, but I deliberately don’t put much content (ie text) in them, because I find it just distracts from the delivery.

So, here are some notes. And a couple of pictures of cats. 🙂

confused-cat-7-im-confus

Making people feel stupid: What does it mean?

  • People listen to what others say.
    • They overhear them judging people for not being clever enough or not knowing enough.
    • They internalise it. They worry that they will be next.

What’s wrong with me?

  • Maybe you’re quietly judging me already. Maybe you’re thinking, she’s probably not very clever and doesn’t like it when she gets exposed.
  • Maybe you’re right! I often think that about myself. But I have a maths degree, 18 years experience, I’m a tech lead with a major international consultancy, etc.
  • So, that was me imagining that you might judge me for being stupid. Maybe you did and maybe you didn’t. But the point is, I imagined that you would. Because I’m so used to people in tech judging each other for being stupid.

My story

  • I’ve always felt there was something wrong with me because I struggle to understand things unless they’re explained in simple concrete terms.
    • And yet I can do complexity.
    • I can build complex systems out of simple parts.
      • If anything, my flaw is a tendency towards too much complexity.
      • …which is why I deliberately break things down into simple parts.
      • …but I also forget complex terminology – I recall easier-to-remember equivalents instead.
    • I have missed out on jobs because people were bemused by my apparent lack of expertise.
      • I have been told in interviews that I wasn’t competent because I couldn’t respond to the kind of question that requires you to have memorised stuff.

The Impact:

Impact on the industry

  • Facts, Figures, Statistics:
    • There will be an estimated 1 million more computing jobs than applicants who can fill them by 2020. This figure was projected by Code.org, based on estimates from the U.S. Bureau of Labor Statistics on job creation and separately, estimates of college graduation rates by the National Science Foundation.
    • Only 11% of employers (US) believe higher education is “very effective” in readying graduates to meet skills needed in their organisations.
    • Some 62% (US) said students were unprepared.
    • US: There are more than 500,000 open computing jobs nationwide, but less than 43,000 computer science students graduated into the workforce in 2016.
    • In 2016, the White House claimed the federal government alone needed an additional 10,000 IT and cybersecurity professionals.
    • Source, March 28th 2017: https://www.usatoday.com/story/tech/talkingtech/2017/03/28/tech-skills-gap-huge-graduates-survey-says/99587888/

Impostor syndrome

  • Hands up if you feel like other people are cleverer / doing things better than you?
    • Impostor syndrome: “Somehow everybody has failed to notice how rubbish I am.”
  • I hate – am almost incapable of – playing the game where everything is obfuscated and translated into a language that only the elite can understand.
    • Ironically this means that my impostor syndrome is at least partially based around the fact that I don’t seem capable of doing the things that entrench everybody else’s impostor syndrome.
  • I can hit the ground running, but I keep forgetting.
    • I have to prove it to myself over and over again.

Scenario A: Meeting where people talk jargon & nobody understands

  • Me: Hi, sorry I’m late.
  • Them: It’s fine, we were just talking about the ARM processor.
  • Me: Ah right, yes of course.
    • Shit, ARM, I know I’ve heard of that before. ARM, um…
    • [some stuff I don’t hear cos I’m trying to remember what ARM stands for]
  • Me: “Look folks, I’m so sorry, but I’ve forgotten what ARM stands for?”
  • Them: “Articulated retention matriculation.”
    • Me: I have NO idea what that is. I’ll work it out as I go along.
  • Somebody else: Actually guys, I think we should be considering AMRM at this point.
  • Me: AMRM?
  • Reply: Articulated meta-retention matriculation.
  • [someone else, not me]
    • That’s a very good point! We definitely need to get meta at this juncture.
    • Oh God, I was only just following this, but now they’ve lost me. Meta? What does meta mean in this context? What does meta mean in any context? It’s one of those terms that always confuses me, I know that much.
    • Oh well, I said juncture. I love saying juncture. It’s the perfect word for situations like this.
  • [some stuff that Person2 misses cos they’re worrying about what meta means]
  • Etc

Diversity and Inclusion

  • People want to fit in.
    • Two effects:
      • They use jargon to create a shared identity.
      • They feel bad if they feel like an outsider.
    • People will leave, or not join in the first place, because they feel excluded.
      • This disproportionately affects under-represented groups.
    • Stereotype threat: https://en.wikipedia.org/wiki/Stereotype_threat
      • “…men in STEM subject areas overestimate their own intelligence and credentials, underestimate the abilities of female colleagues, and that as a result, women themselves doubt their abilities — even when evidence says otherwise.”
      • Stereotype threat has been shown to reduce the performance of individuals who belong to negatively stereotyped groups.
      • If negative stereotypes are present regarding a specific group, group members are likely to become anxious about their performance, which may hinder their ability to perform at their maximum level. Importantly, the individual does not need to subscribe to the stereotype for it to be activated.
      • It is hypothesised that the mechanism through which anxiety (induced by the activation of the stereotype) decreases performance is by depleting working memory (especially the phonological aspects of the working memory system).

Talking in jargon

  • These insecurities cause people to increase the amount of jargon they use.
    • They want to prove how clever they are.
    • Their colleagues struggle to understand them, but they pretend they do, to avoid looking stupid…
  • Complex impenetrable language is what people deploy as a kind of force field
  • Weird vicious cycle: everybody obfuscates to protect themselves from potential exposure as somebody who doesn’t fully understand.
    • In the process they confuse everybody around them, who in turn become terrified that somebody is going to notice that they don’t fully understand what’s going on, so they join in the game, make everything they say sound complicated, and so the cycle continues.
  • There does come a point where you’ve been immersed in it for long enough that only some of it is confusing, and some/most of it makes sense.
    • That’s quite a kick!
    • You have to pay your dues to get to that point, and it feels good. You feel special.
    • So you pull the ladder up behind you.
    • You had to go up it, and so should everybody else.
    • You’re in the club now, and you want to savour that.
    • So you join with your new comrades in mocking those who still haven’t arrived.
    • You make no concessions in your language.
    • You’ve learnt what it means! It was hard! Why would you waste all that hard work and abandon your hard-won vocabulary by explaining things in simple terms?
    • Explaining things in simple terms takes twice as long anyway.
  • Giving the answer you think people want to hear:
    • The hairdresser asked me whether I had straighteners and I answered Yes. Why? Because I felt like it was the “right answer”. I don’t have straighteners. I’m never going to manage this labour-intensive haircut I’ve been given.

Scenario E: When talking jargon feels good

  • I felt all pleased with myself recently when I worked out how to join in with a hangouts conversation by using words like “discoverability” and “distinguishable”. I felt less insecure, and like I was now a proper grownup, a member of the club. But meanwhile there may well be somebody somewhere hearing nothing but “blah blah blah”…

Reasonable reasons

  • Is it sometimes ok?
    • “I can’t spend my whole time teaching people, I need people who can hit the ground running.”

Unreasonable reasons

  • Many people project a sheen of knowledge.
  • Many limit themselves by seeking to preserve knowledge once they find it.
  • People focus on their own experience – making themselves look good.
    • But when they look at someone else, they have a different agenda.
    • They don’t stop to wonder whether they have ever said anything “stupid” like that themselves – and if they did, WHY?
    • Or they remember it full well and don’t want anyone else to remember, so distract attention by joining in with the attackers.
  • We identify the things we CAN remember, then we fetishise them.
    • We push them over alternatives.
    • Not necessarily because they are better – just because we feel more comfortable there.

Definition of competent

  • What really impacts on you and your team?
    • Is it lack of knowledge?
  • What does it actually take to be good at your job?
    • What is the definition of competent?
    • What is the definition of intelligent?
    • “Be curious. Read widely. Try new things. What people call intelligence just boils down to curiosity.” – Aaron Schwartz.

People who are not techies are impacted too

  • Examples of support staff, stakeholders, non-technical people… being made to feel stupid.

My personal experience – the happy story

  • How I learnt to attack new knowledge outside my comfort zone.
    • The irony is that my career – and my enjoyment of it – has improved dramatically since I started admitting ignorance.

Why Empathy is so Important

  • “They only care about making themselves look good.”
    • This in itself is judgmental.
    • Think about how it feels like to be them!
  • You can find yourself alienating others without ever having conscious malicious intentions.
  • Other people know other stuff.
    • Two effects:
      • One: When they know stuff you don’t, you feel insecure.
      • Two: When you know stuff they don’t, you can get impatient.

Conclusions / Advice

  • Maybe you see me as an idealist. Or maybe I’m a pragmatist. Over the years I’ve paid attention to what works in life and what doesn’t. What makes people ill, what doesn’t. These are all practical hints for survival.
  • Does it actually matter how much people know? Industry constantly moving, people forget stuff.
  • Some of the most important moments in my career have been the times I’ve realised that my colleagues are also confused.
    • Eternal thanks to those that admitted it.
    • People are often scared to admit confusion.
    • I often don’t know what I’m doing.
  • The range of knowledge in our industry is VERY WIDE.
    • Don’t expect other people to know what you know, and vice versa.
  • People forget things they once knew.
  • If people don’t know enough, WHY is that? What’s deterring them?
  • What would happen if we changed the rules?
      • Focus on aptitude – recruitment becomes easier.
      • Encourage people to explore and experiment and learn WITHOUT RISK.
        • Stops people pushing less optimal solutions.
      • Make explicit statements to newcomers to your team, at start of meetings, etc – have a policy towards curiosity – keep repeating that simple questions are ok, that mistakes are ok, that if somebody doesn’t know something it’s in the interests of the whole team to help them learn. Plus, active encouragement to give feedback if these aims are not being met.
      • Tweet from Tim Post (@TinkerTim) re Stack Overflow:
        • “You can’t work on problems that you’re unwilling to admit. Wanting help often means being vulnerable enough to ask for it, and that’s where we are. Let’s keep making the internet better, without hurting people in the process.”
      • WE SHOULD ALL ENCOURAGE PEOPLE TO ASK QUESTIONS.

    The Stupid Manifesto

    LET’S STOP MAKING EACH OTHER FEEL STUPID. INSTEAD, LET’S…

    • Have an explicit policy of curiosity towards all things
    • Encourage each other to shout out if we discourage curiosity
    • Ask what people NEED to know, not what they know
    • Never judge someone because their knowledge doesn’t match ours
    • Give our colleagues every opportunity to learn and explore WITHOUT RISK
    • Give new people a chance to show us what they can do
    • ENCOURAGE EVERYONE TO ASK QUESTIONS
    • Acknowledge the broad range of knowledge in our industry
    • Remember our industry never stays the same
    • Remember we all forget stuff
    • Lead by example: Be honest when we’re confused
    • Focus on aptitude, not knowledge
    • Remember what it feels like when we are still learning
    • Prioritise clarity over jargon
    • Remember this is not idealism, it’s pragmatism
    • LET’S STOP MAKING EACH OTHER FEEL STUPID.

     

    Useful resources and references:

confused-cat-10-dafuq

Hack Manchester 2017: The Challenges

Hack Manchester 2017: The Challenges

I couldn’t find anywhere all the challenges / prizes were listed together, so I’m putting them all on one page for easy comparison.

Follow the links to find out about the sponsors and see more detail / explanation about the challenges:

WebApp UK: Best in Show.

MediaBurst: “Use SMS to help save lives.”

Dunnhumby: “Use technology to enhance the retail experience for a customer in the home or in store.

Ombudsman Services: “Create something that will make a consumer even angrier when registering a complaint.

GCHQ: “Develop a new way to send information to the emergency services.

Carbon Co-op: “Create an energy monitor which would be suitable for a visually impaired user.

Co-op: “Design an app, game, website or other digital service that re-connects people who’ve lost someone with other people and things to do in the places they live.

Texecom: “Use our network of wireless sensors to show us something interesting about the behaviour of people at Manchester Science Festival and the Texecom head office.

Resources for Women Arriving at or Returning to IT

Just a bunch of links really, but hopefully useful…

Women returners:

Mums in technology: 

Tech Mums: 

Equate Scotland: 

13 places where women can learn to code: https://learntocodewith.me/posts/13-places-women-learn-code/

Code First Girls: https://codefirstgirls.typeform.com/to/ks4bsj

Code First Girls professional courses: http://www.codefirstgirls.org.uk/female-professionals.html

Offline resources: