Idempotence: What it is and what it isn’t (What is an idempotent function?)

Idempotence: What it is and what it isn’t (What is an idempotent function?)

tl;dr: The mathematical definition of an idempotent function is subtly different to the definition used in software engineering. In software engineering an idempotent function is one that has the same impact on state, no matter how many times it is run. In mathematics, an idempotent function is one where f(x) =f(f(x)).

The concept of idempotence came up recently at work, in the context of infrastructure. A statement along the lines of “When treating infrastructure as code, it’s often important to ensure that your functionality is idempotent.”

I asked what “idempotent” meant, and I was given the (incorrect in all contexts) answer, “for the same input, you will always get the same output.” I was also told (correct for maths, but not for software) that the following function is not idempotent:

f(x) = x + 1

…but the following function is idempotent:

f(x) = x * 1

This instantly got me asking questions, because the first two definitions I was given were at odds with one another. The first definition describes a deterministic function. For instance: Both f(x) = x + 1 and f(x) = x * 1 are deterministic, ie given the same input, you’ll always get the same output (3 + 1 will always be equal to 4, and 3 * 1 will always equal 3).

But then I was given a better example, more relevant to the original conversation: If you have a function that adds an entry to a hosts file, you want to know that no matter how many times you execute that function, you will only ever add one entry to the hosts file. You don’t want to add more than one entry.

For instance, we start with a hosts file that looks like this:

127.0.0.1 localhost

We run our function, and now it looks like this:

127.0.0.1 localhost
255.255.255.255 broadcasthost

We run our function again, and nothing changes. The broadcasthost entry has already been added. Our function has nothing to do.

And then I found a true (in mathematics) definition of idempotence:

A function f(x) is idempotent if f(x) = f(f(x)).

To reiterate: The originator made a mistake: It is NOT true that idempotence is defined as “the same input always gives the same output”.

That is to say, if a function is idempotent and you apply that function to x, then you apply the function again to the return value, you still get the same result. Keep taking the result of each pass and sending it back into the function, and you still get the same result.

At this point I didn’t know about the difference between mathematical idempotence and software idempotence, and I was happy with my new definition: In our hosts file example, if our function takes the file content as an input and outputs the transformed result as an output, you can keep reapplying the function and you will keep getting the same result.

Using our new definition, we can easily see how f(x) = x + 1 is not idempotent, but f(x) = x * 1 is idempotent.

So far so good. But then I found this article claiming that pure functions are always idempotent, and my head exploded.

A pure function is one that has no side effects and no hidden state. The example given was this one:

f(a,b) = a + b

My confusion stemmed from two sources: Firstly, how can I apply my definition above – f(x) = f(f(x)) to this new example? It takes two parameters, but only returns one result! But secondly, how can it possibly be idempotent? It’s deterministic, yes, but any way you can find of repeatedly applying the same function to a new output will surely produce a different result? And what on earth does its pureness have to do with anything??

Well. I asked a bunch of clever people, and discovered that I had been dealing with the mathematical definition of idempotence, which is subtly different to the software engineering definition.

In software engineering, it’s all about state. My hosts file example was flawed because I had assumed that the hosts file content was being passed in as an input and then returned as an output. In fact, we are talking about a function that acts on the hosts file. This function’s input may be the hosts file path. Its output may be some kind of success code. It is not a pure function, because it will have the side effect (sometimes) of altering the state of the hosts file.

BUT our idempotent hosts-file-editing function can be run several times, and its effect on state will always be the same. No matter how many times we run this function, the hosts file will always be impacted in the same way.

The article that confused me so much was in fact making a very simple point: Pure functions are idempotent because they do not alter state. Therefore state is always impacted in the same way by multiple calls to a pure function, because it is simply not impacted. So in reality, most idempotent functions are not pure, but all pure functions are idempotent.

One common example of idempotence in software engineering is the HTTP specification – which states that GET, PUT and DELETE requests should all be idempotent, but POST should not.

At this point I will quote my colleague Mouad (and the Stormpath blog), who between them say this:

“The HTTP RFC have a better definition which goes:

A request method is considered “idempotent” if the intended effect on the server of multiple identical requests with that method is the same as the effect for a single such request.

The ‘intended effect’ as defined above is not the same thing as the returned value, example: calling PUT two times may return a different result in the second call (e.g. 409 conflict), but a PUT is still idempotent if the state and effect didn’t change by the second call, in other words, ‘HTTP idempotency only applies to server state – not client state’ ref. https://stormpath.com/blog/put-or-post.”

Hopefully your head has not exploded. Or if it did, I manage to unexplode it and return it to its former state. Hopefully also, no matter how many more times you read this post, your head will remain unexploded. And we have ourselves an idempotent blog post. Voila!

Tricking Yourself Out of Procrastinaton

Tricking Yourself Out of Procrastinaton

I was just reminded of a trick I sometimes play on myself to avoid procrastination, which is surprisingly effective:

I tell myself I’m just going to open all the relevant files / documents, remind myself what the task is and get everything set up ready to start work. I tell myself that I’m not actually going to do the work, just get ready to do the work.

Invariably I get drawn in, and before I know it, I’m actually doing the work.

I just played it on myself. It worked. 🙂

(I also wrote a piece here about procrastination: https://insimpleterms.blog/2017/12/22/fighting-procrastination-in-solidarity-with-all-teachers-everywhere/)

Let’s Stop Making People Feel Stupid.

Let’s Stop Making People Feel Stupid.

Intro here.

There’s been a lot of interest in this talk, which I’m doing in various locations this year (details here). People have asked me to share slides, but I deliberately don’t put much content (ie text) in them, because I find it just distracts from the delivery.

So, here are some notes. And a couple of pictures of cats. 🙂

confused-cat-7-im-confus

Making people feel stupid: What does it mean?

  • People listen to what others say.
    • They overhear them judging people for not being clever enough or not knowing enough.
    • They internalise it. They worry that they will be next.

What’s wrong with me?

  • Maybe you’re quietly judging me already. Maybe you’re thinking, she’s probably not very clever and doesn’t like it when she gets exposed.
  • Maybe you’re right! I often think that about myself. But I have a maths degree, 18 years experience, I’m a tech lead with a major international consultancy, etc.
  • So, that was me imagining that you might judge me for being stupid. Maybe you did and maybe you didn’t. But the point is, I imagined that you would. Because I’m so used to people in tech judging each other for being stupid.

My story

  • I’ve always felt there was something wrong with me because I struggle to understand things unless they’re explained in simple concrete terms.
    • And yet I can do complexity.
    • I can build complex systems out of simple parts.
      • If anything, my flaw is a tendency towards too much complexity.
      • …which is why I deliberately break things down into simple parts.
      • …but I also forget complex terminology – I recall easier-to-remember equivalents instead.
    • I have missed out on jobs because people were bemused by my apparent lack of expertise.
      • I have been told in interviews that I wasn’t competent because I couldn’t respond to the kind of question that requires you to have memorised stuff.

The Impact:

Impact on the industry

  • Facts, Figures, Statistics:
    • There will be an estimated 1 million more computing jobs than applicants who can fill them by 2020. This figure was projected by Code.org, based on estimates from the U.S. Bureau of Labor Statistics on job creation and separately, estimates of college graduation rates by the National Science Foundation.
    • Only 11% of employers (US) believe higher education is “very effective” in readying graduates to meet skills needed in their organisations.
    • Some 62% (US) said students were unprepared.
    • US: There are more than 500,000 open computing jobs nationwide, but less than 43,000 computer science students graduated into the workforce in 2016.
    • In 2016, the White House claimed the federal government alone needed an additional 10,000 IT and cybersecurity professionals.
    • Source, March 28th 2017: https://www.usatoday.com/story/tech/talkingtech/2017/03/28/tech-skills-gap-huge-graduates-survey-says/99587888/

Impostor syndrome

  • Hands up if you feel like other people are cleverer / doing things better than you?
    • Impostor syndrome: “Somehow everybody has failed to notice how rubbish I am.”
  • I hate – am almost incapable of – playing the game where everything is obfuscated and translated into a language that only the elite can understand.
    • Ironically this means that my impostor syndrome is at least partially based around the fact that I don’t seem capable of doing the things that entrench everybody else’s impostor syndrome.
  • I can hit the ground running, but I keep forgetting.
    • I have to prove it to myself over and over again.

Scenario A: Meeting where people talk jargon & nobody understands

  • Me: Hi, sorry I’m late.
  • Them: It’s fine, we were just talking about the ARM processor.
  • Me: Ah right, yes of course.
    • Shit, ARM, I know I’ve heard of that before. ARM, um…
    • [some stuff I don’t hear cos I’m trying to remember what ARM stands for]
  • Me: “Look folks, I’m so sorry, but I’ve forgotten what ARM stands for?”
  • Them: “Articulated retention matriculation.”
    • Me: I have NO idea what that is. I’ll work it out as I go along.
  • Somebody else: Actually guys, I think we should be considering AMRM at this point.
  • Me: AMRM?
  • Reply: Articulated meta-retention matriculation.
  • [someone else, not me]
    • That’s a very good point! We definitely need to get meta at this juncture.
    • Oh God, I was only just following this, but now they’ve lost me. Meta? What does meta mean in this context? What does meta mean in any context? It’s one of those terms that always confuses me, I know that much.
    • Oh well, I said juncture. I love saying juncture. It’s the perfect word for situations like this.
  • [some stuff that Person2 misses cos they’re worrying about what meta means]
  • Etc

Diversity and Inclusion

  • People want to fit in.
    • Two effects:
      • They use jargon to create a shared identity.
      • They feel bad if they feel like an outsider.
    • People will leave, or not join in the first place, because they feel excluded.
      • This disproportionately affects under-represented groups.
    • Stereotype threat: https://en.wikipedia.org/wiki/Stereotype_threat
      • “…men in STEM subject areas overestimate their own intelligence and credentials, underestimate the abilities of female colleagues, and that as a result, women themselves doubt their abilities — even when evidence says otherwise.”
      • Stereotype threat has been shown to reduce the performance of individuals who belong to negatively stereotyped groups.
      • If negative stereotypes are present regarding a specific group, group members are likely to become anxious about their performance, which may hinder their ability to perform at their maximum level. Importantly, the individual does not need to subscribe to the stereotype for it to be activated.
      • It is hypothesised that the mechanism through which anxiety (induced by the activation of the stereotype) decreases performance is by depleting working memory (especially the phonological aspects of the working memory system).

Talking in jargon

  • These insecurities cause people to increase the amount of jargon they use.
    • They want to prove how clever they are.
    • Their colleagues struggle to understand them, but they pretend they do, to avoid looking stupid…
  • Complex impenetrable language is what people deploy as a kind of force field
  • Weird vicious cycle: everybody obfuscates to protect themselves from potential exposure as somebody who doesn’t fully understand.
    • In the process they confuse everybody around them, who in turn become terrified that somebody is going to notice that they don’t fully understand what’s going on, so they join in the game, make everything they say sound complicated, and so the cycle continues.
  • There does come a point where you’ve been immersed in it for long enough that only some of it is confusing, and some/most of it makes sense.
    • That’s quite a kick!
    • You have to pay your dues to get to that point, and it feels good. You feel special.
    • So you pull the ladder up behind you.
    • You had to go up it, and so should everybody else.
    • You’re in the club now, and you want to savour that.
    • So you join with your new comrades in mocking those who still haven’t arrived.
    • You make no concessions in your language.
    • You’ve learnt what it means! It was hard! Why would you waste all that hard work and abandon your hard-won vocabulary by explaining things in simple terms?
    • Explaining things in simple terms takes twice as long anyway.
  • Giving the answer you think people want to hear:
    • The hairdresser asked me whether I had straighteners and I answered Yes. Why? Because I felt like it was the “right answer”. I don’t have straighteners. I’m never going to manage this labour-intensive haircut I’ve been given.

Scenario E: When talking jargon feels good

  • I felt all pleased with myself recently when I worked out how to join in with a hangouts conversation by using words like “discoverability” and “distinguishable”. I felt less insecure, and like I was now a proper grownup, a member of the club. But meanwhile there may well be somebody somewhere hearing nothing but “blah blah blah”…

Reasonable reasons

  • Is it sometimes ok?
    • “I can’t spend my whole time teaching people, I need people who can hit the ground running.”

Unreasonable reasons

  • Many people project a sheen of knowledge.
  • Many limit themselves by seeking to preserve knowledge once they find it.
  • People focus on their own experience – making themselves look good.
    • But when they look at someone else, they have a different agenda.
    • They don’t stop to wonder whether they have ever said anything “stupid” like that themselves – and if they did, WHY?
    • Or they remember it full well and don’t want anyone else to remember, so distract attention by joining in with the attackers.
  • We identify the things we CAN remember, then we fetishise them.
    • We push them over alternatives.
    • Not necessarily because they are better – just because we feel more comfortable there.

Definition of competent

  • What really impacts on you and your team?
    • Is it lack of knowledge?
  • What does it actually take to be good at your job?
    • What is the definition of competent?
    • What is the definition of intelligent?
    • “Be curious. Read widely. Try new things. What people call intelligence just boils down to curiosity.” – Aaron Schwartz.

People who are not techies are impacted too

  • Examples of support staff, stakeholders, non-technical people… being made to feel stupid.

My personal experience – the happy story

  • How I learnt to attack new knowledge outside my comfort zone.
    • The irony is that my career – and my enjoyment of it – has improved dramatically since I started admitting ignorance.

Why Empathy is so Important

  • “They only care about making themselves look good.”
    • This in itself is judgmental.
    • Think about how it feels like to be them!
  • You can find yourself alienating others without ever having conscious malicious intentions.
  • Other people know other stuff.
    • Two effects:
      • One: When they know stuff you don’t, you feel insecure.
      • Two: When you know stuff they don’t, you can get impatient.

Conclusions / Advice

  • Maybe you see me as an idealist. Or maybe I’m a pragmatist. Over the years I’ve paid attention to what works in life and what doesn’t. What makes people ill, what doesn’t. These are all practical hints for survival.
  • Does it actually matter how much people know? Industry constantly moving, people forget stuff.
  • Some of the most important moments in my career have been the times I’ve realised that my colleagues are also confused.
    • Eternal thanks to those that admitted it.
    • People are often scared to admit confusion.
    • I often don’t know what I’m doing.
  • The range of knowledge in our industry is VERY WIDE.
    • Don’t expect other people to know what you know, and vice versa.
  • People forget things they once knew.
  • If people don’t know enough, WHY is that? What’s deterring them?
  • What would happen if we changed the rules?
      • Focus on aptitude – recruitment becomes easier.
      • Encourage people to explore and experiment and learn WITHOUT RISK.
        • Stops people pushing less optimal solutions.
      • Make explicit statements to newcomers to your team, at start of meetings, etc – have a policy towards curiosity – keep repeating that simple questions are ok, that mistakes are ok, that if somebody doesn’t know something it’s in the interests of the whole team to help them learn. Plus, active encouragement to give feedback if these aims are not being met.
      • Tweet from Tim Post (@TinkerTim) re Stack Overflow:
        • “You can’t work on problems that you’re unwilling to admit. Wanting help often means being vulnerable enough to ask for it, and that’s where we are. Let’s keep making the internet better, without hurting people in the process.”
      • WE SHOULD ALL ENCOURAGE PEOPLE TO ASK QUESTIONS.

    The Stupid Manifesto

    LET’S STOP MAKING EACH OTHER FEEL STUPID. INSTEAD, LET’S…

    • Have an explicit policy of curiosity towards all things
    • Encourage each other to shout out if we discourage curiosity
    • Ask what people NEED to know, not what they know
    • Never judge someone because their knowledge doesn’t match ours
    • Give our colleagues every opportunity to learn and explore WITHOUT RISK
    • Give new people a chance to show us what they can do
    • ENCOURAGE EVERYONE TO ASK QUESTIONS
    • Acknowledge the broad range of knowledge in our industry
    • Remember our industry never stays the same
    • Remember we all forget stuff
    • Lead by example: Be honest when we’re confused
    • Focus on aptitude, not knowledge
    • Remember what it feels like when we are still learning
    • Prioritise clarity over jargon
    • Remember this is not idealism, it’s pragmatism
    • LET’S STOP MAKING EACH OTHER FEEL STUPID.

     

    Useful resources and references:

confused-cat-10-dafuq

Greek: Don’t Confuse Genitive Definite Articles with Possessive Pronouns

I got confused after using the word “his” to explain the genitive case of the definite article (here).

It’s especially confusing because the possessive pronoun for 3rd person plural (ie “their”), is the same word (“τους”) as the accusative form of the definite article for male plural (here), but NOT the same as the genitive form of the definite article for male plural (which is “των”).

Anyway, just to note that the definite article and the possessive pronoun are often, but not always, the same.

Full explanation here on the DuoLingo forum.

Find the nth term of a Quadratic Sequence (Maths GCSE)

Find the nth term of a Quadratic Sequence (Maths GCSE)

I’m currently helping my 15-yr-old son revise for his maths GCSE, and one topic is “finding the nth term of a quadratic sequence”. I’m an ex high school maths teacher, but I had forgotten how to do this. I couldn’t find decent complex examples on either of my favourite GCSE maths revision sites (Maths Genie and BBC Bitesize), and when you’re doing the more complex examples, a step-by-step guide is really useful.

So I’m placing my notes here in case they’re any use to anyone else.

You’re aiming for a result of an2 + bn + c, but easier examples might have a solution of an2 + b, and even easier ones will just be an2.

Simplest Example (an2):

Find the nth term for the following quadratic sequence: 3, 12, 27, 48, …

First calculate the gaps between the numbers – these are 9, 15 and 21.

Then find the gaps between the gaps – these are 6 and 6. Like this:

nth-term-05

Take that 6 and divide it by 2 (it’s easy to forget to divide by 2!), to get 3. This tells you that your final result will contain the term 3n2.

I’ve already told you that this is a simple example – we’ve reached our solution: 3n2. But you should always check your results:

n 1 2 3 4
n2 1 4 9 16
3n2 3 12 27 48

Yup, that’s our original sequence.

More Complex Example (an2 + b):

Find the nth term for the following quadratic sequence: 1, 10, 25, 46, …

First calculate the gaps between the numbers – these are 9, 15 and 21.

Then find the gaps between the gaps – these are 6 and 6. Like this:

nth-term-04

Take that 6 and divide it by 2 (it’s easy to forget to divide by 2!), to get 3. This tells you that your final result will contain the term 3n2.

Create a grid, which starts with your original sequence. Below that, add whatever rows you need to help you calculate 3n2.

Now, subtract 3nfrom the original sequence. So in the below grid, we subtract the fourth row from the first row, and that gives us a new sequence, which we have placed in the fifth row:

start 1 10 25 46
n 1 2 3 4
n2 1 4 9 16
3n2 3 12 27 48
start minus 3n2 -2 -2 -2 -2

We now have a row of constant numbers. This tells us we can reach a solution. It tells us to add -2 to 3n2, and that will be our solution: 3n2 – 2.

We can easily check this by adding up the fourth and fifth rows, which gives us the first row (the original sequence).

Most Complex Example (an2 + bn + c):

Find the nth term for the following quadratic sequence: -8, 2, 16, 34, …

First calculate the gaps between the numbers – these are 10, 14 and 18.

Then find the gaps between the gaps – these are 4 and 4. Like this:

nth-term-03

Take that 4 and divide it by 2 (it’s easy to forget to divide by 2!), to get 2. This tells you that your final result will contain the term 2n2.

Create a grid, which starts with your original sequence. Below that, add whatever rows you need to help you calculate 2n2.

Now, subtract 2nfrom the original sequence. So in the below grid, we subtract the fourth row from the first row, and that gives us a new sequence, which we have placed in the fifth row:

start -8 2 16 34
n 1 2 3 4
n2 1 4 9 16
2n2 2 8 18 32
start minus 2n2 -10 -6 -2 2

We don’t have a row of constant numbers yet, so we need to keep working. We need to look at the gaps between the numbers in our new sequence (in the bottom row of the table):

nth-term-06

Now we have found a constant difference. This tells us that there will be a 4n in our answer. Note that this is because we have found a linear sequence. Note also that in the case of a linear sequence, we do NOT divide the number by 2.

So now we add some more rows to our grid. First we calculate 4n, and then we calculate 2n+ 4n. Finally we subtract (2n+ 4n) from our original sequence (subtract the 7th row from the first row):

start -8 2 16 34
n 1 2 3 4
n2 1 4 9 16
2n2 2 8 18 32
start minus 2n2 -10 -6 -2 2
4n 4 8 12 16
2n+ 4n 6 16 30 48
start minus (2n+ 4n) -14 -14 -14 -14

We now have a row of constant numbers. This tells us we can reach a solution. It tells us to add -14 to 2n+ 4n, and that will be our solution: 2n+ 4n – 14.

We can easily check this by adding up the seventh and eighth rows, which gives us the first row (the original sequence).

More worked complex examples

nth-term-01

Note that in this next one there is a NEGATIVE difference between the terms of the sequence on row 5. This one can easily catch you out. Rather than thinking of the difference between the numbers, it helps to ask yourself, “how do I get from each term to the next one?” The answer in this case is, “subtract one”. This one can also look a little tricky because it contains fractional numbers, but you just follow the same rules as before:

nth-term-07

Telling the Difference Between a Linear Sequence (an + b) and a Quadratic Sequence (an2 + bn + c).

When we calculate gaps between the numbers in the sequence, if the first level of gaps is constant, this means it is a linear sequence:

nth-term-06

If the second layer of gaps is constant, it is a quadratic sequence:

nth-term-03

Basic Rules of Modern Greek – Some, a, the (Cases, definite articles, indefinite articles)

Basic Rules of Modern Greek – Some, a, the (Cases, definite articles, indefinite articles)

I’ve been learning Greek!

This is one in a series of cheatsheets. Full list here.

THE (NOMINATIVE / SUBJECT)

Singular masculine ο ο άντρας = the man
Plural masculine οι οι άντρες = the men
Singular feminine η η γυναίκα = the woman
Plural feminine οι οι γυναίκες = the women
Singular neuter το το παιδί = the child
Plural neuter τα τα παιδιά = the children

 

CASES

Nominative The Subject of the sentence She
Genitive Possessive His
Accusative The Object of the sentence Him
Vocative Calling someone Calling someone

 

A/AN/ONE

MASCULINE FEMININE NEUTER
NOMINATIVE ένας μία or μια ένα
GENITIVE ενός μίας or μιας ενός
ACCUSATIVE ένα or έναν μία or μια ένα

 

THE – CASES

MASCULINE FEMININE NEUTER
Nominative singular ο άντρας = the man η γυναίκα = the woman το παιδί = the child
Genitive singular του άντρα = of the man της γυναίκας = of the woman του παιδιού = of the child
Accusative singular τον άντρα = the man τη γυναίκα = the woman το παιδί = the child
Vocative singular άντρα = man γυναίκα = woman παιδί = child
Nominative Plural οι άντρες = the men οι γυναίκες = the women τα παιδιά = the children
Genitive plural των αντρών = of the men των γυναικών = of the women των παιδιών= of the children
Accusative Plural τους άντρες = the men τις γυναίκες = the women τα παιδιά = the children
Vocative Plural άντρες=men γυναίκες=women παιδιά = children

 

SOME

MASCULINE FEMININE NEUTER
NOMINATIVE μερικοί μερικές μερικά
GENITIVE μερικών μερικών μερικών
ACCUSATIVE μερικούς μερικές μερικά
VOCATIVE μερικοί μερικές μερικά

 

Basic Rules of Modern Greek – Phrases

Basic Rules of Modern Greek – Phrases

I’ve been learning Greek!

This is one in a series of cheatsheets. Full list here.

PHRASES

Καλημέρα Good Morning
Καληνύχτα / Καλό βράδυ Good night
Καλησπέρα Good evening
Όχι No
Ναι Yes
Ευχαριστώ Thanks / Thank you
Παρακαλώ Please / You are welcome
Λυπάμαι I am sorry
Συγνώμη Sorry / Excuse me
Αντίο Goodbye
Σ’ αγαπώ / Σε αγαπώ I love you
Γεια Hi / Hello
Τι κάνεις; How are you? / What are you doing?
Πόσο κάνει; / Πόσο κοστίζει; How much does it cost?
Εγώ είμαι ο / η ….. I am ….
Εγώ ζω (or μένω) στον / στην / στο …. I live in …..