Category: Agile

Paired Programming: Useful Articles, Resources and Research

Paired Programming: Useful Articles, Resources and Research

During my talk at NDC London this week, I promised to publish a list of resources you can use if you are trying to persuade people of the efficacy of paired programming as a software development technique. Here it is!

(incidentally, the image is of Sal Freudenberg and me doing some remote pairing – we use Zoom and we find it works really well – I often forget we are not in the same room).

Martin Fowler webinar, new Refactoring book

Martin Fowler webinar, new Refactoring book

These may well be the notiest notes I’ve ever published, but just in case they’re of any use to anyone… if nothing else they may whet your appetite for the new edition of Martin Fowler’s Refactoring book.

I confess I never read it first time round (not on purpose, just there are so many books in the world and so little time…), so I’m looking forward to reading it this time. It hasn’t come out yet in the UK but should be some time in the next few weeks. Mine is already on order [drums fingers impatiently].

So anyway, Martin did a webinar today on the topic of his new book, and here are my notes:

Refactoring should be done via small changes

  • v small semantics-preserving changes
  • So small they’re not worth doing on their own
  • String together small changes to make a big change

When adding new functionality:

  • Alternate between refactoring and adding functionality
  • Often it’s easier to add new functionality if you refactor first
  • Balance between adding new functionality and refactoring – it’s a matter of judgement

V1 of Martin’s Refactoring book vs v2

  • Some things that were included in the first book he decided not to include I the second
    • Eg Unidirectional vs bidirectional – not important
  • Some things he thought were too trivial for the first book, he has now changed his mind about and included
    • Eg moving statements within a function
  • Most notable changes = no longer centrally object-oriented
    • “Extract function” used throughout instead of extract method
    • One of the refactorings is around the choice between OO and non-OO
    • The most visible thing here is the choice of language (javascript)
    • But language is not necessarily relevant anyway
  • OO vs functional
    • He doesn’t see that as a huge shift
    • OO should still have functional elements – referentially transparent
    • They are overlapping paradigms, not distinct

Favourite refactorings?

  • “Split phase”
    • Hunk of computation which can sensibly be divided into two phases with a data structure communicating between them
    • Eg parsing – separate the tokenising out – deal with a series of tokens instead of a stream of text
    • But if you split a loop, what if you are introducing performance problems? See below…

Performance and refactoring

  • Split loop is something people often worry about because it will run the loop through twice
  • Most of the time it doesn’t matter – a push for clarity will not change the performance of the code
  • Most of the time by refactoring you open up an opportunity for performance improvements you would never have noticed otherwise
  • Then again you should run performance tests frequently
  • Not necessarily with every build, but every day or two
  • But most micro-changes have no impact on performance
  • You will only find real performance impact by performance testing

Recommendations for other books?

Refactoring architecture:

  • Always have to ask, how do you make the smallest possible change?
  • Eg extracting a data-intensive microservice from a larger system
  • Most of the moves are happening within the monolith BEFORE you start firing up the external system
  • Then you move stuff in really small pieces
  • Things that change for similar reasons should be together – you need to see the relationship between them

Branching / trunk-based development

  • Develop on a branch and merge at the end, with branch being long-lived – days, weeks or months
    • Still a really common pattern
  • But if something has been refactored, you don’t find out until you merge – this can blow up on you
    • It means you become scared to refactor! There’s a disincentive.
  • Continuous integration: Integrate with each other fully at regular intervals – at least once a day
  • Every day you receive everyone else’s code changes and push your own changes back
  • Removes the big painful merge problem – not saving up for a big horrible crunch
  • Adage of Apollo – it hurts if you do it more often (?)
    • Sorry, I wrote this down but I’ve since Googled it and I don’t really know what it’s about – I may have misheard!
  • Open source development:
    • Actually this CAN be a good place to use branches, because everyone working separately and you can’t guarantee all changes should be merged
  • People call it trunk-based development but “continuous integration” is a better name because that’s what it’s really about
  • People worry about continuous integration but it’s less painful than they think
    • If you do integration every day it becomes easier and less painful until you stop even noticing

Responsibility

  • Keep code base in healthy state
  • If not you are stealing from colleagues, clients and employers!
  • But no point saying you should do it for moral reasons, that won’t motivate people to do it
  • It’s economic: Want to get more changes out more quickly
  • It’s professional: We’re being paid to produce features rapidly
  • It’s not up to managers or business: They don’t know how to keep code healthy – that’s what they pay us to do

Refactoring and design patterns – are they incompatible?

  • With a lot of patterns, you shouldn’t introduce them right away cos they’re not appropriate and you don’t require the full strength of them yet, particularly the more complicated ones
  • Introduce patterns as a result of refactoring: Refactor towards them
  • This was recognised by the original Design Patterns authors (Gamma, Helm, Johnson, Vlissides)
  • Also Josh Kerievsky: Refactoring to Patterns

Code smells

  • Code smell of the week!
    • “This week we will look at long methods” – whenever we find them we try to refactor them

When not to refactor!

  • Code that is never touched
    • Sitting behind an API, working quite happily, doesn’t need to change – doesn’t matter if it’s a mess!
  • Don’t clean code all in one go – just a little bit more every time you go in
    • Over time that will concentrate your efforts where they are most needed – ie areas modified most frequently

Refactor as a step in the red-green-refactor cycle

  • You should only refactor when tests are green
  • Sometimes that means you have to remove a failing test, refactor, then put the failing test back in again
  • Ralph Johnson, John Brant, Don Roberts wrote the first Smalltalk refactoring book (A refactoring tool for smalltalk?)

If you don’t have tests you can’t refactor?

  • “Probably safe” refactorings? Interesting route to go? Not a great thing for most people to focus on, requires a lot of expertise
  • For most people you should have tests as well – after all, how do you know whether you’re breaking what’s currently there?

Final takeaway:

  • Small steps, commit frequently.
  • Then you can always roll back.
The Tech Lead’s New Project Checklist

The Tech Lead’s New Project Checklist

These are my notes from a session on what a tech lead needs to do and know when arriving on a new project. The wisdom came from @JimBarritt at Thoughtworks – I am just the conduit.

(August 2020 – I’m planning on updating this list, at the moment those updates live on my wiki site here)

Note that this list applies both to projects that existed before the tech lead arrived, and those where the whole team / product is new.

The first two are apparently “the two most likely causes of death or injury”:

1) Infrastructure

  • Where’s the production environment? It’s the most important one! What does it look like? When will it be ready? (Development environments are an afterthought after prod)
  • How are we going to go live? What’s the plan?
  • Load balancers
  • Firewalls
  • Certificates

2) Third parties

  • Integration!
  • If environments beyond your control are chaotic, it’s even more important to make sure your own environment is super-organised and disciplined, smooth, automated, with good scope planning, etc

3) Find the movers and shakers

  • Find all the people who can say No, and the people who hold the purse-strings/can sign off budgets, and the people who can say “Yeah, you did a good job” and get to know them
  • Eg:
    • Architects
    • The person who has control over data protection (Beware “organisational scar tissue”)
    • In the public sector, the SIRO (Senior Information Risk Officer)

4) Find any top-level documentation of your team’s work, read it and understand it

5) What’s the budget and the value proposition?

  • How well defined is it? Why are you doing the work? What’s the vision? Do people understand it?
  • Once you find this info, be really repetitive about it
  • If there are assumptions which are crucial to the whole success, keep repeating them at the start of every showcase

6) Try to have a one-to-one with each person in the team

  • Try to understand what their goals are, particularly on the project – and be transparent about your own goals. For instance if you have areas you want to move away from, find people who are keen to move INTO those areas
  • What are their main pain points and blockers?

7) Befriend a project manager and get to understand how they manage budgets

8) If there are multiple teams, find and get to know the other tech leads

9) Understand the importance of the Trinity of delivery: Delivery manager, product owner, tech lead

  • There is often a tension between these three areas
  • Build relationships with these people
  • All three should be present for all decisions, otherwise it’s likely to be skewed
  • Tech leads can sometimes get a bit side-lined in these relationships

10) Backlog: make sure the scope is clearly structured

  • Use story trees and backlog hierarchies
  • People can struggle with higher levels of grouping / scale
  • You need to get into the helicopter, go up and look down on the thing you’re building

11) Code:

  • Make sure you have access to it
  • Remain connected to it
  • Get a couple of the devs to give you a guided tour
    • Can be particularly illuminating to get newer team members to do this
  • You need to be able to visualise the entire code base (overall architecture)
  • How easy is it to run it from scratch for the first time?

12) What are the pain points and blockers?

  • Project manager is good person to talk to about that

13) Cross Functional Requirements

  • Resilience & scalability
    • Is it going to fall over with the load
  • Security
    • Is someone going to break into it?
  • Automation
    • Although automation is awesome, don’t fall for the trap of thinking you have to set everything up at the start
    • Don’t necessarily prioritise automation – but do prioritise what you need to actually deliver something
    • Once you have something up and running and iterating, it’s all good – but how do you break that initial blank sheet of paper?
    • You want pipelines etc, but as a principle, maybe there are different laws of physics right at the start of a project
    • Don’t spend six weeks building a pipeline with no actual deliverable – make sure you have something iterative
    • Imagine this product that you’re building… imagine you’re doing it with no software at all. Then bring the computer into it, and ask why – what is it going to give you? Where can it give you the most value? Don’t add stuff you don’t need
    • Imagine a walking skeleton in business terms – for instance, delivering whisky by physically posting it*. Your deployment walking skeleton can be over the top – only automate something when you need it
    • It’s a subtle art though – you don’t want to end up with tons of tech debt
    • You can spend a bunch of time working out the concept without having the capability.
    • Could have a simultaneous technical alpha – looking at cross-functional requirements etc
    • Try a walking skeleton of the architecture – a “scale model”

14) As a tech lead, what things are scaring you most about this project?

  • Do a futurespective graph with time on the x axis and clouds of uncertainty. If the cloud is really close and has a lot of lightning coming out of it… pay attention to it!
  • This is partly about risk management

* Whisky idea inspired by JK Werner.

How to Establish a Culture of Learning

How to Establish a Culture of Learning

I’ve pulled this out of a larger discussion. The detail of that discussion is not important, but a thing happened there which often happens in many discussions in IT: The possibility was raised that some people might be less well informed than they ought to be.

It’s a bugbear of mine. I think it’s a hidden menace in the software industry so I wanted to put something here.

And no, I do NOT think the menace is that people in IT are ill-informed. Quite the opposite: I think the menace is that we constantly judge one another for being (we believe) ill-informed.

I am often reluctant to admit that I don’t recognise a piece of IT terminology, because I think I might be judged for that. I think it might lead to people taking me less seriously in the context of whatever the discussion is.

But here’s the thing: There are thousands of books out there. Thousands of principles and phrases which encapsulate software development principles. Every single one of us will have books that we have not read and yet which some of our fellow colleagues believe are crucial to a good understanding of software development. Every single one of us will have phrases and terminology we have, by whatever chance, not come across before, even though they may seem crucial to a particular colleague.

There is a temptation sometimes to judge one another for not having knowledge of a particular thing. This attitude feeds strongly into impostor syndrome and the insecurities we all feel about how well equipped we are to do our jobs. The solutions to this are:

a) To rejoice whenever we encounter somebody who doesn’t have knowledge of one of our favourite things. It means we get to introduce it to somebody new! Hurrah!

b) To not assume that just because somebody has not come across a particular way of framing a problem or solution, does not mean they do not have knowledge and understanding of the underlying principles.

c) To always always feel safe to admit any gaps we might have, as that encourages others to do the same. This means we all get to learn more and foster a culture of continual learning and experimentation, which is crucial to good software development.

d) To have faith in the fact that we are part of a larger community of seasoned experts. This compensates for the fact that there is NO SUCH THING as a fully informed individual. We all have gaps. But together we can inform one another and create a powerful force.

Why pairing and other XP practices can mean that developers can move between teams with minimum disruption

Why pairing and other XP practices can mean that developers can move between teams with minimum disruption

A colleague and I had a conversation about moving developers between teams, and associated worries about causing disruption and losing knowledge / expertise.

I told the colleague about the book I was reading – Joy, Inc by Richard Sheridan, about a company in America called Menlo Innovations. Richard describes how at Menlo, they can easily move engineers between projects with minimum disruption. This is because they are used to pairing and knowledge sharing, they all adopt the same working practices, and no one person has code-base knowledge that isn’t shared by others.

Below are some extracts from the book that I think are relevant to this conversation. The basic premise is that pairing, a culture of learning and a focus on joy in the workplace all help to ensure that there are no towers of knowledge or lone genuises, and everyone can easily switch from one team to another.

I should add that I emailed Richard to ask for his permission to post these extracts, and he was extremely helpful.

Joy, Inc. excerpts are Copyright 2013, 2015 by Richard Sheridan. Used with permission. All Rights Reserved.

Chapter 12, Scalability (starts p194):

Brooks’s law is the motto of a broken hero-based culture that still largely exists in software development today. It states that the onboarding cost of a new person, particularly to a late project, far exceeds the potential contribution of that new person within the first few weeks or months of joining an effort. If a project is already late, then adding someone to it will make it later. The only alternative to a missed deadline then is an eight-hour-per-week death march for the current team.

… If you have no systematic means of transferring knowledge in a way that can quickly lead to mastery and autonomy for the new person, Brooks’s law applies to you, too.

…At Menlo, we have defeated Brooks’s law so many times that its premise is but a faint reminder of a quaint time in our industry’s history. Our entire process is focused on breaking this law. Pairing, switching the pairs, automated unit testing, code stewardship, non-hero-based hiring, constant conversation, open work environment, and visible artifacts all topple Brooks’s assertion with ease.

…Our pairing system, particularly the part where we switch the pairs every week, gives us a chance to practice scalability even when we don’t need it. We borrowed this concept from the airline industry (there it is called cockpit resource management). By switching pairs every week, we practise onboarding a new team member into a project even if the project stays the same size. We also practise onboarding new team members by bringing in four to six international interns annually.

…As the basic planning and coding practices are consistent across all projects, there is no need to train these new team members in any project-localized or personality-based methodology. The fact that everyone on the team has a shared belief system in our approach ensures there is no friction around these shifts in project assignment.

…Because the team is not based on towers of knowledge, because no one developer owns part of the code, because everyone has been working in all sections of the code, and because the people who have moved to another project are still in the same room and within eyeshot and earshot of one another, there is no meaningful loss of information about the project.

…How does a team get back up to speed when the project has been placed on the back burner for months at a time? … This falls back to documentation and standard work process. The automated tests provide the first foundation. They catch misunderstandings as new coding changes happen. If a programming pair returns to the project, makes a change they believe is okay, and one or more of the unit tests fail, the automated tests inform the programmers immediately that a mistake was made. This begins to refresh their memory of decisions made in the past. Code stewardship is also a big part of this, as the code has been viewed and adapted by so many different people over time that it has evolved into a body of work that reflects a commonly understood expression of intent, not a convoluted expression understood only by the hero originator and no one else.

…At Menlo, we measure cost against the final product: joy. We want the end users of our work product to thank us someday and tell us they love what we created for them. Most software teams never even get close to this effect. Can you measure something as intangible as joy? Perhaps not exactly, but you can measure usage, long-term quality, market share, customer satisfaction, market dominance, end user delight (at least anecdotally), and a lack of support calls. If we establish the team’s accountability to get these kinds of results every time, we get to joy.

Chapter 3, Freedom to Learn:

p52:

Assigning pairs and switching them each week helps avoid a variety of typical team ills that could interfere with learning outcomes. For example, it prevents team members from feeling alienated by another team member who may not have a particularly outgoing personality. It may not be a personality problem at all, just normal human shyness. We avoid cliques forming, as everyone gets a chance to work with one another.

We’ve found that people are more likely to engage in a casual pop-up conversation with others whom they’ve previously paired with for forty hours. This time spent together dispels a lot of misconceptions about one’s fellow teammates. Once you spend forty hours in conversation with another human being, you are going to get to know that person in a deeper way than a general impression could ever accomplish.

Working in pairs also provides an emotional safety net for our employees. As much as they are stretching the boundaries of what they know, they are also testing the edges of what is ‘safe’ in a work environment. When a new technology or process or domain is introduced, there’s always the prospect of fear creeping in, a natural human response to the unknown.

Imagine a six-year-old child on an unfamiliar walk near the edge of a deep, dark forest. The child will stand frozen and frightened, unable to move forward. But place an equally frightened six-year-old friend beside him and together they may confidently travel this unknown space together, hand in hand. Just like the buddy system we learned in childhood swim classes, pairing helps us move into the unknown with confidence and courage, comforted by the safety such a system provides.

p58:

We don’t tolerate towers of knowledge or lone geniuses at Menlo

p59:

The hidden benefit of giving your team members the freedom to learn is that, eventually, they will embrace the idea of taking the time to teach. Knowledge isn’t kept away in secret personal vaults like the crown jewels of little kingdoms. The workers on your team become more valuable as they learn new practices; they also become more flexible. Some conclude that this diminishes each team member’s unique value and turns each one into a cog in our little software factory machine. Usually, in the very next breath they tell us how amazingly valuable each of our team members is because of his or her mastery of so many different technologies and domains. This can happen only when you truly believe your team members are equally valuable and equally able to learn and carry certain knowledge.

Pairing is the atomic element of our learning organization. It produces a joy in learning that most of us haven’t experienced in years, perhaps since elementary school, when everything was new and all we had to do was absorb it.

Joy, Inc. excerpts are Copyright 2013, 2015 by Richard Sheridan. Used with permission. All Rights Reserved.