An Introduction to the Invariants of Building Software

Opening Acknowledgement:  While I am the person authoring and posting this series on software development, these are not solely my ideas. Notably there are several people from Stack Overflow, Unchained, and Zaprite that have helped me form these opinions on how to build software. Some of them even named things I mention here as we developed our own identity as respective product and engineering teams. Those people are: Joel Spolsky, David Fullerton, Matt McManus, Parker Lewis, and John Magill. Notably, the atomic unit and invariant framing was developed by Joel, David, and myself as we battled to scale the product and engineering org at Stack Overflow, which I later built upon with Matt McManus at Unchained. The Invariant framing was a collaboration with Joel Spolsky internally at Stack Overflow, but I believe those initial internal writings are long lost, and this represents my own take on that framing.  

Introduction

If you’ve ever been on a team building software (product, design, engineering) or tangentially related (marketing, sales, ops), you’ve likely wondered if you’re doing it all wrong. Things take longer than expected, the end product isn’t what customers wanted, and getting to the end required Herculean efforts that are not repeatable. Because someone involved read a book one time, you get together a week or two after a project ships and do something called a retrospective, where no one blames anyone, and then you do the same damn thing all over again. If this is you, I want to propose an alternative way to think about software development.

Perhaps greater than the holy wars of linux vs windows, tabs vs spaces, emacs vs vim is ‘how the hell am I supposed to get seven people to agree and ship this damned product’ vs ‘holy shit I have to get 78 people to agree and ship this product'. The latter holy war has high priests known as Scrum Masters, Product Managers, and Product Owners. They have read the Good Book of the One True Product Development Methodology, and as long as you don’t sit during a meeting, you’ll ship on time. Or you won’t. You probably won’t, because methodologies don’t really matter. 

I am concerned with the state of product development. Planning has either been demonized (you can’t ever know anything so just write code) or trivialized (our planning is a to-do list of tasks and features on a kanban board) which cripples the abilities of software teams to consistently do their jobs well. Waterfall, or anything that smells like waterfall, is grounds for mutiny. In this world where everyone is agile and we move fast and break things, we’ve handicapped the engineers, designers, and PMs who make software work, and we’ve given a pass to CTO’s and VP’s of Product who are supposed to, well, know what to do.

That’s why the companies I’ve worked with invented something that was not a methodology, but a process that can fit differing methodologies. If you’re on a product team shipping features, you may use Scrum or Shape Up. If you are on an SRE or DevOps team, a simple Kanban methodology may serve your purposes just fine. Once you’re at the point of defining how you and your team build software, your only actual limit is the capabilities of the people in that team. Nearly any methodology could work if followed by hard working competent people. I argue there are things good software teams do–I call these Invariants–no matter what methodology is employed, what title the guy leading the meeting has, or to whom a software team answers.  

An Invariant is a motto,a guiding light that–no matter where you’ve found yourself in the agile to waterfall spectrum, and no matter the rituals your product team follows–will help you ship quality software consistently. The Invariants themselves have their own proto atomic units, which are the primordial soup by which Invariants are built.

Atomic Units

All high functioning product teams, regardless of org charts and specific methodology do some form of these five things in their product development process:

• Strategy - Set vision and goals, and figure out how the product teams are going to move the business forward.
• Discovery - Iteratively figure out exactly what we want to build, and how it’s going to work in very specific detail. Show early versions to people.
• Build - Actually implement the thing you designed in the discovery phase with lines of code. This should be the easy part.
• Quality Assurance - Make sure that what’s been built both satisfies the requirements, and also works well within the existing system without unintended consequences. Show the software to people.
• Delivery - Deploy the new product, communicate to the world it’s here, and get people to use it and give feedback for future development.

Differently sized teams and different products will have slight variations, but there’s no getting around the atomic units of product development. The key to understanding the atomic units, and therefore the Invariants, is that we try very hard to solve problems at higher levels of abstraction before we move to problems at lower levels of abstraction. 

You Said Phase, Is this Waterfall?

No, it’s not waterfall development. We’re salmon, we can swim upstream.

If the strategy is wrong and we’re already writing code, we’ll fall back to strategy. If we’re ready to ship but have a design issue, we’ll fall back to discovery. We expect this to happen sometimes, because you inevitably discover things later on.  

This is the core argument between the agile and waterfall camps. The agilistas pervert the agile manifesto and claim you can never know anything so why even bother writing stuff down and that changing code is actually fun and easy!  While the waterfallistas claim that everything will be known in advance, nothing ever changes, and programmers are people that translate english words line by line into code that computers understand, and also what is a customer?

Both arguments, even when told in less absurd ways than I’ve chosen, are entirely unconvincing. As salmon, we can navigate both ways through phases, while trying our best to solve problems at higher levels of abstraction before solving lower level problems.  We can also prioritize getting working software into customers hands as quickly as possible and iterate on their feedback as quickly as possible. We’re agile, we just Think Before We Do.

So What, Doesn’t Everyone Do This?

Hahahahahaha.  No, almost no teams do this, at least not consistently over a long period of time. If you’re skeptical, do three things for me real quick:

1. Find the last three specifications you wrote or worked off of
2. Find the strategy artifact that led to those specifications
3. Ask someone in marketing what the two most important things the product team is doing right now

You’d be lucky in most organizations to get positive results on one or two of these challenges. If you check all the boxes, give your product and engineering managers raises. They are special.

The reason people don’t do these is that they form their org charts and their product development processes on a tactical basis that unintentionally results in a lack of planning and strategy.  Product and engineering teams have been hoodwinked into following tactical guidelines that have us reinventing to-do checklists and calling it strategy (I do love checklists). Worst of all, we avoid accountability when everything is tactical. This in turn makes the actual building of software a nightmare, where code is constantly being changed and no one can remember why features are being built and for whom they’re being built.

Failure at a tactical level is supposed to happen and doesn’t doom a team or project. As Peter Thiel states, failure at a strategic or systemic level is a tragedy, and no one gets better from it. If your team’s process does not have strategic ownership that is owned by known individuals, true accountability is impossible to achieve. Now we’re venturing into invariant territory, so let’s get to it.

The Invariants

If the atomic units are the tactical descriptions of the things all good software teams do, the invariants are the strategic stories of why and how these tactics work.  Each atomic unit has a corresponding invariant that we’ll discuss at length through this series. The invariants are:

1. Think, Then Do - Strategy must be present and written down so all key people on the team are aligned.
2. Don’t Design Without a Strategy - Discovery is the act of determining in precise detail what is going to be built. This is how we get designs and specifications.
3. Don’t Build Without A Plan - Building is where we start writing code, and we try to do as little of this as possible without a plan in place.
4. Do What You Said You’d Do - QA is our accountability check before we go to market. Our last chance to put our best foot forward.
5. Make Products Earn Their Keep - Delivery is bringing the product to market, and listening for what the market says in return.

These brief descriptions of each invariant are not enough to understand this process though. You’re going to have to bear with me and read this five-part series where we break down each invariant in as much detail as I can provide. I promise that if you stick with me, it won’t matter if you’re sitting or standing during meetings, you’ll have a better chance of consistently shipping quality software that matters.

Next Up: Invariant 1 - Think, Then Do

If you like my writing, I accept tips via Zaprite, which makes bitcoin payments easy. It's the product I'm working on these days.