What, Why, How

Getting projects off the ground is something we all try to do all the time—from projects as small as a single change to the massive multi-year multi-engineer efforts we all know and love. For projects that take at least a week, most of us are familiar with the process of making a proposal/ticket, then a design doc, then writing the thing, then launching the thing, etc., and have probably tripped right into some of the pitfalls. I'm here, today, to talk about my theory on how to take an hour upfront so that you can start successful projects that solve the right problem in the right way and avoid some of the pitfalls.

I had an internship in which I needed to build a tablet app that rendered information on a sphere using Open GL ES. The tablet OS didn't provide a matrix library, so I had to write one.

I laid out the class and method names, wrote tests, coded it up, and then moved on to the real problem.

Several weeks later, my triangles just weren't behaving—I mostly had a sphere but triangles would just show up in weird places or be missing entirely. I spent weeks debugging this. And eventually I realized that Open GL ES expects row-major matrices and my implemented matrix library was column-major. That intern project failed because I jumped straight into how to solve the problem without doing enough research into what the problem I was actually solving was.

This internship story is a simple version of what a lot of us do all the time and what it costs us. It costs us time, it costs us impact, it costs us reputation, and it holds us back. Specifically...

That's really easy to do. You see a problem and immediately start thinking of some ways to fix it—that's our natural instinct as engineers.

So how can we fix it?

The easiest way I have found to do this is...

A week or more? That's a really short project! Yes, it is. But you know what only took a week and cost me a summer? Coding a matrix library for the wrong requirements.

And remember: a week of development time usually becomes a month of full delivery time. And estimates are really hard, so always take your initial estimate and multiply it by two, or maybe five.

For me, in my career, I have found that a week is the right tradeoff point. The other tradeoff point I look at is "is this risky"—that is, "could this break something important" or "am I unsure about the risk here". If so, then I want another set of eyes anyway. Your organization or team might be different.

A proposal should be a short document that explains a problem and why it needs to be solved. They are requests to prioritize projects—really, they're project initiation proposals. They aren't miniature design docs. They don't go into full details on all of the alternative implementations and tradeoffs. They aren't choosing an implementation or convincing anyone of the best solution—the goal of a proposal is...

You don't want to spend a bunch of time designing a solution for a problem that your team doesn't think needs to be solved at all.

In order to do that, we need to be able to answer a very important question: why? Arguably this is the hardest question to answer.

We'll talk more about figuring out what the problem is (you probably have a pretty good idea of that already), but let's stop for a second and talk about how to figure out if you should even try to solve a problem. Because maybe you don't need to solve it at all or it isn't worth solving.

Here are a few questions to help you figure out why something is worth doing:

1. Use the 5 Whys to avoid assumptions and logic-traps and get to your real goal—why do you really want to solve this problem?
2. Think carefully about the impact of solving a problem. What is the impact of solving the problem? What do you need to set up to know if you've completed this project successfully? What metrics do you need? If you're lucky, it's already measurable (such as "we can reduce our compute resource usage by X"), but if not, measure it.
3. Sometimes I find it easier to answer "why" when I think about the impact on people. How does solving this benefit your customers? What about your team, org, or company? How about the world?

Ultimately, you want to be able to answer: what is the real reason for solving this problem? If you can't answer that, then drop the project or, if you can't drop it, find someone who can answer it. After all, why spend time on a project if you don't have a reason to solve it? So keep these questions in mind.

Check out Tanya Reilly's talk "Continuous" for more on why "But why?" is such an important question and a great example of using the 5 Whys technique. Simon Sinek's "How Great Leaders Inspire Action" is also a classic example of why answering this question is important.

What you're doing and why you're doing it translate directly into your objective and requirements.

Especially at the start of a project, during the proposal phase. This is because...

A 2009 study on the impact of poor business requirements in tech projects found that tech projects with poor requirements often take more time while also missing their requirements at least half the time. Who wants to take longer for worse outcomes?

So that was 10 years ago, but it seems to still largely be true: the worse our understanding of our requirements is, the longer our projects take, even when you factor in the time to figure out those requirements.

The study also found that your development practices—agile, self-guided, waterfall, whatever—had much less impact on the project outcome than requirements maturity. How you do it is a lot less important than what you're trying to do.

Not only do they waste time, but...

Sources: https://www.zdnet.com/article/study-68-percent-of-it-projects-fail/, https://www.batimes.com/articles/the-impact-of-business-requirements-on-the-success-of-technology-projects.html, https://www.iag.biz/resource/business-analysis-benchmark/

And even worse,

A lack of communication—clear communication and expectations, that is—causes a lot of weird project failures.

So we now know that the objective and requirements are important. Let's talk a bit about what bad ones look like.

This is the most common antipattern I see in design docs and proposals. It's kind of like a math proof—if your solution is the objective or is part of the objective, how does a reviewer decide if it's the right solution or even a problem worth solving?

We get away with this all the time because people reviewing our docs probably already know what we're saying, but that doesn't make for a very useful doc.

Here's an easy test: if the objective is answering how something will be done, it's probably a solution, not an objective.

It's really easy to get caught up in designing the world. A castle in the sky would be pretty cool, but if what you actually need is a house with a place to park your car it won't work very well.

The castle in the picture here is actually a great example. This is Neuschwanstein, built in the late 1800s by King Ludwig II of Bavaria—also known as der Märchenkönig ('the Fairy Tale King'). Beautiful castle, inside and out, and the Disney castle is based on it. But it was so ambitious it was never finished—nobody properly lived in it and, though it was built in honor of Richard Wagner, Wagner died before visiting it. So Neuschwanstein is a perfect example of this—it was built as a tribute to Wagner, who never saw it, for Ludwig to live in, but he mysteriously died before living in it. It never fulfilled its minimum viable requirements.

That leads me to another common antipattern I see...

This is super easy to do—especially if you don't fully understand why the problem needs to be solved or what the real goals are.

So how do we do this properly?

Let's talk through the process. First, set a timer for 60 minutes or less (type "60 minute timer" into Google Search to start a timer). We promised ourselves this would only take an hour, so let's stick to that.

Then write down what you're doing and why you're doing it using a whiteboard, a notebook, a doc, or whatever medium works best for you. Take as long as you need on this step—if you don't get to the next step before the timer goes off, that's OK.

Healthy brainstorming looks like this tangle of lightbulbs. Give yourself permission to come up with all the ideas and figure out which ones are good later. Healthy brainstorming means throwing all the ideas down, even the ones you know to be bad. You can note why they're bad, but you don't need to—I find I have to to make my brain shut up about them, but feel free not to.

Bonus points: think more about who should actually solve it and when it should be solved. Maybe the project has a prerequisite or maybe your team doesn't actually have the context needed to do it properly.

Step 2 here is actually pretty critical. In general, it's really important to have someone you can bounce ideas off of—preferably someone with some context on what you work on that will help you suss out if this is worth pursuing or not before you spend any more time on this. Cultivating this kind of trust with a coworker is a phenomenal way to level up—you both end up with better results and hopefully a friend too.

And remember: deciding not to do a project is a win too. You just spent about an hour deciding not to spend a week doing something not worthwhile and that's awesome.

Here's a secret: what companies often really value is engineers who have initiative, drive, and tackle the problems that really matter. To do that, you need to think strategically instead of tactically. Writing a one-off script is a tactical choice. Figuring out that you have a class of similar problems you can automate together is a strategic one.

Note: sometimes tactical solutions are the correct strategic choice. For instance, you might decide that the best thing for your team/system/etc. is to put a quick patch in now—that's a tactical fix for a strategic decision.

Important: Getting into the habit of thinking strategically instead of tactically is critical to your growth as an engineer—and remember you can flex those muscles when reviewing proposals and designs as well as when you write them.

Thank you all for reading.