Troy Magennis

This talk's about prioritization.

I was very good at math in school and very bad at athletics, so this talk isn't going to get deep into the mathematics. It's more about how to think about the important job of choosing what to start next.

So this is Donald Reinertsen. He wrote a book a few years ago. He's written a couple of books. The Principles of Product Development Flow is the most recent. And one of the things that he stresses in that is that whenever we choose to do something, we actually also choose not to do other things.

And he sounds right, and that sounds reasonable, but it's a very easy thing to fix, and I don't understand why he didn't get it. You just need to employ unlimited people. You just need to actually get enough staff to do every single thing that you've ever wanted to do in your backlog. And so I don't know why he didn't come up with this, but it's just like hospitals. We should just build hospitals with an ER to cater for mass casualty events. It's easy to fix.

But we know that that's silly because the reason we need to prioritize is that often the demand on the things that we have to do, work, is greater than the capacity. So it's the constraint that causes the need to prioritize. It's the constraint which causes the need to actually choose to do something, and therefore choose not to do other things.

And that's what this talk is about today. It's about how to understand the two edges of the sword. Once we find a constraint that it's uneconomical to fix, we now need to make a harsh decision about what we spend those resources on. So it all comes down to constraints.

So this is the goal. If we are going to choose to do something and not do other things, how can we do it in a way which is advantageous to us? How can we do it in a way where we can actually choose something that's more beneficial? Because, as I sort of said, the challenge is that once you choose to do one thing, you have to not do other things.

And what we're going to talk about today is how to come up with very simple models, the simplest model possible that you can use in your context to try and understand: what is the impact of something being delayed? Because that's what we want to do. We want to understand, if I delay something, is that more expensive than delaying something else? It's all about the delay cost, not about the value, and that's what we're going to be talking about.

So this is what I always get: given that if we just sort of tried to do things in value order, what would go wrong? And the first thing I get is that we can't quantify value here. How do I quantify the value of a refactoring item, or burning down some technical debt, or doing a certain type of automation of one component and not the others?

Well, it turns out that people are very good at describing value once you threaten them.

So given a blank slate, it's very difficult for them to come up with value A versus value B. But if you say, "Well, given that you won't give me order, I'm going to use my order, and my choice is to do the work alphabetically," they will start then saying, "But you can't do that, because this thing will solve this problem," or, "This thing will solve that problem," or, "This thing will earn this value."

So threatening alphabetical, people are always better about critiquing rather than creating from scratch. So a lot of the best techniques I've seen for getting people to articulate value is by just assigning something an equal and random value and then having them sort of say, "No, that's not right." They seem to be very good at saying it's not right. And that's the discussion that you want to run.

So this is from a case study done by Joshua Arnold and Özlem Mus at a shipping company. And what they did is they looked at the backlog of items, and they looked at how things were prioritized now. And that company used the MoSCoW, the must-haves, the should-haves, and the could-haves sort of model of describing work, which is great.

But what they found is that the difference between the top 25% of the items was much more, over ten times more: $230,000 a week of opportunity loss from the next level down. So just the difference between the must-haves and the should-haves was the difference between $230,000 a week to $18,600 a week in opportunity loss.

So the typical techniques of just assigning things to buckets like that undersells the value of the very high end. There are a few items in your backlog, or the work that you could do, which have an extraordinarily high payoff factor.

So this is what we're trying to solve. We're trying to understand what's in this first bucket, and what I find is we argue about what's in the bottom bucket. So a lot of the stress around putting value on things are the fact that there's very little value between them because they're in the bottom two buckets of money impact.

And that's what it looks like. When they plotted it, you'll see that on the left here, this is the cost of delay per week, the amount of money it costs them not doing it, and this is the number of requirements that were in the backlog. And you'll sort of see there's a few really high earners and hitters up here, and that's what we're doing.

When we're prioritizing, our goal is to not miss those obvious things. It's not about getting a perfect order. It's about not screwing up. It's about not missing those obvious things. And what your job is to take back is to stop people arguing about these. Pick them at random. It really won't matter, okay?

So these ones, picking at random will matter. If you miss these, you're incurring a very high delay cost.

So this is the first thing you can do: just start capturing the historical value of certain features in hindsight. Stick them on a wall: feature one, feature two, feature three, and just give them a small description, so that when someone comes along with another feature and you're trying to understand where it fits in the value continuum, people will very easily be able to compare against it.

Well, it sort of does support checking out, or it sort of does support automating an area where we have a high number of defects. We know in the test area, we want to reduce the number of failures. People will very quickly find the right spot given good reference features and stories and prior deliverables.

And this is a very low-effort thing to do. All you're doing is reserving a wall for posterity and helping people remember and recollect what the outcome was of doing certain features over another. And you're giving yourself the opportunity to bring all of that company knowledge along with you over time.

So even if you get new hires, they will equally be able to estimate the value of a feature, not even knowing that there was history. So this is about keeping that intellectual capital inside your organization, of understanding how value turns out. So now you're not having to crystal ball as much.

So when I went down this pathway, I really did want to understand roughly what all of the arguments were. So I found the most violent and aggressive people on Twitter who were arguing about this topic, and I started an email chain saying, "I think we're losing friends on Twitter. Let's do this via email."

And what I wanted to learn was why they were so passionate about their way and why everyone else using other ways should burn in hell, and why it worked and when it worked on the case studies.

So that's what I did. What I wanted to do was I wanted to create a single-page handout that I could give to people to capture this knowledge. I'm not getting any younger. I have memory issues. So single pages help me remember and look smarter when I go into clients and do some consulting in this area.

So this is what we did. We ended up coming up with, and it took a year. Actually, it took over a year, about a year and a half. It was like being a relationship counselor at times. They were all in different countries, but they were different.

So what we came up with was this single-page PDF where we did everything from random choice all the way up to, in hindsight, perfect selection, perfect prioritization. And then we documented the pros and cons about each one.

Now we're going to see a lot more of this, so don't get worried about it. But if you want to download it, it's just Bitly, Better Prioritization, capital B, capital P. So if you get that, you won't have to read the slides, which you can't from the back anyway.

This talk is not going to go into detail about any one of these techniques. It's going to be giving you the thought process that goes behind choosing what technique will work in your context. Because this is a very contextual question, which when you're just tweeting it to a couple of thousand followers is completely contextless. So I want to stress here that there is no one right answer here.

So we did the front page. But me not being known for brevity, also, if you printed it double-sided, it still says one page, so I'm technically still correct. But on the other side is a bit more detail about the main ones, about different techniques that you could use to get it.

So you've got enough detail to get started, and I found with just the front page, you had enough detail to do damage. So we put together the reasons and a little bit of dos and don'ts on the second page. And again, Bitly, Better Prioritization. Just grab it.

So what did we come up with? We came up with everything from random choice to crystal ball, and then we grabbed all the techniques that we could find and come up with and argue about, and we laid them out along the axis.

On the left here, the chance of a suboptimal decision is at its highest, but its effort is at the lowest. Random doesn't require any effort. Just do it alphabetical. At the other end, there's a lot of effort, but your chance of making a suboptimal decision is lower, at its lowest, being at the crystal ball side of sight.

So this is what we tried to do. We tried to come up with what they were, roughly make them, whether it's subjective and how they sort of fit, and put them from left to right based on the chance of making a suboptimal decision and effort.

And this is what you've got to do. If you learn one thing in real life, or statistics, or anything like that, it's that being at the extremes is never right. The answer is always in the middle somewhere, especially when you're dealing with problems that are so contextual about prioritizing work in your organization, which I can know nothing about because I don't work with you all.

So the thing that stuck us most of the time is every single one of these techniques can be done poorly. And if you do them poorly, you'll get a crappy result. So we are going to assume that you are going to invest the time to actually learn these techniques and apply them without fear or favor.

You want to actually do these. So yes, you could disagree with where they are on that continuum, but our base assumption for placing them where we placed them was the fact that if you actually tried to do it well rather than trying to break it, you would get a good result.

And it comes out like this. It comes out that that continuum is, we start off with just intuition. Highest-paid person's opinion is probably the lowest that's most used. You have a product owner. They choose. That's great.

The highest-paid person, the HiPPO, doesn't always have the larger hippocampus. And this fails if they don't have all the bits of the information they need to understand the problem they're trying to solve.

So you'll see very quickly we move into understanding value, and this is the most common, and this is the one you hear about: prioritize your backlog in value order. Work out the return on investment for the items.

So the thinking process I want you to go through is the fact your job is to understand, to use the one on the leftmost possible. Your job is to understand, in our context, what's the simplest way I could break this?

And with value, the simplest way to break it is the fact that not all stories are measured in dollars returned. Some are measured in dollars saved. So the next step up is that we start increasing what value means, and we encompass more of cost savings as well as revenue-generating items.

And then we sort of say, "Well, what could break that?" Well, things that are of the same return, the same value, one could take a year and one could take a month. Which one should we do first intuitively? And you start thinking, well, if they're the same value, let's do the one which is fastest to give me that value, which will be the one of one month.

Again, all we're thinking about is what could break the prior method, what could give us a suboptimal decision. So then we sort of start getting into how we sort of break value down, and then right up the very high end there, we start looking out long-term, that we may not get immediate value for doing something, but it may set the company up in the future to be able to take on a new style of customer, or a new style of work, or accept something, or defeat a competitor, make it hard for a competitor to compete.

So on the very right-hand side, what you're really modeling is the whole competitive landscape and the impact of doing one job over another on that landscape. That takes some time. That's why it's the highest effort. But if you're in a certain context where it matters and you only can invest in one item or you're going to go out of business, it may well be worth choosing that item carefully.

So again, stick away from the edges. That's normally too much unless it's a do-or-die decision. Normally, you want to make sure that you actually do understand how value is created in your organization and know that there's not one type of value, and you don't want to disregard that duration matters.

The time to actually deliver it will block the development pipeline or delivery pipeline of delivering something else. So you want to make sure that doing two smaller things won't amount to a better financial outcome than doing one big thing. So that's why I think you have to have multifaceted value and worry about duration.

So how do you choose? Well, let me just give you a quick rundown of some of the options you have here, and then I give you some pretty explicit sort of advice at the end there.

This sounds bad, but it really depends on whether that highest-paid person's opinion, how much context they have on the industry and the marketplace that you don't have. They may not be the best communicators, but if they're the founder and they've created a whole industry, they're probably on a journey which they know inside their head.

So I've actually seen this work quite well on some quite large companies. Thinking of one here, Tableau. Tableau was founded out of a PhD from the founders, and they pretty much have an unfulfilled vision of what they want to deliver.

So us getting in the way and sort of saying, "Oh, but you need to put a value on that undefined, on that journey," just loses friends. It doesn't help. And they're probably right anyway because they do have such a contextual knowledge of the market.

So it's not wrong unless you don't have access to the founder or someone with that strong visionary feel that has a track record of getting it right.

So this is where Scrum comes in. We sort of say, well, now we parcel that out and we give it to different product owners, and now we expect them to actually play part of that journey. It's great again, but the product owner has part of the story. They probably understand some of the value that it's going to generate, and the good ones will go out and investigate where the other value is coming from.

But they're probably not as good as understanding the options that could be used to deliver that feature and how long, and trade off which ones will be easier and harder to do. You need the technical skill in that area. So that's where we sort of start moving on to needing to balance duration and value.

Now, I'm going to back up a few slides because I want to point out one thing on this sheet. Let me go back one here. This little matrix technique where you just said, grab all the features which are in contention, or the options that you have, and you load a high value and how long it will take to actually deliver or block the pipeline, and just get people to start sort of placing them where they should fit roughly on how long it would take to deliver, how much end value it will give.

This is pretty much the easiest thing you can do, even with your product owner, to start balancing out the technical effort and the value. And your job is the ones that are fastest to do, which get the highest value, you would do first, down to the ones which are really large and give low long-term return.

So that's where we're sort of heading there. Let me just get back to where I was. That's what this relative value and effort is, that matrix.

So that's the first one I would say is right to do if you don't have access to a very strong founder-led organization.

Then we get to the SAFe values. What we found, this was where we spent most of our time arguing.

Now, I'm a math person. The SAFe formula out of the box is terrible. But what we don't tell is the story of, it will still give a good result because there are some compensating factors in the SAFe process that don't allow the reasons it would give a bad result to ever reach that point.

So everything's right in context, but if you just go to their website and you don't attend their training, it will mislead you badly.

So what we wanted to do was come up with a way, or challenge them to sort of say, "Well, what could be the simplest thing we could do to change that formula to make it safe without education?" And what we found was there was a very easy modification of it where it used to sum together multiple factors. We multiplied priority against those other factors, which meant that now you didn't get that counterintuitive result of something is the highest priority but never chosen because it just had low value.

So think about these things. Think about, if I'm going to use one of these techniques or these formulas, what's the simplest thing that will break it? And in our case, for the SAFe formula, it was that the highest-priority things could still be lower prioritized than things of just high value.

And then when we ask, does that ever happen, the chances were, no, we would never do that. It's high priority because if we don't do it, we're opening up a security vulnerability. That should go before getting the cool new Bitcoin checkout feature, even though the Bitcoin checkout feature may or may not return investment.

So that's the thinking process that went into this sheet, and it's all documented on the back side of the one-pager.

Now, the other one we went to was, any Kanban people in the room? People use Kanban? Any of the early sort of side of the David Anderson style of class of service, where they looked at things may be of no value now, but if we don't do them by a certain date, the world will end.

So there is a point here where we looked at the SAFe, all of the ones to the left of the cost-of-delay curve ignore the fact that cost of delay can change over time. So it's the first time in these methods where due date become important, and that things could have very low sort of cost of delay or low value, but if we don't get them done by a certain date and our competitor does, it becomes a nightmare for us to respond to.

So that's sort of the next level.

What happens is, Joshua Arnold has another set there where he tries to combine these two together, where you've got the SAFe style of the multifaceted value. It handles how long things are going to take, and it looks at what's the impact in the future. It averages it out, but it's not perfect.

Now, when you get to the very right-hand side, every one of those factors is taken into account, and it's only going to be your imagination. You build a spreadsheet, you go to finance, you understand the model, you understand by putting a feature A in, you're going to grow market share by a certain amount, and that will project out into the future.

You get what I'm saying. It's hard work, but if this decision is life and death, it's the right process to use.

So let's look at it. This is what we came up with as the why and how you would choose. One is if you're in an emerging market and you don't have a lot of time to react, otherwise a competitor will, you're in this row. If you're in a mature market and your customer base is pretty stable and your competitors are pretty weak, you're up on this top horizontal scale.

Now, if you're cash poor, you're a startup, and you only have fixed amount of money, or you can't change the budget, it's done yearly and you're in no control, you're in this left column. If you are really good at building a business case and getting the funding you need to expand the team to do more, you're in the right-hand side.

So A, the cost of being wrong is fatal. You will lose the business. You don't want to make a wrong decision.

B, if you choose wrong, you're going to slow your growth. Remember, it's going to be a big factor, and it's certainly going to be larger than being slow in a mature market, because in the emerging market, you need to respond faster to get features out to grab market share.

But B and C are where we spend most of our time. D, if you've got unlimited cash and you've not got really strong competitors, it really doesn't matter. Toss a coin. Do it alphabetical. You're going to get a similar result in the end of time. It's not going to change the outcome.

But this is where we are. This is where most of people in this room will be. They'll be in B and C. And that matters because I find all the cases where we argue online and on Twitter are A and D.

So the quick way to understand and to get through the emotion when I was doing this in an email forum was to remind people whether they were, yes, I can find cases of every method here being needed for A and every method here being needed for D, but we're not there. We need to give better advice to people about how to do it when they're in these other areas.

So just remember that. So we're in C. C is you're in a mature market with fixed investment. B is you're in an emerging market and you have cash to burn. Startup, well-funded land. Companies my age or older.

So this is what we came up with. We then put a background on here and said the greens don't really matter. Just let the highest-paid person make the decision because they're probably going to get it right.

And at the other end of the scale up here, the ones where if you get it wrong, you're out of business, you should be doing real estimates using the same values. You shouldn't be using the story points. You should be doing full economic impact, and you should at a minimum be using something in that white paper that's referenced here, blackswanfarming.com, cost of delay divided by duration, CD3, because the stakes are important enough that it matters.

But most of you are in this sort of area. And I want you to probably, at a minimum, you need to start looking at balancing duration and value. And you should not use the SAFe formula out of the book, given that the only difference is changing addition operator to a multiplication operator. Use the modified version in all cases.

And look at the Kanban side of using the different delay factors to understand where you're going on how will this value and delay impact change over time.

This will happen a lot. There will be emotion around each end of an argument, and you really should be worried about the center. So always think about, I want to be the leftmost one as possible to do the least amount of effort. That might be me because I'm Australian, just nature.

But you also want to say, what's the simplest thing I could do to break this model and choose something that defies intuition? If you do that, you will always choose the right value or the right technique.

So that's it really. We do a set of dos and don'ts on there. You've got to start looking at prioritization as an economic decision. You've got to use the lightest method that you can to get the decision that's adequate. You don't want perfect, because remember, we saw that we're just trying to not do the wrong things and to make sure we don't miss the no-brainers.

Have a conversation about it, and you've got to invite other people to add diversity to the opinion. Don't go just sticking to your dev and your product owner. Go and ask your support or your help desk. Go and ask the people talking to the customers. Go and run an experiment.

When you do those experiments, put them on the wall behind you with the experiment and the rough value you saw from a certain type of feature, so everyone remembers what it is. And now when they're given another feature to do, they can just affinity map it to closest to what a feature similar to that was.

And the don'ts: don't go using complex analysis if it's not going to matter. So understand where you're at.

You can't ignore delivery time. Remember, when we're choosing delivery time, we're blocking a whole pipeline from doing other stuff. It's really important to not overlook that.

And don't create a race for value. Create a race for balancing and understanding why you're doing certain work.

And use the highest-paid person's opinion. I'm a consultant, so I can act as your highest-paid person's opinion. If you actually ever need me to do that, I'm available.

With that, I think I have a minute and four seconds for any questions and comments or aggression. Sir.

Q&A

Q: Is there a concern with option D here, for your rich and mature, which is just adding people? Doesn't that add a lot of risk, like—

A: Yeah. The question was, adding people, is it a golden ticket to prosperity? No, it's not. Adding people has its own risks. It's certainly not a linear benefit.

So I find, depending on where your constraint is, and if you add people at the constraint, you will get a benefit. But I find that very rarely happens. We add people not at the constraint and have a very low improvement.

So add people at the right places. If you could do one thing in prioritization here, it's you always know where your constraint is and make sure you add people at the right place.

Cool. All right. Thank you. Go to lunch. Get there first.