Laksh Ranganathan

Hello. It's an absolute pleasure to be here today at the virtual DevOps Enterprise Summit at Las Vegas. I hope you're having a lovely time so far.

I am Laksh Ranganathan, senior manager of product operations at Tasktop. And I'm thrilled to be here to talk about how we measure the right things to align to the business.

I want to start with this quote, which is a constant reminder for me that just because we can measure something doesn't necessarily mean it's going to provide the answers that you need. It goes on my whiteboard quite a lot to remind me to focus on the right things, really.

So how do you determine what you should be measuring? Let's take Sam here, for example. This chart gives you a sense for the miles that she has been walking over the past few weeks. Have a look at it and tell me what you think. How is she doing?

No matter what your answer is, I suspect it's likely true, because it really depends. If she's out to improve her fitness, she's doing fabulous. If her goal was to run a marathon next week, though she has done the right amount of miles, she has been walking, versus what she needs is to be running, so perhaps not as great. What if she was trying to prepare to climb a mountain like Kilimanjaro? That chart there doesn't quite provide all the information that you need, like what sort of elevation is she doing, which is necessary if you want to be preparing to climb a mountain. What about the weight she's carrying? Is she carrying a pack while she is preparing? And I think she would have to consider those in order to be able to determine if she's really ready for that.

This sort of goes to the crux of the disconnect between IT and business, because they have a different view of the world. So in terms of being able to determine what's the right things to measure, you have to first look at what is the goal? What are the questions that the business needs answering?

Organizations are hierarchical for a reason. As the number of people increase, you need that kind of a structure to be able to scale the operations. A consequence of that is that organizations, as they grow, become optimized for teams and functions, which is not wrong. You tend to have different departments for product management and development and testing. You have vendor dependencies, operations, and so forth. Sometimes you do have sort of full-stack teams, but that does not quite encompass every one of these operations. It's just not practical sometimes.

So when it comes to providing metrics aligned to business, you have to pull together metrics from each of these functions to sort of paint that full picture. And what ends up happening is each of these metrics, though accurate, are optimized for their own context, which means that they don't quite tell the story at that business level.

The business typically looks at things like, how do we satisfy our customer demand, especially when it's changing? How do we adjust and pivot to that? What sort of investments do we need to put in to provide the most value to our customers? How quickly are we delivering value? What is that rate? Notice that a lot of this focus is on the customer itself.

So the question that comes up often is how do you measure this value from that customer view? It's hard and it's difficult to articulate because it varies depending on who you ask, really. So the thing to remember is that value, when it comes to a customer, transcends across all of these functions.

So in order to be able to track delivery, you have to look horizontally across the end-to-end. You have to put that customer lens and view it from that customer perspective from request to release. And you can only do that if you apply a product lens because a functional outlook or a project outlook only looks at part of the equation and doesn't quite provide that customer perspective. You can only do that with a product or a value stream lens in place.

When we talk about a product value stream, by definition it has a customer identified and it produces a product, and because of that you tend to be able to describe value because you know who the customer is and what that product is that they consume from that particular value stream. And this is key because this also provides you clues on how to measure that value through business results which are associated with them, either be it revenues, renewals, cost metrics, quality metrics, whatever that is. But you would be able to relate that back to the product and to the customer.

When you think about products within IT, the one that comes to mind are business products, products that have an interface with your external customer. But not all products have to be external facing. In fact, a significant portion of products within an IT organization are internal facing. They serve internal customers, be it a business product, an internal team, or even employees to support them to manage their work. Typical supporting products could be a payment processor, for example, that supports all of your business-facing products, like a point-of-sale system. Or it could be standalone products like HR system, a billing system, a contracting system, which is absolutely necessary and vital for employees to be able to do their job. And it's still within the purview of the IT organization to manage.

And the third layer there is the platform products, tools and technologies, and basically any platform that is needed to develop each of those other internal or external products. Typical customers for platform products would be developers and specialists who derive value from them. And typical examples could be your IaaS, your CI/CD pipeline, your tools that developers use that need maintained. If you don't focus on some of those platform tools, it can directly impact how quickly you can drive value through the other products or push value to the customers of the other products.

To determine and chart out the products within your IT organization, the easiest thing to do would be to pick applications and start asking questions like: Who is using this particular product? What is the purpose of it? Who is deriving value? Who is driving prioritization for these? Who has an influence on value creation or value protection, like risk officers? And who is paying and supporting this product?

Asking some of these questions will help identify the customers and the influencers for this particular product. And in turn, it'll determine the right business outcomes for each of those products, which in turn can then give you a sense for the impact that it has on the customers. And that could be the results that the business desires out of that particular product.

Once you get to a point where you have some of your products identified, it then becomes much easier to now start identifying what metrics will relate and drive that business value for that product.

Now, there's no shortage of metrics within software delivery. Most dashboards look like a plane's cockpit controls with way too many measures in place. Let's look at this view here. Both are a view of a bank statement. One, a summary view, the other is a detailed view. Which one do you prefer? For me, if I was looking to find out how much balance I have before I make that big payment, and I just want to make sure that I have enough funds to cover, that first one does the trick. However, if my focus is to cut non-essential expenses, then I would be looking at the second one to figure out what changes I need to make. So the metrics for the business are no different. You just need to be able to align the right level of detail and align it to the context.

And this is where flow metrics come in. Your standard software delivery metrics are great and are absolutely necessary because they provide vital information about team-level functions like processes, team-level decisions like capacity planning, utilization, optimization, standards, and so forth. However, they tend to have too much detail to be able to easily align to business results.

And this is where flow metrics come in, which provide a more aggregated view of those metrics in such a way that it's correlated, or which can be correlated, to business outcomes easily. The two are two sides of the same coin. You can't have one without the other, and you should have both levels of metrics. The difference is in the level of details and the context in which each of those are used.

So if you look at flow metrics, their focus is that summary view for that value stream, and they align to the business outcomes. And more importantly, they are focused on predictability and efficiency and identifying where strategic investments need to be made. The software delivery metrics that we currently measure are great and needed to measure your day-to-day activities and focus on things like capacity utilization, process improvements, and individual team improvements and so forth, and provide that deep-dive layer on outputs that can impact outcomes.

So if we look at what constitutes flow metrics, we have five core metrics that you can look at. Velocity, which gives you a sense for how much value-adding work you have delivered to a customer over a period of time. Efficiency talks about how are my processes doing. Are teams spending too much time waiting before they can complete their work? And where are those wait states? Flow time provides that customer perspective of what is that time between request to release, and how quickly are we delivering value to customers. Load focuses on work in progress and can be a leading indicator of challenges within a value stream. For example, if there's a lot of demand for a particular product and not enough capacity to take on, load can provide that insight to figure out what adjustments need to be made to address that concern. And distribution perhaps is one of my favorite ones because it gives you a quick sense for how you are breaking down your finite resources across the types of work that you do within a value stream, across features, defects, risk, and debt.

Much like the vital signs that a doctor takes when you go visit them, the flow metrics provide that pulse of the value stream at that first instance. You can't rely just on those metrics in isolation, because in isolation, they're just pretty pictures. What you need to be able to do is align them to business results, anchor them in business results, without which they're not going to be valuable at all. And this is absolutely important because what the business is looking at is how the value stream is performing against those results that we want to measure its performance on.

And another key factor is to keep it real time. Stale are the metrics. The meaning and the insights that you get is not going to be valuable enough to you.

Once you get to a point where you have your metrics measured, some of the questions that come up, especially around load and distribution, is the factor: why doesn't size of work factor into load and distribution? One of the key reasons is that size is a very team-specific metric, team-specific context. Every team does sizing differently, and they batch it differently. The other reason is that from a customer perspective, size is invisible. They only see what value they get out of a particular feature or a fix that has gone in. So while the size is important when you do your team-level metrics, it is better to focus on value when it comes to the flow metrics. And the value is measured when you look at the number of items delivered rather than the size of it.

Let's take an example of Tasktop itself. Over a year ago, we had to spend a whole cycle, a PI cycle, on architectural changes, which meant we had to earmark a significant capacity on debt work, which left us with very few cycles for feature work. So we decided to do what we call quick win days. Over a period of two days, we identified all the features that had been in the back burner, which would have taken an effort of two to eight hours to be able to tackle them within a day, so we can pull those together.

So from our perspective, from the developers' perspective, this was great because they had the opportunity to focus on debt work, which they hadn't been able to for a long time. But from a customer perspective, they saw 50 new features come in across all components. Pretty much every customer had at least two or three features that they could utilize, which meant our customer NPS was through the roof.

So just to give you that sense for how it did not matter how long it took to complete each of those features, but each of those was quite important on their list, and it enabled them to do their work on the product much better. So this has now become a regular feature in most of our cycles now to be able to tackle some of these quick wins as we go.

So the key there is that value doesn't always correlate to size. What's better is to be able to prioritize what impacts business outcomes, and these tend to be value units, whether it's features, defect risk, or debt, which will thrill and provide that benefit to the customer rather than the size. And if you want to gain this metric, reduce the batch sizes, because if at a lower batch size you're able to deliver features to a customer and close that feedback loop quickly, that actually works for the benefit of the customer and for the product, and which is much more desirable for that value stream.

The other consideration is that at the end of the day, if you have a lot of load, no matter what the size, you can only take on one at a time. One person can only work on one at a time. They would have to stop one to pick the other. So at the end of the day, across the value stream, you have to look at the quantity or look at the number of items you have that you need to pull through from your backlog, and the size, generally, at a value stream and flow metrics level is less relevant. Sure, on the team level, when it comes to capacity planning, it is. But for flow metrics, you're better off focusing on how many versus how big.

Once you have flow metrics, the other question tends to be now which one of these metrics do I focus on? It's challenging because there isn't one metric that rules them all. And let's take an example of this customer who had pulled through their flow metrics. And initially, the first observation was they have great flow efficiency. It was in the 90s, and top 90s actually, which suggested that they're doing great in terms of being less wasteful of people's time. There's less wait times, and they're very efficient in getting things done.

However, that story didn't quite correlate with their flow load because their load kept rising, their flow time kept rising. So it sounded like everyone had at least three or four items that they are actively working on, which does not make sense. No person can work on five different things or four different things at the same time. So as they delved into what those states are, and they utilized their drill-down metrics in the individual tools, they realized that their workflows were not adequately tagging wait states, which meant that the flow efficiency was artificially up.

Well, if this customer had only focused on flow efficiency, they wouldn't have been able to see the problem. And what you need to be able to do is look at one metric but correlate that with the others so they paint the same picture.

The other key thing is to focus on trends. What works for one particular product won't work for the other. You might have to focus more on defects for one older, mature product, versus you're probably pushing more features down a new product. So instead of focusing on absolutes, focus on trends. That gives a better gauge of what's right for that particular product. And finally, when it comes to frequency for improvements, it's better to align them to business results to identify how they demonstrate outcome rather than focusing on setting just levels on individual metrics. What you want to be able to do is set a level on, say, velocity, but correlated with results, which is much easier and much more realistic to track.

If I had to pull one slide to summarize some of these points, I think it would be this. This would be that one slide that you would want to take away. So I think the crux here is that determine what the outcomes desired are because that will enable you to identify the right level of metrics for your business partners. And anchor them to business results. Whatever metrics you choose, if they have that business context, they paint the right picture.

Leverage flow metrics that are at an aggregated and summary level, but are correlated to business results to provide that business visibility, rather than throwing every metric that you have on a dashboard, which might dilute the message that you're sending.

Value does not always relate to size. So focus on value units when you're looking at flow metrics, and focus on size when what you need to be doing is capacity planning and utilization planning and identifying individual bottlenecks within a particular function or team.

And finally, focus on trends because they are a better indicator. And don't focus on one metric. Rather, look at a few of the metrics together to make sure that they paint the same picture.

That's all from me. Thank you so much for listening, and have a lovely rest of the conference.