Naomi Lurie

Hi, everyone. My name is Naomi Lurie. Thanks for joining me today, last session of the day. I appreciate your attention. I'm the VP of Product Marketing at Tasktop, now Planview since we got acquired, and today I'm going to be sharing a lot of cool data. Some of it is survey data. Some of it is systems data, to tell you the story about how the shift from project to product is actually going.

For those of you who were here in the previous session, a little groundwork was set. We talked about value stream management and project to product. But people have been doing this for years, right? We've been talking about this for years. So really we've come here today to kind of see how it's going.

If you've been to DOES before or attended the keynote yesterday, you may have seen Dr. Mik Kersten. He is the CEO and founder of Tasktop. He's the author of Project to Product, so that gives us our expertise on the subject.

Let's get started. To set the stage, back in 2018 there was a survey that Gartner conducted, and 85% of respondents said that they were planning to adopt a product-centric model or were already doing it. So here we are four years later. Most organizations were preparing this journey way back then, so like everyone else we were wondering: how is it going? How are they doing? How are people traversing this complex process? Because often we talk about how to start, but we don't understand where it ends and how we get there.

Now that we've been doing this for a while ourselves, we're going to share some of the findings that we've had. Today I'm going to take 10 minutes to talk about what we've observed as the five stages in the shift from project to product, and then we're going to talk about the dimensions of change that organizations have to go through as they go through those different stages.

Don't be triggered by the word maturity. I know some people don't like it, but there are steps to get to the end goal. If you don't like the word maturity, feel free to replace it with the term that you're more comfortable with. In the next 15 minutes after that, I'm going to share state-of-the-industry data that we have from a self-assessment survey that we've been running for a while, and also from the data that Tasktop actually collects from over 3,000 value streams, so we can see how the data represents what the actual state of the industry is.

Let me tell you a bit about Tasktop, so you know why I am qualified to be on this stage giving you this talk. Tasktop was acquired by Planview, so I'm always confused whether I should say Tasktop or Planview. Since the middle of 2020, we have been supporting organizations with their shift from project to product, and we have a turnkey value stream management platform called Tasktop Viz. We explicitly measure the shift from project to product for the entire technology portfolio, and we also accompany that with coaching so we can ensure our customers are getting results from teams all the way up to executives. We've had first-hand evidence of how the journey is going, how people are navigating the transition, and we generate flow metrics and flow insights into those transformations, as well as benchmarks. Those have allowed us to describe the journey from project to product in five stages.

The five stages of project to product can be roughly described as starting out, experimentation, expansion, operationalizing, and approaching maturity. The survey results that I'm going to share with you will tell us where most organizations are, and that's why I want to set the stage so we have a framework to have this conversation.

When you're starting out, you're really starting out: you're in the project model. Everything is on an annual funding basis. You're funding projects, except for run work. Teams are constantly being formed, stormed, and disbanded. Lots of time is spent in project-related meetings. Application integrity is hard to maintain. Dependencies are invisible and hard to manage, and you have these large-batch-size releases that are highly governed. Your ticket out of stage one is understanding that there are negative consequences to being in this model and wanting to get into stage two, where you begin experimentation.

Typically we see a very highly visible portfolio chosen as the test case for experimentation, where you start to have more continuous funding, cross-functional teams, and a product manager assigned to the product. It is frequently coupled with adoption of other lean and agile frameworks.

After you've experimented for a while, you say: now we're ready to expand. That means giving more and more portfolios the same treatment. What we often see at this stage is that there is still a lot of focus on external-facing products, and not enough on the internal products and platforms that underpin those external-facing products. Because of that, there is still a problem with slow lead times.

When you get to operationalizing, you've worked those things out and can really start to reap the benefits of the shift. You have independent and rapid paths to production for your products. You emphasize achieving business outcomes and better flow. Finally, when you're approaching maturity, all the last pieces have fallen into place: more responsive and flexible funding models, well-defined interfaces between systems, patterns of self-service, independent releases and dark launches, and typically feedback within a quarter.

What actually has to change as you're moving through those stages are the seven differences between how you operate in a project model and how you operate in a product model. When Dr. Mik Kersten wrote Project to Product, he outlined the seven differences: budgeting, timeframes, definition of success, how risk is handled, how teams are organized, how work is prioritized, and how visibility is achieved. Because this is a multi-year process, it can become very hard to see what you have done and how you have progressed across the different dimensions. You can lose sight of where you are and lose momentum for the effort itself.

We've reframed those differences into seven dimensions that you can use to self-assess and progress evenly through the process: team organization and resourcing, customer centricity, definition of value, backlog management and prioritization, dependency management, feedback speed, and delivery team metrics.

For example, how clear is the definition of value? In a project model, the emphasis is on being on time and on budget, so most individuals would be very hard-pressed to tell you how they contribute to tangible business outcomes beyond just getting it done. When you begin the shift, some leaders are capable of articulating business drivers. As you get closer to a product model, most product managers can articulate the value they provide to their customers and to the business. At maturity, value is so well defined that it serves as a focal point for everyone on the team, even if they're on platform teams or internal products. Teams are focused on incremental delivery of value to achieve well-understood business outcomes.

It is worth noting that, based on the survey responses, organizations do not necessarily move evenly through these different progressions. A large financial institution might be improving on backlog management and prioritization, but still cannot revise plans rapidly based on customer feedback because it has only a few highly governed release windows per year. An insurance company might have its entire portfolio organized in continuously funded, cross-functional build-and-run teams and be very advanced in that dimension, but still have only a few leaders who can define value. It's kind of all over the place.

Now I'm done setting the stage, and I'm going to tell you what the data is showing us. We've been running a survey about how the shift is going. The assessment launched in May. We had around 260 respondents then, from 223 companies. The survey is still running, but we paused to do a quick data analysis. We saw many unique job titles: lean or agile people, engineering, architecture, product people. Clearly this is a self-selecting audience, because these are the people being put in charge of doing this transformation.

What we found is that 84% of organizations have not fully operationalized the shift. They are clustered in the starting out, experimenting, and expansion phases. Only 16% have gotten to operationalizing and maturity, where they are really reaping the benefits and capturing the ROI. The insight comes when we do deeper analysis and understand what increases the likelihood of crossing that chasm.

We found seven most important factors, grouped in terms of the dimensions we spoke about. Clearly, we need to get out of the project mode where all IT assets are funded by projects and run budgets. The further you go up that dimension, starting with getting out of annual funding and moving to semiannual funding of cross-functional teams for external and internal-facing products, the likelihood of crossing the chasm increases. The highest increase comes when the entire portfolio is organized in continuously funded cross-functional build-and-run teams.

The second-highest factor that increases the likelihood of operationalizing the shift is feedback speed. Organizations where all products have an automated and independent path to production and can incorporate customer feedback within weeks, not quarters or years, will succeed. The final category is delivery team metrics: organizations that go beyond cost, quality, Agile and DORA metrics and start to measure flow and business outcomes increase their likelihood of crossing the chasm. When they programmatically measure flow metrics and business outcomes as part of operational reviews at all levels, that's the best for them.

Now I want to show you how these three dimensions actually play out in the value stream data itself. We have 3,000-plus value streams under management, mostly from the Fortune 500, and it is helpful to know what good and bad look like so you can establish where you're at and get buy-in for change as you try to advance to the next stage.

Let's start with team organization and resourcing. For many organizations going through the project-to-product shift, the footprint of annual funding is all over their value stream data. Annual funding encourages upfront, large-batch creation of work: epics and stories. We can see in the value stream data that the ratio of stockpiled demand is so much higher than what is actually possible to deliver. Organizations are generating too much relative to what can be completed.

One example is a Fortune 100 financial services firm that has 90 epics in progress, but only completed six in the last two months. Below that is a Fortune 100 telecommunications firm where the business is grooming 80 stories a month and their velocity is only 30 stories. Why does this matter? People are setting business targets against wishful thinking about what can actually be accomplished. It sets the stage for miscommitments, disappointment, and frustration.

We know that people lose their jobs when they miss a commitment. Look at the Volkswagen CEO who got fired earlier this year for severe software development delays that set back scheduled launches of key new electric vehicles. People lose their jobs when they set expectations that cannot be met. At the same time, technologists are under so much pressure to deliver due to these unrealistic expectations and the avalanche of work bearing down on them that it causes burnout and low morale. This is a recipe for a culture of finger-pointing and despair, the exact opposite of what we in the DevOps Enterprise Forum want to achieve.

For our long-lived products, we have to get out of the annual funding cycle. It creates a false expectation with the business that they can buy all the capacity they want within a year. Capacity isn't created out of thin air, and it takes time to ramp up. That's why it is not uncommon that by the end of the year you've neither spent all your money nor met all your commitments.

The good news is that this information is very compelling to CFOs. They can understand this data, and they will make a change based on this data. We should produce it and put it in front of them so that we can get them to shift the funding model. We can also use this information to do better, more iterative planning, not only for feature work but also for the care and feeding of a product through defects, debt, and risk. We can find where value streams are constrained and fund the constraint to relieve them.

That brings me to feedback speed. We saw that the likelihood of operationalizing a product model increases when feedback speed is within weeks. Our data shows that organizations are running fast, but toward the bottleneck. Unlike Toyota, which inspired lean, we are not singularly focused on removing waste and relieving bottlenecks. As a result, we pile more and more work faster in front of the bottleneck. Dependencies are often at the root of these issues. They come up time and again as the bottleneck.

In Tasktop Viz, where we measure flow for cross-value-stream dependencies, we can see this clearly. Looking at 10 value streams with cross-value-stream dependencies between them, when it comes to delivering features, seven of them are okay. They are well within the median of 18 days. But three products are delivering at a significantly slower speed than the median. The longest is taking 120 days, four months, for a feature. This value stream is negating the speed gains achieved by all the other value streams. It would not surprise you to know that this is an internal product, a platform, or shared service that is perpetually under-resourced and under-invested in.

That matters because to leaders and business partners there appears to be no ROI to transformation budgets and initiatives, new Agile and DevOps practices, or continuous improvement budgets, because we are spending them in the wrong place. For lack of proving ROI, the next round of budgets is even harder to justify. If you can be armed with better data, you can avoid a blanket application of DevOps practices and instead be wiser about choosing which one to adopt, where and when.

Let's talk about flow metrics and business outcomes. Full disclosure: I work for Mik Kersten, who introduced the Flow Framework. He told everyone to measure flow metrics and business outcomes, and that man is responsible for my livelihood. But there is an abundance of data to support the importance of doing both things.

The third set of factors increasing the likelihood of operationalizing a product model is measuring flow metrics in addition to Agile, DORA, and quality metrics, and making them part of operational reviews alongside business results. When we begin measuring value streams, we find that before flow metrics, plans were made without considering flow time, flow velocity, flow load, and flow distribution.

We have benchmarked the industry below 25% predictability. That means there is less than a 25% chance that the work you have taken on will complete within your current measured flow time. Everything you are delivering is rush jobs and ambulances, and the rest of your delivery is all over the place, because work is committed and pushed into the pipe without validation that there is capacity to complete it. I am speaking about organizations that are Agile. Even after adopting Agile methodologies, we are still pushing too much work and not really respecting the pull mechanism.

Furthermore, we found that value streams are so overloaded that half of them have no capacity for new work in the next six months. That's the opposite of what we strive for in business agility: being able to iterate quickly, get feedback quickly, and continuously learn. At this point, the learnings will come at least at a six-month delay.

The first negative consequence is that, despite Agile celebrating its 21st birthday this year, we are not Agile. We cannot quickly pivot or respond to market change. The second is the staggering amount of waste in our value streams. Every time new work enters the value stream, it pushes in-progress work lower down the priority list, where it collects dust and goes stale. That diminishes the value we hoped to get out of it by getting it to market on time. We have benchmarked the waste at 40%. For every $10 we're investing in our value streams, $4 are going to waste. That is another great stat to get in front of your CFO, CIO, CTO, or CEO to get attention if you're a change agent trying to move your organization along.

We reviewed the factors that increase the likelihood of crossing the chasm. Now let's turn to what decreases the likelihood. A single most important factor emerged: customer centricity. Organizations that cannot universally identify their customers and have no programmatic feedback mechanisms in place are much less likely to operationalize the shift.

In our customer base, we start by helping customers identify their customers. For many, it is very hard to articulate. They know there is a fuzzy thing called the business, and they don't talk to them very often, let alone get feedback from them about the capabilities they deliver. Value streams start and end with a customer. This could not be more fundamental.

The negative consequence is the inability to prioritize high-value work. There's no framework to make the call. This is where we also see very poor application of cascading OKRs, specifically key results. People have objectives, but they do not have a metric they are trying to increase from X to Y or decrease from X to Y. They only have a list of things to do and deliverables. There is no agency and no ability to learn what is helping and what is not.

Our recommendation is not to let yourself off the hook with this one. Transformations fail if they are not related to a clear outcome. As leaders, you have to make sure that at a minimum you can track a single most important customer metric driving your work. For some it will be cost reduction, for others top-line related, and for others quality.

One more recommendation: bear in mind that even your project or product transformation is starting at a place where there is probably a high level of dissatisfaction, and any change effort will require engagement to succeed. You have to make it a point to measure a value stream's engagement frequently. That's not the same as a once-a-year company satisfaction survey. We're talking about taking the pulse with monthly engagement surveys for the people in this group. They can be light, but engagement is a great proxy for whether members feel like they are delivering value and getting relevant and timely feedback. As people managers, if we don't start caring more about the humans involved in a transformation, the transformations won't succeed.

Last little bit here. You may have noticed there are a lot of people still experimenting and they want to get into expansion. For this one, we haven't found one single factor that improves the situation. There is no silver bullet. You actually need to progress evenly across all the dimensions. You have to get yourself around a level two in all of those dimensions in order to build a strong foundation to move to the next step. Rushing is where you fail. You have to take the time to develop the good patterns and the learnings, and then you'll be armed with enough to get through to the next step. You'll have those playbooks to expand.

My takeaways are that the project-to-product shift takes many years, as evidenced by the fact I shared from 2018. Here we are four years later, and not many have operationalized the product model. But I think we're in a good place now, where as an industry we have tools and learnings and a road map that can help. Don't rush out of the learning stage and don't try to expand until you're ready. Learnings come faster when you can visualize the data and show it to your leaders, compelling them with hard numbers in terms of waste, capacity, predictability, costs, and engagement.

You need to have an engaged workforce, so make sure you are checking the happiness of your teams. The assessment is still open. We've already gotten around a hundred more responses since I did the data analysis, so you're welcome to take it for your own benefit and find out where you are. It might help you rally around and find how to take the next step. If you want to continue the conversation with me, of course I'd be happy to do so. We can all walk together to the lightning talks. Thank you so much.