Dr. Mik Kersten

Just as a quick introduction to me, I spent the past two decades trying to figure out what's broken about the way that we try to build large-scale software.

The first 10 years of that was working as an open source developer. I wrote a ton of code, did a bunch of research at Xerox PARC, worked on the Eclipse IDE, did my PhD at the University of British Columbia to try to figure out how we can make our coding activities, really going from code to deployment to cloud, about 10 times more productive, and realizing that we can't. We're basically hitting diminishing returns at that point.

I then spent 10 years building a company, this is now Tasktop, and working with Global 500 organizations to try to figure out how to take some of the ideas that did work in open source, in terms of the very fast flow and feedback loops and learning that we have in those kinds of software projects, to large-scale enterprise IT. Through that, I got to learn just how big a disconnect and chasm there is.

As you may have heard Gene say this morning, I've tried to capture these ideas in this new book on IT Revolution Press, really inspired by what Gene has been doing with this community and with every one of you. The book's called Project to Product. It's how to survive and thrive in the age of digital disruption with this new thing I'm going to show you today, a preview of it actually, called the Flow Framework. The book's going to ship at DevOps Enterprise Summit in Vegas in the fall, in October.

The book is anchored on three epiphanies that I learned through this journey of the last two decades. The first one is that we've got this very large disconnect between what we're used to as, those of you who've done development, in the software architecture, and then the thing that shapes the software architecture, what we now call the value stream. These structures don't quite align. There's this bigger mismatch that I'll tell you a bit about.

Software value streams, in my career as a developer, getting connected to them properly, understanding what I should be working on, what the flow of work was, what the demand and the feedback from our users was, was really critical. We made some tools to do that better.

But as I got to working with large organizations, I got a sense for just how broad these value streams are and how many IT specialists. It's not just ops. It's also business analysts and designers and the support teams, and there's this whole world of ITSM out there as well. Each of these are underrepresented at the various conferences that you go to. We've stovepiped our conferences the same way that we have our tools and our organizations. So there's something fundamentally wrong there. Again, we needed some breakthroughs to see through that.

Finally, in trying to understand how this has worked in previous ages, and how we solve these problems with large-scale manufacturing, which I'll get into in a moment, I had this third epiphany that things are fundamentally different in software. While we try to make our delivery pipelines be linear, they're actually not. There's some other bigger problem here. These value streams are these more complex learning networks and dynamic systems. So I'll touch on each of those things.

But first I'll start by giving a bit of historical context because I think we all have a sense of how through basically our entire careers, the pace of change we've been seeing in technology has been tremendous. It seems every time we learn something at a conference, we go back to organizations, something changes. One of the big tech giants has a new platform, a new automation technology, a new toolkit, and so on. We've had this tremendous pace of change that's very difficult as technologists and as business leaders to keep up with.

It feels like it's been this way forever because it basically has been this way our entire career. But if we look at what Gene introduced today, he actually introduced this work to me about a year ago, the work of Carlota Perez. If you zoom out of our career spans, which are 20, 30 years or so, 40 years, she has this model which is based on the model of Kondratiev waves, these long waves of economic progress, of technological revolutions. If you zoom out, it actually looks like we're in this very specific point of this particular technological revolution, in this age of software and of digital.

Each of these waves has had this installation period where the new technology, be that electricity, be that railways or steam or something else, basically starts maturing, and the new organizations start learning how to leverage it. You end up with the organizations who learn how to leverage it, for example, the railroad companies and the robber barons, amassing more and more wealth as they figure out how to leverage this new means of production and often leaving a big chunk of the economy behind. Then we shift into this new period, in these golden ages and so on.

It looks like, and I've had some conversations with Carlota Perez since that time, it looks like we're right now in this turning point, in the point where we're moving from this installation period to this deployment period. That would actually explain the sort of change that we're seeing, how much disruption there is, and how quickly the technology platforms are shifting.

It looks a little bit like this, as Gene showed you, where you've got these two or three decades of installation period. What's really fascinating about these installation periods is during that time there's a lot of financial capital. This is where financial capital can make its 10X returns.

If you look at what's happening to a lot of the companies represented in this room, this is just a visualization by CB Insights. If you look at it after this talk, or during this talk if you like, if it's boring, you can Google for "unbundling of" and CB Insights, and you'll actually see these really cool graphics of the unbundling of a bank.

This is an illustration of a bank's website, and you see there are about a couple thousand fintech startups out there today, and they're going after every single part of the financial experience. There's all this venture capital funding the disruption of a bank and every single aspect of the banking experience. This is the kind of thing that happens in this installation period. In Detroit, there used to be around 300 car startups.

But then we shift into this, at the turn of the century, we shift into this deployment period. In the deployment period, some companies have figured out how to master the new means of production, and how to master it not just from a technology point of view, but from a whole-company point of view.

We're seeing this. You've heard Gene refer to the FAANGs today: Facebook, Apple, Amazon, Netflix, and Google. They have mastered this new means of production. Within their organizations, they have these connected value stream networks. Companies like Microsoft have 3,500 developers working on their internal tools, the tools that they use to build their software.

So these are, whether you think of them as the amazing innovators providing us some products that we're now completely dependent on, whether you think of them as the new robber barons, these are the companies that have mastered software at scale.

What's interesting is these companies are getting better at moving into other businesses than traditional businesses that grew up in the last age are getting good at building software. This has some potentially dire consequences, because what it means is that a lot of those companies might not make it through that turning point, might not last through that deployment period.

We're actually seeing already, because these things are always a little bit fuzzy, we're seeing signs of even unicorn-like startups not being able to make it. Here's an example of a company I really like, Jawbone. I used almost all of their devices. They had $930 million of venture capital, and they failed completely. You cannot compete against the production capital of Apple, even in this period anymore. That production capital's gotten that powerful.

It'd be very difficult right now to compete against the production capital of Facebook if you're building a social network. They will buy you, or they can replace you by reimplementing your feature set.

So I think Gene's call to action to us was that we can make it through here, that we can take these practices. But I think if we keep going at it the way we have been, most of the companies who grew up in that last age will not make this transition through the turning point. In the book, I go through some of these case studies, again, to figure out how we can avoid this and how we can end up with a world that's got a much broader economy and distribution of wealth than if five or six companies control the majority of it.

I got to witness some of these failures firsthand because some of them happened early, some happened late. Nokia was, I think, one of the fascinating ones. For those of you who were around the Agile conference around 2009, 2008, those years, they were a poster child for transformation. All of the Agile consultants were using Nokia, saying, "Yes, Agile works at scale. Yes, you can do this. You can do what Nokia did."

Then I actually had a chance to work with Nokia. They were one of our first customers. The way that they were measuring how agile they were was by how many people. They had this thing called the Nokia Test, actually developed by Nokia Siemens Networks, was how many people were trained on Scrum.

So now think back. I hope all of you saw John Smart's talk this morning, and think back to that longer value stream of his. So not only were they the "yay, we're agile" tiny segment over here that they were focused on at a business level. The only way they were even testing how agile they were was if someone was trained on Scrum.

In the meantime, their bottleneck had nothing to do with how many people were trained on Scrum. They had a platform, this Series 60 operating system, which had so much technical debt in it that even if someone handed them a capacitive touchscreen and said, "Okay, make this thing work. It's now going to be a bigger screen that's easier to touch and a bit heavier," they had no chance to innovate on software that quickly.

This was a company that was over 100 years old who got to making some beautiful, and I had that phone, beautiful and amazing devices in terms of mass manufacturing, but they never, at a business level, made the transition to the age of software. The people I knew at Siemens at the time, they actually were great technologists and great software people. But Nokia as a company did not understand what technical debt was at a leadership level.

So how could you replatform the way Amazon replatformed, spent probably $1 billion replatforming, when you don't actually understand those dynamics of software and how software innovation and value streams work?

It's similar for a lot of other organizations. I've had a chance to witness a top 25 bank make those same Nokia-like mistakes over and over again. Three failed transformations, each of which looks like it's on track because they're tracking the activities of the transformation, how many people are using this tool, or how many people are trained on Scrum, rather than actually looking at those outcome-based metrics.

There's just something fundamentally wrong with this approach, and we see these common problems in our companies. IT seems disconnected from the business. Leadership is tracking activities, not results. Project funding, there's something wrong with it because projects end, they haven't delivered success. IT seems to be solving its own problems, not the business problems.

Then to the business leaders, IT feels like a black hole, and to the technologists, the business feels like it's always buffeting them in this or that direction and never giving them a chance to actually replatform properly, implement the frameworks that need to be successful, and compete with the startups and the disruptors.

So I wanted to take a look back at what happened at this other age over here, kind of where we ended up in the age of mass production, because things worked there. These companies made it through that turning point.

I had a chance to visit the BMW Leipzig plant, which I think is one of the most advanced mass production facilities on the planet. This is what it looks like. From walking into this building in Leipzig, it struck me as just a very different philosophy and mindset to how production and business are connected. Literally from when you walk into the building, because once you walk in, so I took this photo. There's my colleague, René, who invited me. You see the assembly line above the office desks moving there.

There's this intimate, both in terms of the metaphors of the plant, in terms of the fact that the CIO and all the IT staff wear the same vests as the production workers on the line. You see that there's this intimate relationship between production and the business.

I've read all the lean stuff and the Toyota Production System. I'd read this book, but I started rereading some of these things I'd read with this view of what I learned on that plant. If you look at these lean principles, for example, from Lean Thinking, these are the lean principles: that value is specified by product, and that you have to identify the value stream for each product, make value flow without interruptions, let the customer pull value from the producer, and pursue perfection.

Those are what the plant was and everything it embodied. For example, the customer pulling value: the cars are delivered off the line in the same sequence that they were ordered by customers. It's not just-in-time inventories, it's just-in-sequence delivery.

The entire architecture of the plant embodies these lean principles. If we look at a bird's-eye view, and you can do this, just Google for BMW Leipzig on maps, if we look at the architecture of the plant, the whole thing matches the business needs of BMW. This is that cool building of the assembly line.

Over here, you've got this five-finger structure. This is the assembly line for the 1 and 2 Series, and it winds its way like this. The reason it's got that shape is because BMW does not yet know what new innovations might be added to manufacturing steps. So it will actually extend those buildings to add new manufacturing steps. The architecture of the buildings matches BMW's business needs there.

In contrast, this really interesting white building, this is the i Series building, where BMW's actually been able to scale electric car production. That building can actually reconfigure its own lines in order to scale up electric car production, which we know from recent news stories from Tesla, how difficult that can be. I think two weeks ago, BMW announced that they're going to be delivering another 120 cars a day out of the electric part of this plant.

The entire architecture of the buildings and everything that you see there is all geared towards meeting the business need, and it's architected around the value streams, not vice versa.

If you look at how business value flows in BMW, these quality cars that deliver sheer driving pleasure are designed in yearly cycles, but they're delivered every 70 seconds off that plant.

Interestingly though, the creative and the manufacturing process are decoupled. The design of the cars, and I had a chance to look at that as well, the design of the cars is separate. So there is definitely something different, because I was trying to make all the metaphors line up, and they don't quite all line up. There's definitely something different happening with the way that business value flows in IT.

We're not delivering the same thing over and over. We're not delivering shrink-wrapped boxes packaged in Ireland anymore of CDs. We're now delivering new features. Our users are not pulling releases because the releases are, if you've got continuous delivery, basically available. There's no longer a bottleneck that's getting the releases out for companies that have made it through that. It's the value that customers get, and that value tends to be things like features, and they tend to be designed much more quickly.

The creative process and the manufacturing process are actually the same thing once you've adopted the principles of DevOps.

So what can we do there? How can we adopt what BMW learned about this product and value stream orientation to the way that we look at our own software delivery, and how do we create those manufacturing lines?

The only thing is, as I kept doing this, our VP of products there, Nicole Bryan, I said, "Nicole, how do we make ourselves be more linear?" You realize, okay, well, what we're doing within our organizations, if we could actually take a look at how information flows in our value streams, just imagine taking this moving MRI of every single IT worker in your company and how they talk to every single business person, you would not end up with a linear manufacturing process.

You would end up with something that looks much more like a network of different teams, of different stakeholders, of communications and conversations and artifacts flowing across that network. That network would not be a network of planes or cities, but it'd be a network of the ground truth of where we work, which is not these assembly robots and rails on the line, but it's the toolchains that we use to implement the business processes and the different workflows in building software.

What we did is we decided that we wanted to understand this a little bit better. How does that ground truth, just like I had a chance to, through this Gemba walk at BMW, witness the ground truth of the Leipzig plant, what does the ground truth look like across the Global 500 IT customer base?

We realized that through our work with our customers, we actually had 308 different toolchains documented across a good chunk of the Global 500. We did a whole bunch of analysis and research on what was in those toolchains, how are they connected, where are those artifact flows?

We've got the DORA report, which has some super interesting statistics, but it's very hard to find cross-industry information on that actual ground truth, on what's happening in the tools and the real work happening, not just what people think is happening, not just what they perceive is happening.

So what we did is we noticed that, for example, 70% of people connect three different tools. Forty percent of you connect four different tools. We've got all these integration patterns that are connecting different things in interesting ways, and the artifacts that flow get very interesting.

The really interesting thing is that what we noticed is that these flows that you've got within your organizations, they're not linear flows with one team not talking to another team. They're these networks. These networks have this ad hoc structure, and the biggest problem is that structure is actually aligned to projects, tends to be aligned to what the PMO set up or project management set up as this project structure, which comes from the way that IT work is funded.

But if you look at what's actually flowing, there seems to be this really massive mismatch because if we look at what's flowing, if we look at the question the business people actually want answers to, the answer's not, "Is this project on track?" Because the answer's always yes till the end, when it's not. That's why I think that whole view, we now often hear the term watermelons, where the projects are always green on the outside, but red on the inside, and you don't find out till you cut them open.

But there's some very important questions that business stakeholders always have, like: how long will it take to get this new feature, this new application to the customer? We need to move fast. We've got this fintech startup disrupting us. How much wait time do we have for these high-severity incidents? What's our mean time to repair? Or if we actually do go ahead and implement these fixes for GDPR to deal with those risks, how will that affect our future delivery?

These kinds of questions are almost impossible to answer for this kind of project management structure. So we need to look at something different.

Again, think about what's happening at BMW and think about what's happening within the project-oriented structure. You've got this black box with a mapping and status updates on whether a project's on track, versus what you saw at that plant, a direct mapping to the business. We know exactly how many cars are going off the line, how quickly they're going off the line, what quality problems there are because of how many of them are going into the rework area.

We've got this waterfall orientation of the project structure, where we assume there are these discrete phases, whereas with a proper product-oriented process, it's just about flow. How much flow, how much value can you deliver to the market?

We've got these funding cycles, again, that are more based on budgeting cycles than based on value delivery, where if you learn that you can deliver more value halfway through a cycle because you've experimented some or you've succeeded with an MVP, you can now inject more budget into that.

Over here, of course, is one of the biggest problems that no mass manufacturing business would ever implement, which is that you can't reallocate people to different projects because they need time to build up expertise whenever they're building something complex. If BMW were switching people between the electric line and the gas lines, that would not work that well whenever they restructured projects.

The other key thing is how you manage risk. In projects, you try to bake in all that risk up front, whether it's a market fit risk, it's a delivery risk, all of that. You bake in up front, you bring in contingencies, and you don't find out until the end of the project. Versus what we saw, for example, BMW starting out with the i3 Series. It was probably less clear how many of those there would be demand for until Norway said they didn't want to go all electric or all zero emissions by 2025. At which point, it could make sense to inject more because the risk was lower and the opportunity was higher.

So the very big realization I had through this, when, again, working with a lot of IT leaders across the Global 500, is if we look at, you don't need to read all this, but if we look a little deeper into those different ages, what's happening is that each of the ages had managerial innovations that helped them thrive. They had financial capital innovations and so on.

If we look at the age of electricity, that's where Taylorism came from. That's where we learned about separate division of labor and functional specialization. That's the age of electricity.

In the age of mass production, we learned about Fordism, and we learned how important it was actually to invest in people's expertise and in stable product lines and so on. The big problem is that most IT organizations are managed two ages back. They're using managerial systems two ages ago. We haven't even shifted into the age of mass production.

There's this massive gap here, and this is the gap that we need to fill. What we've done is tried to collect everything that we know about this and create this new managerial approach to help organizations shift from project to product, and do basically what the tech giants, what the startups are doing in terms of aligning around product-oriented and value stream-oriented software delivery.

I'll give you a preview of this. This is a draft. The book is still in draft. There'll be the final version of this thing called the Flow Framework to help you do this, to create that common language between the business side and the IT side, in November at DevOps Enterprise Summit in Vegas. But here's the preview.

This is the Flow Framework. The idea is that you align all of your delivery, and I'll have to go through this fairly quickly, but you align all of your delivery around these product-oriented value streams. For each of those value streams, you track both the flow metrics, so the equivalent of the cars, and you track the business results of those metrics so you can correlate the two. You can say, "Okay, if I pump out more i3s, do I get more revenue results? Are they being sold?"

The flow metrics proposed by the Flow Framework, and I'll go through each of those quickly, are flow velocity, flow efficiency, flow time, and flow load. These are very much inspired and based on the Kanban community, the Agile community, and what's happening in the DevOps community.

The business results are value, cost, quality, and happiness, and I'll go into those in a moment as well. But the key thing is the Flow Framework proposes you only measure four different units.

These are the four things that capture business value, which means each one of the work items, for example, we use an evolution of the Scaled Agile Framework internally at Tasktop and Scrum processes. We have dozens of types of work items. Each work item maps into one of these four flow units, and they are features, so net new value visible to an end user. That's what customers really want to pull.

Defects, so customers pull those as well because they want things to work, not not work. Risks, so this is all of the compliance security work that you need to do. Then debts, this is all technical debt, infrastructure debt, value stream debt potentially, if you're missing automation and all of that.

Every single type of work being done by a technical team, by a product and engineering team, fits into each one of those.

Then for each of those, for example, we'll track flow velocity and see how that contributes to value. For example, if I get a higher velocity, if I deliver more features, does that get me more value? The value can be revenue, it can be monthly active users, it can be whatever your value metric is internally that matters to your strategy, matters to your OKRs, and so on.

Cost, but the key thing is that value is tracked per value stream. So you're thinking more like BMW than you're thinking more like an enterprise IT shop. You track value for every product, and the activities for that product's value stream are what delivers that. Same thing with cost. You track cost per value stream. So you need to know how many, let's say, full-time equivalents you have on the value stream, how many people in product, how many people doing design, analysis, testing, are doing it for that value stream.

You track quality per value stream. That might be escaped defects or however you like to track quality. And you track happiness for the value stream, which means, and this is something I actually did learn from John Smart. We weren't doing this till I learned that John implemented this at Barclays. We have been tracking eNPS, employee net promoter score, for ages. We now track it for our value stream.

So we know, for example, if one team is sitting on a platform that's so difficult to evolve, yet the business side is saying, "Why can't you get more features out the door?" there might be something wrong because they want to be doing the right thing, but they've not been given the chance to replatform, to rearchitect, to adopt that new framework.

So you track those four things accordingly to the business outcomes, and I'll show you a little bit of each now.

The first and most important thing about each of those is their distribution, how much allocation you have to each of these flow units, because these are mutually exclusive and comprehensively exhaustive, which means that if you do more feature work, you can expect that that won't take away from risk work.

So a CEO of the business cannot say, "Okay, we're now doing GDPR and investing everything in security," and then not expect feature work to decline. This is part of the core things, is that both the business side and the technology side need to understand at least these four concepts. Maybe our CFOs then and CEOs won't understand what technical debt and story points are, but at least hopefully we can get them to speaking in this language.

Here's an example of flow distribution for a team where you, for example, see exactly what happened with a feature flow after a 1.0 release of a product, where, of course, there were more defects and risks that had to get worked on after the thing was released and before it was released, so feature flow declined.

This is the kind of thing that product engineering managers already know how to do. This is their job. The idea is that this is what we need to get the business to understand. If you're going to have an initial release, well, you need to plan for a net new feature decline. Or if you're going to do a whole bunch of risk work this quarter, you need to make those trade-offs as well, so you don't end up in this Equifax CEO situation where you think all the security work is being done while you're pushing your teams to do all this feature work.

To do that accurately, we also need a better notion of time, and I think we've had a bunch of confusion. This is where I've learned a bunch from Dominica DeGrandis' book, Making Work Visible, on how we need to have a better understanding of time for software because this is a little bit different than manufacturing.

The Flow Framework has this metric of flow time, and it's really meant to move us away from this notion that time or lead time is code commit to code deploy. That's a cycle time of development. Flow time is really when we accept work into the value stream to when it's finished.

So if you want to solve a bottleneck, we don't want to have a team going, "Yay, we're agile." We want to celebrate when we're bringing business value to a customer faster than we were before. That's what flow time is all about.

Lead time is still very important because lead time is from customer request. But if you're Apple, your lead time is basically infinity because you ignore every single support ticket that you get. This is pretty well documented. They get so many issues reported. Same with open source projects. Their lead time is basically infinity because you get so many more customer requests than you can handle. What really matters is when you accept it as delivering value, and that's flow time.

Flow efficiency is critical, and this is a way that we can actually look at how things are actively being worked on versus how often they're waiting. That's another key part of the Flow Framework.

With that, we can create these flow dashboards. Here's just a mock-up flow dashboard where we actually see the flow distribution over time so that we can use that to plan to figure out how to invest in this value stream. We see flow velocity, and we also see flow load. So we can track WIP, work in progress, on these value streams and see when we overload them, same effect as with manufacturing, our velocity actually drops. The idea is that, again, we measure all of this per value stream. That's really the goal of the Flow Framework.

To do that, what we need to do is basically rise above the level of that toolchain, create a model of the value stream through this integration model, and then map that model into our product-oriented value streams. I don't have time to go through that today, but it's in the book.

I should mention you can get a copy of the book at our booth. I'll be signing them at the next break.

But the thing that supports all of this, the same way that the manufacturing line in the plant, which the BMW CEO understands the bottleneck of that manufacturing line, as does every single person in the plant, the equivalent of that for us in IT is this value stream network. We have to have a connected value stream network, and we have to have it aligned around products, not projects.

So again, to get rid of these functional silos and move from this world of projects cross-cutting things to these horizontal value streams. With that, this is what we've been doing at my company, at Tasktop. We've got this amazing level of visibility. These are our internal dashboards. I've obfuscated the revenue and cost numbers because these are the real dashboards.

But you can see, for example, here that we've got different rates of flow across different teams, and we can see, for example, that risks are such big things that we're taking on that they're actually taking us much more time in terms of flow time than other things are as well. And that this team over here has a more mature set of frameworks for higher feature delivery, while this team is still building out their frameworks and so on.

With that, we can answer all these questions we were looking at: how quickly we get things to the customer, MTTR, all of that is automatic and real-time and visible.

So with that, the help I'm looking for is if you're interested in this stuff, take a look at the galley copy, that draft copy of the book. Sign up for, I'll be publishing every couple of weeks or so, a blog post about a part of the book. And then, yeah, please feel free to reach out, and that's my email address if you have any thoughts or ideas on this.

Thank you.