Host Intro (Gene Kim)

Thank you, Suja and Nigel.

Okay, the next speaker is someone who should be very familiar to most of us in this community. Dr. Mik Kersten wrote the amazing book Project to Product four years ago, which is being used by so many organizations to change how they think about their software development efforts.

In this community, we all know how important architecture is to enabling fast flow. Over the last two years, I've been exploring with Dr. Mik Kersten and my mentor, Dr. Steven Spear, on the lesser-known corollary, which is that the architecture and the structure of the organization must be isomorphic.

In other words, we can create an organization that is structured so that teams can work independently, where local problems stay local, and amplifies important signals of performance, fully supported by the software architecture. Or we can design an organizational structure that prevents teams from getting done what needs to get done and dampens or extinguishes entirely the signals that leaders need to know that the system is working, regardless of how good the architecture is.

I am so excited that Dr. Kersten will be presenting a real-life case study of an organization that he did at his company, Tasktop, the benefits that he expected, what actually happened, and what he did about it.

I think this is an important talk because he models so explicitly and so well how we need to think through organizational design. It's something I've rarely seen, but I think it is so important to getting the outcomes that we want. Here's Mik.

Dr. Mik Kersten

Hello, everyone. My name is Mik Kersten. I'm delighted to be joining you virtually from Vancouver.

Since founding Tasktop 15 years ago, I spent a lot of time trying to better understand the intersection of organizational design and software architecture. I personally come from a software background, and I've been helping the organization grow and understanding how these different kinds of modules, which are teams and people and things actually with feelings and aspirations, can actually be structured in a more effective way for the growth and happiness and productivity of every organization.

That's actually what led me to my work on studying flow: how we can better make teams deliver more value to customers, and the things that impede that. Over the past few years, I've been working with enterprises making the shift from project to product. What I've noticed is that many organizations have nailed organizational design at the agile team level. But at the team-of-teams level, above that level, things seem to be a bit of a mess.

The different approaches that are being taken, the fact that people would copy-paste the Spotify model and not even recognize the fact that Spotify has moved on from the Spotify model, I've noticed causing all sorts of problems. And the fact that we're not actually measuring these designs is a problem in itself, measuring the outcomes of organizational designs.

Personally, on my own journey, I have found the work of the DevOps Enterprise community as critical to the toolkit that I use for understanding how to configure, evolve, and measure the effectiveness of organizational designs. In this talk, I'll share with you how I've used it and some profound steps and actually some really big missteps I've taken along the way.

This talk is about how flow follows form, and it really plays on this building architecture concept of form following function, where buildings actually look like their intended use. There are some very neat buildings in this presentation's backgrounds. They're all architecture by Zaha Hadid, who's actually the person who designed the BMW Leipzig plant, which was featured in Project to Product.

The point being that just as physical spaces have an impact on our creativity and our well-being, organizational designs have an even bigger impact on our well-being, in our day-to-day work and our day-to-day life. The key lesson I've learned is that the flow that we get from an organization, the happiness of the staff organization, is really a function of the structure of that organization. And so the question is, how can we improve this? And how can we design organizations that are better for their staff and for their customers?

The thing that we've seen as organizations adopt this shift from project to product is that it is being used as a catalyst for change, I think for some very important change. For example, on the architecture front, the fact that you start thinking about software architecture as decoupling value streams to accelerate flow to the customer and to minimize dependencies, it's actually a pretty profound change in how we look at architecture.

The fact that bottleneck detection that comes from studying and understanding flow will actually mean that you automate everything away and you start making more and more of a business case for your DevOps transformation, for more automation, actually has a pretty profound impact on an organization.

The fact that budgeting, changing to become more iterative and more capacity-based, and not something where we're creating these projects and can honestly be throwing work over the fence, is another significant change. As is reporting: the fact that with the change in budgeting, we actually have better visibility into our flow. Is our transformation, is our move to cloud, actually producing the customer outcomes that we want? Is it producing the acceleration that we want to see in our flow metrics, or is it not, and do we need to adjust and pivot?

And finally, organizational design. How do we, if we are now going to bring business and technology together as we need to do in any project-to-product transformation, how do we actually design the organizations? What are the roles? What changes with what we've got today?

At the core of this shift is actually designing, understanding, and measuring our value streams: how we deliver value to the customer, be that an internal customer, an external customer, or a business partner. These value streams need to be aligned. They need to be aligned to customer value. They need to follow that ideal of customer centricity. They need to be autonomous so that they're decoupled and can move as fast for the needs of that particular customer, and they need to be cross-functional. We need to bring together business, design, operations, and all of the different stakeholders needed to deliver value to the customer.

However, that all sounds nice in theory. The challenge for the starting point of all organizations, it's all different if you're building a new organization today, but we have to implement this on top of our existing organizational structures and designs. Often those have evolved around a very specific set of functions, and they have often evolved from a project orientation, not a product orientation. There are functional hierarchies. There are geographies to consider. In large and complex organizations, we have multiple sourcing partners. We're actually dealing with multiple organizations.

What we get from this is, it's being increasingly called this messy matrix: the fact that we want to have these very horizontal, these very flow-oriented value streams, but we have to implement them on top of our existing organizational structure.

The challenge is that these organizational designs tend to be all over the place. Models are changing, the way that hierarchies work are changing. But oftentimes organizations go through these large reorganizations, again bring in something like the Spotify model, basically copy and paste that into the organization, and then actually measuring: did we improve things? Did we make work better for people? Did we remove burden, or did we just introduce a whole other new set of bottlenecks?

One thing that I've found really fascinating is that technology companies just nail this. The way that they continue to evolve their organization design, the way Amazon has moved way beyond what started as two-pizza teams, is fascinating and has become very effective. So as a leader myself, I want to understand, well, what are the first principles I should apply to organizational design? What should I have in this toolkit? How should I work with the teams, with the other leaders in my organization, to again make a better structure and better dynamics for the company?

This is where I really have been drawing on the most recent work of Gene Kim and Steven Spear, where the concepts they introduced are these of structure and dynamics, and how leaders actually get to set and configure system-level goals around organizational design. Of course, with this architecture mindset, we get to configure and then we get to run. We get to execute that new organizational design, that new model, and then assess how it's doing. So the job of leadership becomes designing this organization and then actually assessing the organization, understanding with the design that we created, did we actually get the outcome that we wanted for our customers, for our staff, for our shareholders?

Now, the toolkit, what's interesting is I'm finding is actually evolving quite well. We now have these much richer pattern languages. This is, I think, to me, one of the most important contributions of Team Topologies: that it gives our organizations a language to talk about team structures. Of course, we need a way of measuring those structures. The goal of Team Topologies is to better design for flow. And there we've got the Flow Framework.

So I'll show you how, in the way that we've evolved our organizational designs, I've actually used both those concepts and then measuring and assessing through measuring flow. But the key thing I want to get across is what's underneath all this is, again, these two principles. These, to me, are these first principles around organizational design of structure, again, what the hierarchies, what those matrix lines are, and then the dynamics. Once we've got that structure, which signals got amplified? Which signals are being suppressed? What's working? Where are we actually succeeding with decision-making being lower in the organization, and where is it actually flowing too high? Or should it actually flow higher than we thought it should?

So with this eye on designing and on assessing, I think it's actually very easy to leverage some of these new things in our toolkit, some of these pattern languages. We've got Team Topologies here. There's work in SAFe and Scrum and Spotify and single-threaded ownership. They all start with S for some reason. And then we've got the Flow Framework. We can actually measure flow and understand, are we improving flow? Are we making it easier to contribute the value to the customer? And how does that translate to outcomes through these business results of value, of cost of the value stream, the quality and the happiness of the staff working on that value stream?

What's so interesting is when some great designs on the left here actually don't translate into what we want on the right. A couple of years ago, I was working with an organization who decided that they really needed to shift to a more automated testing and create some platform teams for automated testing, and as part of that, they ended up actually reallocating over 100 manual testers. But they did that without making their core software components, their business applications, more testable, and so it was a complete disaster in terms of the actual quality metrics that they got as a result.

So they seemed to do all the right things from an organization design point of view, but because they were able to see that their flow was reduced, their quality was reduced, they were able to quickly pivot and realize, okay, we actually do need some manual testing until we make our software more testable. So the whole goal here is that organizations are complex dynamic systems. Flow is complex, and we need to be able to measure that and actually have that inform our organizational design.

Here we're actually at Zaha Hadid's designed BMW Leipzig plant, and this is the story I'm going to tell you of how I actually applied some of these principles in this toolkit and some of the things I learned along the way. So this story starts about a decade ago. Tasktop had always been a product-oriented organization, but really started out with just engineers, so just open source engineers. Ten years ago, we added product managers, and we created a product management function. Of course, that was a really important improvement for us in terms of our design, in terms of getting closer to customers and making that a first-class function.

Then four years ago, we realized, okay, we're actually getting a little bit too siloed ourselves, and of course, we're always preaching not being siloed. So we realized, why do we really have a separate product management and engineering organizations in the HR hierarchy? And had this hypothesis that merging product and engineering, or the two organizations, the two hierarchies, will actually accelerate our flow.

The desired outcomes we were after, of course, is even more customer focus, not just for the product managers, but for the entire product development organization. Less us versus them of saying, "You didn't understand the architecture. You didn't understand the performance constraints," but everyone's in the same boat. Decision-making is low in the organization, does not need to go as high because both the product and engineering functions are together, and actually elevating our platforms to be products. So each platform has its own product manager, its own roadmap, and all of that. Those were all of the goals.

We actually documented this in a talk at DevOps Enterprise Summit in London 2019. The rework happened in 2018, so it's actually available in the IT Revolution video library, and you can go back and check the sorts of things we were thinking back then, versus the sort of assessment that you'll see me provide right now.

So fast-forward two years, and here's what we saw in terms of that assessment. You saw the configuration of merging those two. Here's how we assessed it. First of all, in terms of these business outcomes, the value metrics, things are going great. The new product that we launched, Tasktop Viz, it had 200% year-on-year growth over the span of those two years. So from a business perspective, everything looks hunky-dory.

From a cost perspective, this was as planned, that we would continue to grow our R&D. So we actually end up doubling each year the size of the product and engineering organizations together, the product development organization, and the cost on the headcount. The flow velocity, of course, we're growing quickly, so we understand that, well, it takes time to onboard these people. And we kept being hopeful that flow velocity should start accelerating, in terms of our basic flow velocity per FTE. But we realized that it was going in fits and spurts. We did some re-platform things in the process to help. But it really was trending to flat.

So we were not seeing the same kind of output from the investment that we were making, and we're trying to understand, well, we are scaling. It looks like we're scaling effectively. Why are we not getting that? And then there's this other really important signal, again from one of the main metrics in the Flow Framework, is the happiness, which we measure quarterly through an employee net promoter score. And what we were seeing is that it was also never in a straight line, but it was actually trending down over time.

Together, each one of these things in themselves, part of the metrics look good. The business and the finance metrics looked amazing. But we were seeing some really significant problems when we measured the system. The fact that the flow velocity was effectively flat and happiness was declining. Something seemed wrong.

Then we realized we actually needed to go into this diagnosis phase to understand what was going on. I had one of the most profound moments of my entire history at Tasktop over the last 15 years, which is a session with all staff. It was an ask-me-anything session. And Stefan Pingle, one of our most senior engineers, in front of everyone, which is a great thing that he does, he actually said, "Mik, you're such a fan of Team Topologies, so here's my question. If you're such a fan of this book, why do we keep overloading our platform teams with multiple complexity domains?"

I think as a lot of us know, as soon as you put more than one complexity domain on a platform team, things slow down. They don't speed up. You're overloading. You're putting cognitive overload on your teams. So why have we created a pattern, he was asking, of doing this over and over and over again, rather than actually properly architecting our platform teams and how much work they can take on?

I realized there was something really wrong because those signals were never making it up. Those teams were overloaded at this level. Then we kept diagnosing further, and what was happening is the architecture, both between the platforms and some of our customer-facing application, the coupling was growing. There were these initiatives I noticed of more and more meetings that were cross-value stream and identifying cross-value stream opportunities for collaboration and dependencies, these kinds of things. But those meetings were just growing in their number and their importance.

Then culture signals had actually changed. We had code jams. This is where developers at the end of an iteration just get to code whatever they think would be most interesting or most novel. And they had actually disappeared. Then our community events, in terms of our engineering communities, had also disappeared.

To me, these findings were obviously bothersome. I was wondering how the heck did this happen on my watch. My background is all development and engineering. But they were symptomatic of something: the fact that the configuration of the organization design suppressed certain signals and amplified others. It amplified all the customer signals, but actually suppressed some of the signals of what our platform teams were feeling and the cognitive overload that they were feeling. In support of customer outcomes, we actually deprioritized things like code jams and the kinds of things that actually help you get better and better over time.

So this single org hierarchy structure was just so compelling and so seductive. We thought, okay, this will be simpler. It's more modular. It should have fewer meetings. But what we saw in terms of its assessment is it was actually causing these signals that were really important, some of which should go to the very top of the organization, these very high organization, like when we should actually re-architect or when we should invest in another set of platform teams, weren't making it up.

The key lesson from this was by assessing it, we realized this was wrong for us at this point in time. This is the key thing: not so much where we ended up, but the fact that we were able to assess that and understand where we were going. This is actually similar to what you might see if you've been studying how Spotify has evolved beyond the Spotify model. They had a similar challenge. There was no one from engineering who was sufficiently responsible to prioritize that kind of engineering work, where, of course, all the customer work was always taking precedent.

And so in terms of where we are today in this organizational design, we're already seeing this working, the elevating of engineering, the fact that this is now training, mentoring, recruiting are a first-class function of the company. We've seen tremendous results already, such as in the last three weeks, hiring three new 10Xers, as well as elevating the career trajectories of our engineers today, and re-evaluating how we actually bring engineers along those career paths and making those career paths even more first class. As well as, of course, doing those things for other parts of our organization.

We also even adapted our model when we initially did put this. My thinking was why don't we put support engineers right into the value streams? We realized that's not the right thing for us. That's not going to accelerate our flow given the way that we provide our support to our customers today. But for another organization, that might be the right thing. So again, the whole goal is do not copy and paste this. Just use this toolkit. Use these concepts of structure and dynamics, and actually creating this feedback and assessment loop for creating the right organization design for your organization and then evolving it from there.

Do keep in mind, the other lesson for me that was really important is that this product focus and this engineering focus is a pendulum, and you actually want to set that pendulum to be specific for a value stream. For more platform value streams, you probably need more engineering focus. For more external customer-facing products, for faster-moving consumer products, you need much more product focus. So just be very deliberate about setting in the organizational design the structure and the right kind of leadership for each of those roles.

With that, in conclusion, I think the key thing that we need to keep in mind as leaders, as team members, is to design organizations and be aware of these pattern languages, and actually everyone in the organization should be aware of this. Everyone in a platform team should know when their cognitive overload gets too high, which is why these concepts are so important.

To assess this and to have a way of measuring it, which is really what the Flow Framework's meant for: did we make things easier for our teams, or did we actually slow things down and introduce more bottlenecks and more burden? And to continue to improve this, to make sure you've got those signals happening on at least a monthly or quarterly basis, so you can actually create this path for at least a quarterly improvement of the structure of your organization by measuring those dynamics.

To learn more, I've actually been so interested in this topic, my last three podcasts have all focused on it, with Jean-Michel Lemieux, Manuel Pais, and Geordie Henderson. So you can check that out on projecttoproduct.org. The key books here, of course, Team Topologies, but also Working Backwards and Ask Your Developer, I found to be tremendously helpful in terms of helping me on this journey. And then in terms of additional help I'm looking for, it'd be great to have others share their organizational design and what's worked, what didn't. Did you use two in the box, three in the box? What approach did you take? And you can do that on the Flow Framework community at flowframework.org.

And with that, thank you very much.