Ori Keren

Hi, everybody. Welcome. My name is Ori. I'm the CEO of LinearB, and today we're going to cover impacting operational efficiency on a global scale.

I'm joined by Jason, Global Head of Delivery at Syngenta, and we were lucky to have them as one of our customers.

I don't want to bore you with a big agenda, but basically what we're going to talk about today is scaling, because Syngenta actually scaled from 150 to 400 developers in a short time. We're going to talk about data, because the topic is a metrics program and how you can successfully run a program with all the challenges. And Jason is going to do most of the talking because he experiences those challenges.

And the last point will be about impact. What do you get when you run a metrics program? You get a small nugget here of, for Syngenta specifically, we were able to decrease the cycle time for changes by 81%. But I don't want to steal too much of Jason's thunder, and I'm going to hand it over to him.

Jason Krohn

Yeah. Hi, I'm Jason. I work at Syngenta. Not many people know of us. We're a big agriculture company, about 50,000 global, to make crop protection and seed products for everything from commodity crops, flowers, vegetables, fruits, as well as insecticides, herbicides, biologicals, and things like that.

And more specifically, I work in the digital division of Syngenta, where our big focus is really on sustainable agriculture, regenerative agriculture, and how do we use technology to help our growers have more success with the products. How do we help them make the best use of their land, both now and going into the future?

I started about three years ago, coming in to run our engineering organization in North America. About a year and a half ago, I took over globally.

So we'll start with the results of what we've seen out of the metrics program that we've been running, and then we'll dive a bit into the how, effectively.

As we came into the start of this year, we started with a phased approach. And what we've seen in the last six months is, as already mentioned, an 81% decrease in cycle time, as well as a 33% increase in planning accuracy.

It's been really interesting as we scaled to see how these came into play. For the cycle time, it was so much about the things that the teams already wanted to do, but maybe didn't always have the awareness for things that they needed. A lot of our reductions came in things like PR pickup and review times. No team would say PRs should sit there for two and a half days. No teams would say PR review should take a day and a half.

As well as some procedural things for them. It sped adoption of things like greater adoption of our feature flagging platforms, to get around some of the business objections to, "Hey, we can't put to production three times a day. We need to train our users." It gave us the ability to phase rollouts and things like that, as well as implementation of things like the work review, which we'll talk about a little bit later for our teams.

Planning accuracy was also super interesting because this was so much procedural. As we got our teams to really look at this and they started to evaluate their ways of working, we want our engineering teams to be strong partners to the business and our product teams. And so often what we found is, when they came into planning, we went and looked at these things. They'll come in with requirements that just weren't complete, or all the due diligence wasn't quite done.

But because our engineering teams wanted to be good partners, "So we'll bring it in. We'll figure it out." The perception of this was, it's usually okay and sometimes causes problems. When we started to put data behind it, we discovered that, no, it actually often causes problems, sometimes okay.

And so it really caused our teams to increase some of their rigor on, "Is work ready?" and improve some of their ways of working, getting all of our people, engineering, products, business stakeholders, really showing up more prepared for things like PI planning or planning and things like that.

So why did we even start this?

When I joined about three years ago, we were about 150 and we scaled up to 400. And that's not big for some companies, but when you're 150, you're spread across three major global hubs. The things that work at that level, each of those regional leads, they really know all their people. They know all their teams. They know the risks, the challenges. They're able to be really proactive.

And you triple those team sizes, though, a lot of that really goes out the window. You no longer have the touch to all of those people, and you have to change a lot of what you're doing. What works at 150 doesn't work at 400. It certainly doesn't work beyond that.

And so as we went through this year, we really started evaluating what are the changes we need to make from an organizational level, from a procedural level, and from a tooling level to support our teams.

As we grew in this really global manner, time zones and communication introduced so many challenges and delays. There's language communication issues, there's cultural communication issues, there's the time zone itself. We either mash everything into three hours in the morning or wait a whole day overnight.

As our products, which went from small products that were regionally deployed through scope, started to be deployed in common markets across the world, the demands of them to work well with each other, as well as our platforms and connected platforms, increased. And the cost of those handoffs, where we chose to scale some of those things like QA in India or things like that, just increased.

And what it led to was, dollar for dollar, we were much slower at 400 than we were at 150. And so what we were really looking to do is, how can we make changes, get that efficiency back to where we were when we were smaller, but in a manner that also lets us continue to scale?

Ori Keren

So I'll jump here for a second and tell Jason he's not alone.

We released our engineering benchmarks report lately, and I just pulled two key insights that represent that growth or that size. Not necessarily relevant for Syngenta, but I want to share with the audience.

Number one is we know that remote work is not solved yet, but this is an interesting insight: when your teams are more or less in the same time zones, a lot of the delivery metrics are better. So I don't know if you can do something with it, but if you create more overlap for those teams, we can see a lot of merge frequency, pickup time on PRs, review time, all the review process is getting shorter. That's something to keep in mind. That's a very strong insight that we found.

And the second one, as you scale and as you grow from, I don't know, exactly that type of scale from 150 to 250 to 400 developers, one of the things that we've seen in our benchmark report is that, on average, the cycle time of an enterprise, which is 300 developers and more, is 20% more cycle time.

And usually where we pay the price in those big enterprises is, the code is ready, it's merged, and it sits and waits for deployment because the teams are more siloed. It doesn't mean that Syngenta is suffering from that problem. That's a pattern that we saw and we wanted to share with you.

Jason Krohn

So as we looked to tackle this, how do we get some of that efficiency back, and how do we scale? We wanted to start with what do we, at a leadership level, want to understand? What do we want to look at our teams and evaluate?

We came down to three things. Are we working efficiently? As we take our dollars, as we take our people, are we deploying them, and are they able to act in a way that is efficient?

Are they effective? Because just as we're doing work and it's filling time and we're producing things, it doesn't really matter if we're not producing value in the market. Are we having the impact that we need to have in the market?

And the last is, are we working safely? So I think like many other industries, we work with a lot of customer data, a lot of grower data. We're subject to a lot of regulation, privacy laws, things like that. So from security, from vulnerability, from compliance, from regulatory perspective, if we're efficient and effective, but we're not doing it safely, it doesn't matter.

So for us, it was important that anything that we roll out, any of the changes, really address all three of these dimensions together.

For us, we really believe in distributed teams, in a distributed model. We want our teams to be independent. We want our teams to be autonomous. And so we need to give them the scope to report all the right.

I don't want my regional leadership to have to know all the details of all teams under them, effectively. We want each of those individual levels to be able to own all three of those dimensions, good or bad, and report up to be able to do that at the site level, to be able to do that at the global level, so that leaders at each level can focus on the big picture. Where are we? How are we trending? What's working well? How can we help each other?

But to do this really requires us to give tooling down to the teams to allow them to find any of the bottlenecks or issues. It's one thing to just tell them, "Your cycle time is too long." But as already said, there's a lot of reasons that can be the case. It's not always just sitting on the engineering team not doing a thing or PRs not getting picked up. There's business concerns, planning concerns, requirements. So we have to give them the tools to be able to find where those things might be.

And so as we looked at the metrics, we wanted to keep two in mind.

The first is, whatever metric you choose is going to get gamified immediately. And we've all seen this velocity a million times. People padding points, people thinking, "Ooh, this is a soft two. Let's pull it into sprint so we can hit our things." Hitting the number they have to hit, not for the reason that it should be.

And the second is, your choice incentivizes. So if you make a choice, and I don't think anyone here would have lines of code being a metric we want to measure, it incentivizes for both things. It incentivizes picking up initiatives that produce a lot of code, not the ones that bring value.

So at Syngenta, we really wanted all of our metrics to focus on being agile, not doing agile. That's great if you're running Scrum, we're having planning, we're doing retros, we're doing standups. But if we're not getting value out of them, if we're not quick to the market, if we're not getting feedback, if we're not iterating, we're just doing agile.

And the second is supporting a sustainable, steady flow of features delivered. A flow that's healthy for our teams, that's healthy for our people, and that works for our business.

And so for that, this is the basket of metrics that worked for us. It may not be what worked for you. We're all here, so of course we started with the DORA metrics, and they're great, but they just paint part of that picture for us. And so we took your standard cycle time, change failure rate, time to recovery, and we augmented it with security and vulnerability ratings. So coming out of our static code analysis tools where we can put in the compliance checks, we can put in the things to know, are we operating safely?

Bringing in usage analytics, which for us is just from Amplitude, to help our products understand, are we having the impact in the market that we want to be having?

The other thing I like about these is, we're able to define for each of these metrics why we care. Why do we care about cycle time? We care because we want to be able to sit with the business. We have an idea. We want to be able to seize that idea and go from idea to production quickly, and get feedback and iterate. Each of these for us is defensible.

And so we came up with these at the leadership level, and then we took them to our teams to confirm. We justified why do we care about each of these metrics. What do you guys think?

The nice thing was they didn't disagree with these metrics. We had a lot of ideas of many, many other metrics we should bring in, which for now we're holding off on in our first iteration.

It was important for us that we walk the line between giving the metrics that support those dimensions I was talking about earlier, but not overwhelming our teams with 9 million different things and then, what do you do?

So we talked about the why. We talked about what we choose. How do you actually implement it? Because measuring something isn't going to change anything at all.

For us, the focus from an implementation standpoint is really on enabling and empowering our teams through information, through tools, through processes. And so for us, that main process is the operating rhythms. It's something we use at Syngenta for a number of things.

Our teams can embed this in their existing operating rhythms, whether they're running Scrum or Kanban, into their retros, into their PI plannings. Our regional teams can embed it into their weekly staff meetings. We're there looking at it, just a quick review of where we're at. Are we trending in the right ways? Biweekly, we review it at the global level. And then quarterly, we look at it from a strategy perspective.

And this has worked well for us. It's distributed and close up.

The bigger part is then how you talk about it. It's just a big part of this mindset. And I'll start with the second one here: ownership versus compliance. All you get from compliance from your teams, you won't win. It's going to get the numbers to where they need to get the numbers.

And the whole point of this is not to make the numbers be five days, effectively. The point is, is our teams owning, we want to be agile, we want a steady, sustained flow of delivery, we want to be able to partner with the business?

And so a big part of that goes into how you talk about these metrics. As soon as you introduce any metrics into an organization, everyone's initial reaction is, "You're going to use this to judge me, and you're going to use this to punish me." This is all they think, and everyone has seen this with velocity. In a lot of organizations right now, "My team made 13 points and they made 17 points. This guy's 20 points. This one's eight points." It's not even how any of this works, but that's what it always tends to.

So for us, how we talk about it, and I've talked a lot about enabling our teams and giving tools. At the global level, I don't look at any of our individual teams. I focus on the aggregate, I focus on the trends, I focus on what we're doing, how we share. This is really, really important to get your teams on board to own this and drive this.

You saw some of the success we have, and I don't want to go back, but I saw a lot of success early in how you get these teams on board, and they care, and they're able to go and find and make changes that they want to affect.

The next big one is metrics pose the "so what?" Just because a number is not where it should be doesn't mean it's necessarily a problem. It doesn't mean this is a problem, but it's something we should look at. Talking about it in that manner, I think, has been really critical for the success that we've had, along with this focus on moving the needle.

When we first rolled this out, we didn't create a list of, "Hey, all these teams are good and all these teams are bad." This was our baseline. Teams are where they're at. Our focus is really, you ask all of our teams: look at these metrics. What's the one that you want to focus on in the next quarter and move the needle? That's it. Report up your experiments, your successes, your failures, all of this. Then click to the next one and we'll talk about it.

And this is really important, the sharing part of it. We talk about it monthly in our teams, all of our internships, community calls. We do our current summary of state. Where are we? What's working? What are we focusing on?

In this first six months, we asked our teams to really focus on cycle time and planning before we start to tackle some of the other ones. But the bigger part is the sharing. Sharing experiments, successes, and failures.

At least to us, I've found historically we've been great about sharing experiments, we've been great about sharing, and everyone's hesitant to share failures. So the big change we've been making is sharing those failures with the same enthusiasm we share success with.

We talk a lot about psychological safety. If we're going to ask our teams, "Hey, take a change, make a change," they have to feel comfortable that that change might not work out. And that's not a bad thing. They're not going to be punished for that.

So these kind of distributed governance, how we talk about it, and this sharing, I think, has led to the success that we've had and the ownership that we've had with our teams.

Ori Keren

That's great. And I want everyone to have attention on these things, because what we've seen with our customers when metrics programs succeed, it's not just about picking the right metrics, and they definitely did a great job of picking the right metrics. And it's not just about getting the right talk, how we talk about it. It's also about the operational cadence.

If you just choose the right metrics and then want things to happen on their own, it won't happen. What Syngenta did very well is also put this operational cadence: when are we talking about those metrics? In what routines? In what meetings? It's a key factor for success.

And I'm very connected to, "Hey, this metric won't take you to the next level." I think the other element that's missing here, and we're going to talk about it now, is automation.

Now that we have everything covered, how are we talking about them? Choose the right metrics, get the right operational cadence. How do we embed automation into that? I would like Jason to talk about an example.

Jason Krohn

First, there are two things that our teams really liked. I don't mandate our team to use any of these things, because the mandate coming from the top, as an engineer for a long time, they're not going to care unless it actually works to them. They won't use it. It's ownership versus compliance again.

Two things we've used really well are the WorkerB. For us, the value here has been two things. WorkerB provides a little bit of a tickle, a notification to our teams of, "Hey, there's something that we need done." For us, it's around PRs and things like this. If someone put it in somewhere to look at it, but then also as the teams being set their thresholds, something is looking to breach that, it just reminds them.

Our teams have found this impactful, especially in this global remote environment that many of us work in now, a really valuable tool for themselves.

And the second one is we've implemented recently, gitStream. The big things we use it for are annotation. Often teams know there needs to be a PR, they don't know who needs to pick it up, how long is it going to take. People are busy doing things. It might take three minutes, it might take 30.

So this is a note of who's the right person to look at it, who's working in that code block, how long is it going to take, where is the risk there. Our teams have found a lot of value in that orchestration of their day to help them meet some of these goals.

Ori Keren

Yeah. So that's one of the projects we're really proud of and definitely can move the needle in an organization.

Jason Krohn

So the rest of the automation is, what do we do? What are the tools that we give to our teams?

For us, this is Power BI. We use it, but it allows us to use APIs to pull from all these different platforms to aggregate together in a way that our teams can see that whole picture. They can look at it at the right level: the team level, the regional level, the global level. And lets us see outliers more easily, letting everyone at each level focus appropriately.

Next slide.

That's our regular dashboard. We also have one around outliers, which we're starting to focus more on. You see here, "Ooh, cycle time: five," but we have a lot of outliers. It's just a tool for our leads to be able to look at, okay, on aggregate, we're doing great, but there are some things that maybe we want to look into.

There may be a great reason why one of those teams is 11. One of the things we run into on cycle time is mobile. You can't push for mobile all the time. But it's that outlier detection that we like bringing into the BI tool to help our teams.

And this last one, just another dashboard around security and performance, SonarCloud. Once again, just trying to equip our teams with the tools. We have all these things in platforms, but it's not easy to find. So Sonar, for example, it's all tied to repos. With the tool, we can take those repos to GitHub teams, map those products, and then give that visibility out to all of our teams: engineering, product, and business stakeholders.

Our end goal that we're still working towards, we have these in pieces but don't have it aggregated together yet, is a single dashboard for all of our teams that measures health across everything. It's looking at delivery, it's looking at FinOps, it's looking at performance, it's looking at user engagement, with then detailed drill-downs against each of those things.

Cycle time or planning isn't where it needs to be. You can drill down and see all those individual charts around point time, all this kind of stuff. Same with ops cloud spend, engineering spend against budget, how we're tracking against our investments on the product levels.

So that's where we're still going. I'm pretty happy where we've gotten in the first six months. The ownership from our teams is really high. I'm hopeful about where we're going to take it, but we'll continue to evolve it.

Ori Keren

Yeah, and just to wrap it up here. We're inviting everybody to start with us.

In the Syngenta case, we were able to improve the cycle 81%. On average, it's 47%. It's still very, very significant. To put the dollar value on it, it's an amazing contribution. We know how to give you 10.2x ROI in the first half year.

You can see some of our customers here, and we have a booth there. Everybody is welcome. We just released a free DORA version that everybody can experiment with.

Other than that, thank you very much for coming here and listening.