Taleen Armen

Hi everyone. Thanks for joining us this afternoon.

My name's Taleen Armen. I'm head of IT operations at QBE. I look after a number of teams: release management, the QA function, IT risk, and a whole bunch of others. And I've been with QBE for over 20 years.

Len Daniels

And my name's Len Daniels. I lead our release management team, as well as the Agile Release Train engineers. And then I'm the practice leader for our DevSecOps.

Taleen Armen

So a bit about QBE. QBE Insurance is an Australian-based insurance company. We're an international insurer with a local presence in about 27 countries. QBE offers commercial, crop, and specialty products and risk management solutions. We've been around for quite a while, since 1886, and we have about 11,000 people who work at QBE globally.

When I look at North America's transformation, transformation has been incremental over the last several years. Many moons ago we created a CSI program, a continual service improvement program, that Len actually runs for us. And the idea was that we wanted to get as much feedback as we could from everyone to understand what's working, what's not, and where we can make changes.

So over the years it's been quite successful, and we've had quite a few changes come through in the development space, QA, PMO, supplier management, architecture, all areas of technology.

In 2019, QBE decided that we were going to stand up a global IT modernization program. And that was really around IT modernization and transformation.

Sorry. That's one. Sorry, guys. We've got a... Yeah.

During 2019, we decided to have a global IT modernization program. We had lots of technical debt, we had disparate tools being used, and we needed a better way to serve our customers.

There we go.

So if we zoom in on that global program, the global program was initiated with shared opportunity statements and goals across all of the geographies. Our ambitions were threefold. That was to modernize the IT estate, increase the speed to market, because that was something that our business was telling us globally, and also to reduce costs, which is a problem statement I think everyone has.

One thing to note: our development and testing was, and continues to be, outsourced. So the program really meant that we had to partner with our vendors to make the changes that we needed from a people, process, tools, and importantly, from a culture perspective.

We picked four key areas to focus on globally. The first one was around agile maturity, and that was really to initiate a shift in mindset towards planning, delivery, methods, and practice. So we were focusing on the agile process, the metrics, which is something that we'll touch on a bit later, and also risk when it comes to agility.

The second one was around transparency, and that was finding a way to make sure that we had end-to-end visibility of what our trains were costing us to run, including the business benefits that the business were seeing. We're looking at future demand pipeline and also the resourcing that was needed to make us successful.

The third one was around modern engineering. So that was around creating a cross-functional framework where we were combining developers, operations, and importantly, security into the mix.

And lastly, we looked at the IT procurement space in terms of how we manage our vendors and also providing self-service capabilities to the divisions when it came to working with procurement.

We're going to touch on two of these.

So what you see here today is what we look like today. This is not where we want to get to. It's where we wanted to get to right now, but we've got some more plans to go forward.

So this is a culmination of three years' worth of work, rework, negotiations, discussions, and, to be quite frank, some tears as well to get to where we are now. We tried to take a very pragmatic approach because it was a global program, and we wanted to make sure that each of the divisions were going to be able to meet the needs of the business areas within that division.

A few things to note that you don't see here. One of the things we did was we created a train launch pattern. So we took a business area in the first instance, created a train launch pattern. So we looked at the training needed, the roles, tool composition and configuration, business stakeholder management. And we took that train through that train launch pattern.

At the end of it, we did a review: what worked well, what didn't work well, adjusted that launch pattern, and then went into the second and the third. So it really was about the learn-and-adapt cycle.

The other big component was training. We didn't want to train too early. We didn't want to train too late. So we're trying to be a bit thoughtful about when we would do the training for the trains and squad leads and participants. We also had coaches available not only for our teams in the divisions and globally, but also for our offshore partners as well, because it was a partnership with them.

Third thing to point out, and I can't stress this enough, is around communication and around organizational change management. So it was a global program. We had global comms and we had global change management, but it was also very important that we looked internally and thought about what works best for us from a division perspective, because we're all different. So having those components as part of your program was really, really important.

And the last thing, which is an obvious thing, but sometimes we need to point out the obvious, is leadership support. This was a hot topic throughout our program. It was about three years long. We're still not finished. We're almost at the end, hopefully end of November. But we had leaders come and go.

I think authentic and visible leadership was super important for our success. And that's the feedback that we still get today when we have conversations with the different teams.

There we go.

Len Daniels

Thank you, Taleen.

Just to share a story with you before I jump into the metrics slide here, how many of you, is this your first time you've attended the DevOps conference?

About half the group. Taleen and I have been to three conferences now in person. And I remember that first conference that we attended, we were kind of like deer in the headlights, right? I mean, everybody seemed to be so far ahead of us and had all these fancy processes in place, and the tools were all mature.

And then last year when we were here, we had already started this IT modernization program, as Taleen was mentioning, and I whispered in your ear, "We're going to present at next year's conference." And so, yeah, we are. We're here, we're presenting.

And if you have kind of that awe of how other companies are doing it, I'd say stay persistent. I think I heard that message this morning. And make sure you continue to sell a compelling case on why you feel the value for moving in this direction. There's a lot of work, but someday you may be up here presenting like us. So keep the pressure on, and hopefully you'll have strong results like we do.

So agile metrics. I like to call this slide making our metrics simple. We could have measured a lot of different things, obviously, on the agile front, and decided on purpose to slow things down a little bit, pick a core couple of metrics that we wanted to focus on, and really use something called Jack and Jill to sell that to our product owners, our agile teams, our Scrum teams.

And so, obviously, it's a simple nursery rhyme, and it only has four key themes to it: plan well, ensure quality, maturity, and then outcomes. And so we picked five measures to go along with that: velocity, predictability, our defect escape rate, agile maturity, and then outcomes.

And you can see on the right-hand side from that, what we did is built a Power BI report that fed off of our Jira daily. So it actually gets a feed into it daily from Jira, refreshes every day.

This first page is really an overview page that highlights all of our trains and our squads. And you can see on the top left-hand side the open sprint. So right now we have, this is actually data as of yesterday, nine open sprints right now. You can see the closed sprints. As you move further to the right, you can see the story points that have been completed per sprint, and then the number that are sitting in your backlog, how many active sprints we have in our backlog.

Far right is the number of features or epics that we pushed into production every quarter. And then as you get on the bottom, we get more into how long did it take, basically from the day of creation to the day of delivery, for stories to be completed, and then our features to be completed.

Far right side, something we're pretty proud of. One of the metrics that we did put a threshold on or a goal to is our predictability, which is really the committed versus the actual completed story points. We really wanted to focus, at least first with our teams, on trying to reach that goal.

So we ended up with a 50% target. We wanted to stay maintainable. We wanted to make sure we sent the theme to our groups that this should be iterative. And you can see we're blowing that goal away right now. We started in 2022 measuring this. It was roughly around 40%, and we're up near 80% right now for the last PI. So really good results around this measurement overall.

There's other tabs on here that get at the quality side for defect rates. The Scrum Masters and Release Train Engineers use this information as they're going through the retros to come up with improvement ideas for the innovation and planning sprints to try to improve themselves. So very valuable information.

But what does this mean from the business side of the house? We use Scaled Agile. So we use the SAFe program, and you can see in the middle here is a benchmark that they use to measure companies' agile progress. And there are really four metrics up here.

Faster time to market. There's a quote from one of our product owners that is on the renters site, that we actually implemented a new renters partner in record time, 50% faster than what we've done in the past under our waterfall methodology.

As we move to the bottom, you see defect resolution. We're right about at that 50% with the pre-agile, prior agile. That's defects for bugs to the tickets that we deliver. That's how we measure that.

On the left-hand side, increased productivity. Most of our teams are beating that 35% rate. You can see the Squad A, B, C here, the results from before they went to agile till after.

And then on the top is our employee satisfaction. We just did our first survey, so we don't have a benchmark yet on that one, but I'd say we're probably pretty close to that measure as well.

Okay. Switching gears, talk a little bit about DevSecOps and share the journey with you. Before I do that, I want to share a little cartoon or a little humor with you.

The first one's a new developer saying, "The tester's not at his desk, so you can deploy. Hurry up." Second one is, "For a moment, I had a feeling of total security, then it went away." Some more on the security side.

This launches what our definition of DevSecOps is. We pulled this together as our IT mod launched in terms of helping people understand what we wanted to deliver from a DevSecOps perspective. It's really a culture framework. You probably hear that a lot when you hear a definition of what this is.

We use the culture foundation from DevOps and then really embedded the security on top of that feeling and sent a message to our delivery teams that it's really part of everybody's job. So it's something they should take seriously. And it's core to what we've developed from a culture perspective around DevSecOps.

We also have built this into a cross-functional team for our developers, our testers, our security resources, and our operations team. They're all working together.

And you can see underneath here, there's some benefits that we've laid out. I'm not going to go into each one of them, but we really wanted to reduce the complexity on our tooling. On the technical side, wanted to have faster resolution to problems. On the culture side, really become self-reliant or self-autonomous, not have to rely on going to a centralized operations team anymore when you have issues with your pipeline. And then on the business side, obviously delivering the features faster to the business.

Taleen Armen

I think one of the things that we've done really well as a division is engaging security as part of our agile transformation from the beginning. So making sure that the architecture team and the security team aren't an afterthought. We wanted to bring them in because it's paramount that they're involved from the beginning all the way to delivery.

Len Daniels

Yeah, it was really critical. Actually, I'm going to hit on that right here. Good segue.

So we had a goal with the IT mod group to actually reduce our time for builds and deployments by 50%. And I'm happy to report that we were really close in that goal. Most of our teams are actually measuring better than that.

On the right-hand side, it really just summarizes what Taleen just said. One of the things we worked really closely with is our security groups, and not only automating the builds, but embedding security tools into our pipeline process. So vulnerabilities are caught and improved, or you can't pass the pipeline. Obviously, it fails the pipeline.

So anything that's a high, critical, or medium will fail the pipeline and have to be fixed before the developers can actually push that code to the next environment or eventually to production.

We also have SonarQube, which we use from a code quality perspective, built in. So not only do we have automated testing going on, but we also have SonarQube and the testing tools built in as well.

Here's a little picture of that technology landscape. This really was a people, process, technology change. And I know everybody's kind of interested always in the technology and the infrastructure around it.

We actually moved from our as-is state. There you can see the technologies that we were using at the time, to our to-be state. We moved a lot of our tools to Azure, brought in our security tools, as I mentioned below, Fortify, WhiteSource, and also use Tosca for automated testing.

We did stick with SonarQube for code quality scanning, but outside of that, we've changed almost all of our tooling.

We're still working on pulling in some of our heritage tooling of these old as-is states into our new DevOps. We started with 12 core applications that we built out the pipelines for and the projects around, and then are pulling the others in as we speak. So that project's going on just today. Most of it'll finish up by the end of the year. The rest of it'll sneak into early next year.

And then on the right-hand side, it's a little process flow from the beginning, when you plan and release using our Jira tool, to the deployments and eventually the scans that go on during those deployments, all of which are automated, as I mentioned. And if there is a high, medium, or critical vulnerability, the pipeline will fail and they'll have to fix it before they can go on.

Here are the four patterns that we developed. I hit on a little of this earlier, but from the very, very basic, your build-and-deploy pattern, which is just automating your builds and deploys, then adding in automated smoke testing and regression testing. Eventually code quality security testing is our third pattern. And lastly is our unit testing.

So these are the four patterns that we built. We actually put each of our applications into one of the patterns prior to our implementations. And then as we implement, those are the patterns that go in place in our pipelines.

We often get asked this question: so how do DevSecOps and agile work together?

I do like to share that it is really two sides of the same coin. Agile's all about getting requirements to delivery and doing it quicker. And DevSecOps is really about getting the builds automated and more security built into it, so obviously there's no vulnerabilities.

In the intersection in the middle you can see reducing IT bottlenecks, reliability and quality of the software delivery, and then overall time to production. So pretty critical to have both these enablers in place to make your IT operations run smoothly.

And then lastly, I just have a slide here that I want to share around our governance. Taleen mentioned earlier that we're a global company, so that brings complications, obviously. And we have three core regions: North America, Europe, and PAC, or Australia.

We put a global DevSecOps Center for Enablement, or C4E we call it, in place, which really sets the overall strategy, works in improvement opportunities, and builds out the roadmap of where we're going in the future. So leaders from each of those groups sit on that C4E, and then underneath that sits our community of practices. That's where we get the engineers together to share the messaging and do best practices.

And then on the bottom, you can see a global DES platform team that takes care of the tooling. They also sit on our C4E, so they're very engaged with us from that perspective as well.

With that, I'm going to turn it back over to Taleen. Thank you.

Taleen Armen

That was a lot to chunk into three years' worth of work.

So what's coming up next? This year we've been busy doing a couple of maturity assessment plans. At a squad level, we've had the teams doing a self-assessment every quarter. We've just done the second round, and it's been interesting to see not only the feedback, but the actions that the teams have taken. And we've seen some improvements in the metrics and the numbers, which is good to see.

The other thing that we've done is a division-wide assessment plan. And I think Len mentioned earlier that we were looking at SAFe. So we did a review last year. We just did it this year. So in the next couple of weeks, we'll get to see what's the difference in terms of our self-assessment when it comes to our overall maturity.

We've got more trains coming in, and again, we'll be using that updated launch pattern that we talked about earlier on.

We've got a lot of business engagement that we're working on, because without the business' participation and having them onboarded, this whole program is going to be at risk.

And Len and the team have been working diligently on that DevSecOps governance bit.

Next year is going to be a big one. We've got more trains, and we're going to try and tackle lean portfolio management.

So as part of our initial modernization journey within the division, we decided to start a bit smaller. So we deliberately excluded projects from our scope. Next year, the intent is that we bring it in. But again, without the business' collaboration and drive with technology, we won't be able to be as successful as we would want to. So we really need to get our business engaged and in those conversations.

It's a big cultural shift. And anyone that's had success in tackling that issue, we would love to talk to you.

Len Daniels

Yes.

Taleen Armen

We're going to continue working on the T-shaped engineering. So a lot of that cross-pollination, cross-skilling, cross-training in the different areas. Looking at adding some more applications into the DevSecOps pipelines, and also looking at some of the service management process optimization.

I know there was another session now talking about agility and change management. So one of the areas we're looking at is how do we take our IT processes and make them work with where we want to head to in terms of our new ways of working?

Two more important slides. Something that I look forward to when I'm sitting out there is best practices. What worked, and what were some of the challenges?

So this was a globally funded program, but the divisions were accountable for our own destiny. So we had a say, we drove the program, and made it work for us.

The agile pilot was a big success. That launch pattern, trying to get an understanding of what works and what doesn't on a small scale, and then evolving and adapting as we went through, was a big success.

I think I mentioned earlier that our initial scope was limited to the small change. So again, starting small and then building on to the successes that you've had.

A big key for us was roles. We had a number of people who took on different roles as part of the new ways of working. So it was working with HR, making sure that people understood what their new role was. Are they sufficiently trained? Do they feel comfortable? And making sure that they had access to continued coaching and training as we moved along.

The Agile and DevSecOps COE was a big game changer for us, because it allowed us to learn not just from the other divisions. We worked with our vendor partners as well to draw on their experience and get a better understanding of what we should be doing and what we should be looking out for.

The coaches were key. So we had dedicated lean and agile coaches within the divisions and also globally. Again, just so that we can draw on their experience and make sure that we have the right information at hand.

And we mentioned earlier the security and architecture teams being part of our journey. Buying into what we were trying to do was key for us, I think.

Anything else to add there?

Len Daniels

No.

Taleen Armen

Challenges. We had a few.

Being a global organization, it was challenging. I think anyone who's worked in a global organization can attest to that. But add to that the fact that we were working with outsourced development and testing partners made the program a bit more complex. You can navigate through it, it just means that you need to be a bit more strategic and deliberate in some of the decisions that you make.

Business involvement is absolutely critical. We're not going to be successful if they're not engaged and understand what the benefits are for them.

That project-to-project transition is a challenge. It's something that we haven't necessarily tackled yet, but some of our other divisions have. So starting small and then moving to something that might be a bit more challenging.

Embedding, automating the security testing and code quality was critical for us, for our success. But it takes time. I mean, we're still working through it now.

Len Daniels

And money.

Taleen Armen

Yeah, and money. Yes. We forgot. Yeah, we need to add that in there too. And money.

And then, as with all big transformation programs, we've transitioned from this massive transformation program to BAU. What does that look like? How do we adjust to the new roles and new processes?

I think for us, it's a matter of trying to be patient. You're not going to see big wins necessarily from day one. So you just have to go with the flow and learn.

QBE is hiring. So if anyone found anything interesting, feel free to go onto our website and check out the roles that are currently posted.

Thanks everyone.

Len Daniels

Thank you.