Sacha Labourey

Welcome, everybody, to DevOps Enterprise Summit. I'm Sacha Labourey, CloudBees Chief Strategy Officer, and today we're going to be talking about this challenge that organizations frequently have. On one hand, they want to go fast, but on the other, they want to go safe and have peace of mind. How do you handle those two challenges?

To talk about this today, I have with me Ben Angell, Head of DevOps Engineering. Hello, Ben.

Ben Angell

Hi, Sacha.

Sacha Labourey

And I have Sanmat Jhanjhari, DevOps Lead. Hello, Sanmat.

Sanmat Jhanjhari

Hello, Sacha.

Sacha Labourey

Both Ben and Sanmat work for Nationwide Building Society. If you live outside of the UK, you might or might not have heard of Nationwide, but if you live in the UK, there is absolutely no question that you have heard about Nationwide. It's a real historical institution. Ben, can you tell us more about Nationwide?

Ben Angell

Certainly. Thanks, Sacha. I'm sure people have heard of Nationwide Building Society if you're based in the UK, but just as a quick recap, it's a financial services organization. It's a mutual organization, which means we don't have shareholders. Our members, our customers, are essentially the people who are the shareholders of the organization. We sell a variety of financial services products: mortgages, credit cards, loans, etc. We have about 15 million customers or members up and down the country. We employ about 18,000 people based right across the UK, and we have some tech hubs in London and in the southwest of England, in Swindon.

The most exciting thing about Nationwide at the moment is we've just started to look at the future of work, in terms of how we employ people, where our teams are located, and how we enable those employees to work as effectively as we possibly can. That's really good for me as an engineering manager, as I'm building my function, building the engineering function, because it enables us to reach out across the UK and select the best engineers we can possibly find and not be constrained by geographical limitations.

Sacha Labourey

That's great. As part of Nationwide, tell us more about your responsibilities there, Ben.

Ben Angell

I lead the CI/CD platform engineering capability team. To take it back a bit, we've got a very big engineering function, as you can imagine, and we're always looking at how we can be more efficient and embed deeper controls into the way we work. We are very reflective of how we operate, and we've long realized that we really can adopt DevOps methodologies and exploit CI/CD to enable us to give better service back to our customers, our members.

I'm tasked with building the CI/CD function, the platform team, the engineering platform team, which is going to enable us to build the right CI/CD solutions, to enable our broader business area and our engineers who are based across our business areas to deliver really good, effective pipelines, safe and controlled pipelines. Lots of work. Very, very busy. Really interesting stuff actually, and quite exciting as well, because we can already see some real benefits of working in this way.

Sacha Labourey

Thanks, Ben. What about you, Sanmat?

Sanmat Jhanjhari

I'm a hands-on DevOps practitioner and leading the team of DevOps engineers. The responsibility is about designing, building, and optimizing the overall CI/CD tooling that we serve for the enterprise customers, and that enables us to put our customers in a position where they can build their pipelines with greater confidence.

Sacha Labourey

As I was saying in the introduction, today what we want to talk about is really this challenge that most, and when I say most I actually mean all, organizations are going through at one point. You have this initial push that all organizations have to increase their velocity, increase the pace at which they get software. They want to go faster. That's great. It creates a lot of energy, it's a positive change, and they start typically creating a lot more value.

But as things start to pick up, there is this realization that this needs to be managed. There needs to be an overall framework that makes sure that going fast does not happen at the expense of going safe. That is anything from secure, compliant, and so on. There are lots of constraints on businesses, especially in the financial industry. How do you manage those two worlds, and is that even possible? Can you get both velocity and peace of mind? That's what we're going to be covering today.

Ben, you're the go-fast person in that story. When did this journey to go fast start at Nationwide?

Ben Angell

It was probably mid-2018 or early 2018, where we had a number of really critical services that we knew underwent high levels of change, high cadence of change, and were heavily regulated, like all financial service products are. We started to look at pathfinder activity to understand how we could manage the CI elements of CI/CD perhaps more efficiently.

That was really successful. It showed us very early what could be done, what could be achieved in terms of automating our pipelines, and what we could use in terms of tooling and new guardrails, processes, and controls to embed really good compliance, continuous compliance, right the way through the CI journey. That enabled us to show back to our broader engineering community and our wider business areas what could be achieved by adopting DevOps methodologies, being more agile, and using proper CI/CD solutions.

That was a pathfinder, really to give us that credibility and to show what could be achieved. We've gone on from there and started to build out lots of these solutions as enterprise, so they can be scaled and become available to the broader engineering community, and start to see those benefits happen across the wider organization. That's where it started, early 2018 until now. We've gone through the pathfinder proving phase. We now know this is the right thing to do for our organization, and we're in scale-out mode as we speak.

Sacha Labourey

We hear quite a bit this notion of pathfinder that makes it possible to discover, out of what we all know about software delivery and DevOps as best practices, how that applies specifically to your organization. Every organization is slightly different, so that's very powerful.

You were saying a few minutes ago that Nationwide is perceived as this old and venerable institution. At first sight, if you're a developer, that might not be where intuitively you would think, I'm going to have fun with software, I'm going to use modern tools and practices. Yet that's exactly what you're doing, and you described how you initiated that deep transformation. How do you work to change that perception, for example in terms of recruitment, to make sure you find the person that wants to join that revolution within Nationwide?

Ben Angell

It's a good question. We are a very old and venerable organization, and we have a very low-risk appetite, as all financial services organizations do, and we're not going to compromise on that. Certainly the work we're doing is about embedding further control in the way that we operate, but enabling us to go faster and automate a lot of the processes that we have.

There is a perception, perhaps incorrectly, that if you have that low-risk appetite, it means you don't want to be innovative or look at new tooling, new solutions, or new opportunities. That's not correct. We are looking at some really interesting tooling solutions now. We have a really exciting cloud strategy. There is lots of ambition around adopting cloud services, around SaaS, around on-prem cloud exploitation, and the tooling that goes with that as well.

How I'm addressing that is talking at events like this and showing that while Nationwide is not going to compromise on the standards we set ourselves, the controls we know we have in place, and the level of risk we're prepared to take on, we are really keen to understand better some of the technologies that are out there, the solutions, the new ways of working, the processes, and how they can better enable us to be even more effective. It is an exciting place to be right now. There's lots happening, lots of opportunity. We are very reflective as an organization. We set a very high bar for ourselves, and we know that we continually have to evolve and change to be as successful as we've always been.

Sacha Labourey

In that first phase, where you do that pathfinder work, there is an entire environment that gets built, practices that get defined, and everything is focused on going fast. My experience working with customers is that typically the first year or first two years are the best. It's a honeymoon. There is huge excitement across all teams. There is a huge push in favor of change, and good results come out.

But as with any good thing, at some point the bill comes on the table and you have other stakeholders coming up, such as finance, and they ask very basic and fundamental financial questions. We spent quite a bit of money on this transformation, so what was the return on investment, the ROI? Does it really make sense? Did it reduce cost? Did it increase sales? They're looking at it from a PM standpoint in a basic and logical way: show me the money. Did you see that, and how did you go through that?

Ben Angell

We're working through that now. You're absolutely right. The first two years were incredibly busy. It was all about proving and showing what could be achieved. But as agile and DevOps understanding and appetite has grown, the awareness has grown and the challenge, the questioning around the true business benefit, has also grown.

A lot of my time now is talking to people perhaps from outside engineering, not talking about the engineering ins and outs of DevOps or CI/CD toolchains, but the actual business benefit. I can't understate this enough: one of the most important things you can do when you're building your DevOps strategy, and even right at the start when you're pathfindering, is to start to look at how you measure efficiencies. The before and after view. How efficient is your organization really? Do you have some really good metrics, some really good key point indicators of where your efficiencies or inefficiencies lie? How can you address that through adopting DevOps CI/CD ways of working?

The reason that's so critical is that you will be challenged, as every good organization should be challenging. What is the business benefit of doing this? This is a very heavy investment we're having to make. This is a big change to the way we're used to working. What is the business benefit? Why would the business want to sponsor this or support this? You have to have those measures in place before and after. This is how efficient we currently are; this is how efficient we are now. There's a delta here, and that delta shows us this is an investment worth making. If you don't have that, it's very difficult. You may go through pathfinder and then find you can't go further because your business doesn't understand what benefit can be handed back.

Sacha Labourey

One point you made that I thought was really good, pretty obvious when you think about it, is this notion that sometimes when we go through those transformations, we build a way to measure as part of this. Before, everything is harder and messy, so it's hard to define boundaries where you can measure things. However, you have to be really ready to measure that improvement as part of the transformation, otherwise you're blind. That's really good feedback.

What's the next step? You started with this pathfinder, and now you are in this expanding phase where you're trying to increase and scale up that initiative. How is that going?

Ben Angell

It's going well, but there's lots and lots to do. We're scaling our capability. We're scaling the number of people in our teams, the number of services we're looking to support, and the number of engineers we're trying to help. We're building out the platform engineering capability to support that demand. The broader organization has seen what can be achieved, and the appetite has grown, so the demand on the platform team has grown as well.

We're also looking at how we can continue to automate and how we can continue to embed further controls. The work isn't just about a single pipeline or tool. It's about further DevOps and CI/CD activities, making them available at enterprise scale and making sure they are adopted in a way that produces business value.

Part of that is people. Because of what we're doing, and because we're able to talk about this opportunity, we are attracting people from outside the organization. I'm a fan of being more agile and of DevOps ways of working. I've been around the block a few times, to be honest. I'm used to a way of working, but I can see massive improvement and massive benefits to be had. I beat the drum a lot internally in the organization about how this can improve our organization, and I expect engineers to come in with similar passion and belief that this is the right thing to be doing.

Sacha Labourey

I've rarely heard organizations say they need less smart people and developers, so I'm half surprised by your answer. It's impressive to see how, in just a few years, you've transformed what is a pretty traditional IT department and shaken it up in a way that it can do things faster, create excitement, and demonstrate business value to the business.

Because all of the best stories always have a but at some point, let's go through the other side of the coin. Going fast is one thing, but this cannot come at the expense of peace of mind. Under peace of mind, I cover things such as security, safety, compliance, and so on. We know the financial sector is extremely regulated, and for good reason. Sanmat, this is your field. Can you tell us when you started to realize that this new approach following the digital transformation meant that you would have to evolve the processes as they relate to safety, security, and compliance?

Sanmat Jhanjhari

Absolutely. I think that's really important. It's quite easy to deploy fast, but keeping it safe and secure, specifically in a regulated environment, is quite challenging. As we are a centralized team from a CI/CD capabilities perspective, I would like to answer this in two parts.

One is delivering the CI/CD capabilities themselves, which are for the development teams, because development teams are building their pipelines, delivering the code from source code to production, and consuming those capabilities. It is quite important that the fundamental tooling is safe and secure. The way we approach it is that we deploy those toolsets in a release fashion, where we deploy as alpha, beta, and GA, and we perform a lot of checks. We specifically focus on the threat model to make sure the controls are in place if there are any threats related to consumption of that particular tool, as well as identifying the risks and mitigating those risks as part of the pipeline itself.

The second part is the pipelines, because development teams are specifically focusing on building their pipeline and consuming those enterprise tools. The way we encourage customers is by building typed pipeline templates, where we encourage them to bake in all the controls like security, compliance, and quality checks as part of the pipeline itself. If someone needs to override because of some reason, they need to go in and seek an approval from the security representative within their area.

We are also focusing on empowering the customers, empowering the developers to innovate and to practice the new ways of working, but keeping the society safe and secure, as well as delivering the value as fast as we can.

Sacha Labourey

That makes sense. You've provided some of those tools, frameworks, and templates so that developers, without having to worry about the how, can instantiate the right behavior within their lifecycle, and you work on making sure those templates do the right thing.

Do you handle just safety and security this way, or do you also handle compliance in the same way?

Sanmat Jhanjhari

Compliance is a broad term. The code is compliant, the processes are compliant. As soon as code checks into the source control, it goes through various phases: compliance from the pull request perspective, compliance from the scanning perspective, whether the code that the developer has written is secure and compliant, as well as the open source code that the developer has downloaded. It should be coming from the binary repository, which is proxied to the remote repositories. Then it goes into scanning of the container itself, which I'm going to deploy. That's the developer part of compliance.

The second part is about the approach to embed those controls in the work that we do. It helps us keep the products and services safe, keep our members and colleagues safe, and deliver what we call safer and sooner. It ensures a fast-paced deployment, as well as ensuring we have sufficient controls to reduce the risk and deliver safely.

It involves identifying the risk as early as possible, early in the development stage itself, so that we identify the right work at the right time and mitigate those risks. The way we are doing it is moving away from discrete processes, separate controls processes, which are quite confusing and complicated from the customer point of view. For example, multiple points of engagement and multiple assessments create a lot of confusion for customers because there are so many processes.

What we do is have intelligent controls that aim to drive a consistent and standard approach for understanding the risks across the society, which enables confidence in our risk position and ensures the right controls are embedded to reduce the risks. There are various parts: consistent risk assessments, minimum viable controls, minimum viable governance, automation, assurance, and so on. There is a lot of work we've been doing recently about ensuring those controls and risks are identified and embedded within the pipeline itself.

Sacha Labourey

That's great. Essentially, what we can see here, which is super interesting, is that as we talked to Ben, we had this first wave where you spent a lot of time automating the software delivery process. Now, Sanmat, you're talking about how you're automating the compliance process as well by embedding it, and compliance in the wider sense of security and so on, by embedding those requirements into the software delivery process, so that both can be achieved at the same time, in some way at the same cost from a development standpoint.

One thing that's very intriguing is how you encourage collaboration and adoption at the same time while empowering the developers and the team. We know that sometimes when you talk about security, safety, and compliance, it's not the best way to get developers excited. How do you get that going?

Sanmat Jhanjhari

Nationwide believes in empowering people and creating an environment of accountable freedom. That's a buzzword that's been used in Nationwide quite often, and it is one of the key pillars in the Nationwide leadership framework as well.

What developers want to do is constantly look for new experiences, new libraries, and consumption of new tools and applications. Why not do it for the DevOps tools as well? In the enterprise tooling capability, we are aligned with the value stream teams. We are feature teams working with the value stream teams to deliver based on their requirements. We identify the key requirements that need to be enabled from the centralized team. We have a product owner who is closely working with the business to identify their needs and prioritize the backlog for us.

That way, we manage the requirements, and working with the customer helps us assess whether the work being delivered from the centralized team is fit for purpose and fit for use, so we get fast feedback. Nationwide is going through a big transformation phase at the moment. There are a lot of teams still learning what CI/CD is all about, and there are a lot of teams that are literally one-click deployment, no click actually. A developer checks in the code and the code may go to the pipeline and into production deployment, but again, passing all the checks.

We work with the teams and embed the DevOps engineer within the team to understand their ways of working, identify where the bottlenecks are, and work with them to create the value stream mapping. After identifying the gaps, we work on those gaps and fill those gaps. Then the team members are shadowing, the customer we are working with shadows us. Thereafter, we do a reverse shadow to make sure that the controls or improvements we have built into the pipeline for those customers are well understood. Once after finishing the reverse-shadowing process, we sit back, completely come off, and work with another value stream where they need our help to amplify their release frequencies.

Sacha Labourey

Thanks, Sanmat. How did that relation with Ben go? You might have first been on two opposite sides of the table, but ultimately this will only work for the company if you can both win.

Sanmat Jhanjhari

That's really interesting. Me and Ben originally started this whole DevOps and development engineering, which turned into the CI/CD platform team. The relationship with Ben is that we are all aligned. We work in an agile framework, and the way the work comes into our team is decided by the product owner. Ben is in the leadership forum as well, to understand which are the critical customers, which are the customers that we should be focusing on, where we can drive the values faster and better, and then prioritize the work. It's going really well, working with the leadership team as well as the value stream team that we are closely working with.

Sacha Labourey

I'm going to ask both of you a last question. If anybody in a different company was to start the same journey today, what would be your advice to them? Ben, any advice you want to share?

Ben Angell

There are lots of lessons learned. I mentioned it a minute ago. Measuring your organizational efficiency before you even start the work is probably the most important bit of advice I could give, because you need to be able to describe how efficient you are as an organization and how DevOps CI/CD pipelines will enable improvements in that efficiency. I can't understate it enough. It's absolutely critical that you're in a position to be able to do that and describe that. That's a massive lesson learned for me and advice I would give to anyone starting this journey right now.

Sacha Labourey

Thank you, Ben. What about you, Sanmat?

Sanmat Jhanjhari

Just to add to Ben's point, the important thing is to go back to basics. It's important to understand the requirements. We often see in the past, not only in this organization but with other organizations as well, that there is the most emphasis on tooling. Don't go on tooling first. Understand what capabilities are missing. Focus on those capabilities, understand the requirement, what is required for an organization to deliver something. That's really important.

Again, work in a product-centric way. Don't deliver in a big bang approach. MVP is the first thing. Get the fast feedback, and then definitely you'll get a success. Specifically around aligning with the value stream teams: that's a really critical part about delivering any centralized services if you really desire to do so.

Sacha Labourey

Thank you, Sanmat. Thanks to both of you. Thank you, Ben. Thank you, Sanmat. To everybody listening to us, I hope that you enjoyed this speaking session for DevOps Enterprise Summit. I know that the three of us will be online to answer some questions, so have a great day. Thank you, Ben. Thank you, Sanmat.

Sanmat Jhanjhari

Thank you, Sacha.

Ben Angell

Thanks, Sacha. Have a great day.