Your Roadmap to Continuous Delivery and Continuous Testing
Duncan Bradford is Vice President, Presales, and CTO at CA Technologies for Europe, Middle East and Africa. In this role he is focused on helping organisations best leverage technology to excel in the application economy. He is passionate about helping organisations to remove the barriers between ideas and outcomes.
He joined CA Technologies in 1998 and has held several senior customer facing roles. In 2007, Duncan moved to New York to take on a global Architect role in CA’s Infrastructure Management business and in 2010, he was promoted to Vice President, Presales. This worldwide customer experience gives him a unique global perspective on the mounting opportunities and challenges organisations face today.
Duncan is an expert in helping organisations embrace business agility to deliver on the promise of the modern software factory, by focusing on four key principles; agility, automation, insights and security.
Before joining CA Technologies, Duncan held technical customer facing roles at Wick Hill and Exel Logistics, now DHL. A keen motorsport fan, Duncan lives in the UK and is married with two children.
Chapters
Full transcript
The complete talk, organized by section.
Duncan Bradford
My name's Duncan Bradford. I'm the EMEA Chief Technology Officer for an organization called CA Technologies.
I won't go into details about our company. Suffice to say that we have a large booth giving away guitars in the expo hall. So please, if you haven't already, and you want to learn the guitar, head over and maybe learn about some of our technology.
The goal of this session is really to try and have a conversation with you around some of the situation and the challenges we see as we work with many of our customers to help them achieve continuous delivery of value to their customers. So I'm going to talk high-level around some of the things we see. Happy to go more in-depth as we progress through the show and afterwards in the expo hall.
One of the things I want to just preface and frame this conversation with is that technology is changing. The way that we build software has changed. The fact that we're at a conference labeled DevOps, that's the way everyone's working. Agile and DevOps together are really what help deliver continuous delivery of value to our customers.
But really, I just want to touch on a few things that have changed over time. We went from the monolithic, waterfall-based model. I started my IT career as a test lead, and I was very much working in that waterfall world. And I would say it's true: it was one to two years for a release. Many requirements up front. Let's build a product, take a year to build it, and then let's start the cycle again. Very limited feedback loops.
And you guys all know about how that's changed now, and we've heard stories. Barclays earlier were talking about they're doing multiple deployments every 12 seconds. Amazon, very well known. So there's a lot of velocity when it comes to delivering experiences.
Where that's going to change, and I don't have all the answers for this yet, but I see that as we move more into one of the next big things in terms of how applications are going to affect our lives, is around the use of machine learning and artificial intelligence. And that means many things, and that's probably a separate topic. But although humans are going to still be driving the design and the creativity of applications, there are things happening already that are going to change the way we work in the near future.
For example, augmented programming. There's already robotic process automation that is being able to use machine learning to take front ends and build the code for applications. So things are going to be happening a lot faster, and the tools that we end up using and some of the challenges we face are going to ever evolve.
There's been many presentations today, over the last two days, that talk about velocity and quality. I think the two go hand in hand. There's no point delivering poor-quality stuff faster. We need to deliver experiences that our users love with velocity, but absolutely quality is key.
And why is quality key? What are some of the drivers or some of the points that could fall out of a relentless focus on velocity and quality? Security, user experience, and cost.
So let me just elaborate on those. The number there that you see around $7.7 million is the average cost to a company. I guess many people think that's quite high. But if you look at Equifax, who suffered the enormous breach through a web application last year, 145 million people in the United States affected. And I had to check if I was, because I used to live there, and I have a Social Security number, but I think I'm safe.
But they spent $114 million remediating that security flaw. They lost the CIO, the CISO. Massive impact on the company. So you can start to see how these numbers will start to play out.
User experience, I think we're all users. There's no secret sauce there. If the user experience is not optimal, we're going to move away from apps. Gartner talk about the user experience really is the competitive differentiator in business today. So we need to deliver experiences with velocity and quality that also deliver a fantastic experience.
And then just a point on the cost. I have met people who are focusing on Agile or DevOps because they want to reduce cost. I think cost is an expected side outcome, because the very nature of adopting DevOps and ultimately continuous delivery is to implement lean ways of working, lean measures, and remove waste, continuous feedback. So it should be more efficient. So the cost angle should be an outcome.
But then we look at how do we measure if we really are delivering on these outcomes. But if we don't deliver on what the business needs, and I touched on Equifax, don't want to pick on them. But in terms of brand risk and lost revenue, by missing that security vulnerability or flaw, you can see how it opened them up to immense risk, brand recognition issues, revenue, and the value that's delivered.
So that's the situation. So why don't we just solve it and deliver great experiences with that velocity and quality, whilst encompassing all the great user experience, being absolutely sure that we're delivering frictionless security as part of that experience, and we're doing it in an efficient, lean, repeatable manner?
Well, there are some challenges.
Just to really touch on these. Value stream clarity. You can see the references. Actually, there are references for all of these. They're on the notes. But these are all taken from industry reports across DORA, some of our own done through Freeform Dynamics, and State of DevOps Report, and Veracode State of Software Security Report.
But some of these I think are interesting when you look at them at a macro level across the challenges that people face. For example, value stream clarity. So this talks about dependencies.
How I would phrase that for you guys is that when you look at delivering a great software experience, remembering this is the DevOps Enterprise Summit, so this is all about enterprises achieving great outcomes and delivering value at scale. We're not talking about just a brand-new small startup with a couple of Scrum teams. We've got to do this at scale, which means many of the organizations that have a great heritage, I think the quote from John Smart was around Barclays have been around longer than the U.S. has been formed.
So as you can imagine, they have a lot of systems, and Gartner would talk about systems of innovation, systems of differentiation, and systems of record. So when we work with a mobile app from a financial organization, it is traversing many different systems, CICS transactions, maybe talking to a mainframe, legacy sort of web-based app servers, and so forth.
So understanding the value stream is a challenge for organizations. There are dependency challenges, and organizations are looking to solve that through the use of APIs. But what I would add is that we've had a lot of progress working with organizations and performing value stream mapping, which is really a sort of a future-state perspective, where you get all the stakeholders together who understand, from the business all the way through to production, what a day in the life looks like for taking a feature or a story to a customer. So having clarity into the dependencies helps remove those constraints.
And talking of constraints, Computing did a survey. Testing is something very close to my heart, and I know some of the people in the room as well. But Computing did a survey, and amongst DevOps practitioners, 63% said there are constraints throughout the SDLC in terms of getting value to your customers. But the biggest constraint, which came highest here at 63%, was around testing. So how do we move testing on from which hasn't evolved at the speed of Agile? And I'll touch on that.
Cloud migration. You may think, "Okay, how is cloud migration really a challenge?" What I think is interesting here is the drive. IDC published some research quite recently about the desire for organizations to adopt a multi-cloud strategy, and many public large financial organizations have announced multi-cloud strategies.
And I'm not really talking about the nirvana of moving workloads between one cloud on the fly based on cost and capacity. It's really the complication of how do we deal with these multiple clouds that we might be wanting to deploy services that get to our customers.
Fragmented tools. In the ways of working with DevOps, should we be prescribing what tools people use? What I would say here is that it's a balance. So when we work with many organizations, whatever the tools are, and we touch on this, it's a very open toolchain in terms of the SDLC and value to your customer. But there are a lot of drivers and good arguments around being able to have a delivery pipeline, so we're not wasting, so we're not replicating tasks and turning into a tools Wild West.
But it is a challenge, because if you want to scale, and I'll talk about a story of an organization who scaled that delivery to their customer. If you want to scale, having fragmented toolchains that are really focused on their process and their tools isn't going to really scale to deliver value to the customer.
Legacy versus modern. So I touched on this with the value stream. Today, there's five billion new lines of COBOL code going onto mainframes. So those mainframes are hosting 80% of corporate data. So there's a lot of value in data on the mainframe.
Now, I'm not trying to pitch about mainframe here, but I'm just trying to articulate that when we want to deliver value to our customers, we are going to work with new technologies. We are going to utilize APIs to decouple and remove some dependencies. But there is still that concept of innovation, differentiation, and systems of record.
Visibility. We heard from Nicole yesterday around how their measurements with DORA. It's very important to have visibility into, A, my cycle time, so where can I improve? But also, what are the metrics that show that we are being successful? And for organizations, especially when they don't scale across a pipeline, have problems with that.
And then we touched on security already. The whole shift left of security. Saw an interesting presentation by AWS just earlier before lunch, where they're really talking about the shift left of security. That's something we believe heavily in. I don't want to jump on the DevSecOps bandwagon, but absolutely, the ability to shift security left all the way to the IDE is going to ensure that we don't run into these problems where we deliver great experiences, but they affect our brand through security issues.
So as an organization, we help guys like yourself and other large enterprises remove some of these constraints, remove some of these challenges. And we help them do that by helping them build what we call their own modern software factory.
A modern software factory is pretty much predicated on being able to eliminate the barriers between taking an idea. So that's an idea. John Smart talked about the whole PMO, how they're transforming even further left than the SDLC. But take that idea to an outcome to put value in the hands of your customer with the velocity that's required and the quality.
And a modern software factory is predicated on four key outcomes, if you like, which is agility, automation, insights, and security. But I won't go into that in any more detail, but happy to talk about that on our booth area.
So let's just drive a little bit more into some of the capabilities that we believe remove some of those, or help solve some of those challenges in the enterprise.
The first, we've talked about end-to-end value delivery. So I talked about value stream clarity, about having insight into how you're delivering experiences, about being able to understand the dependencies and where you can decouple, where you can, if you have dependencies, how can we break them?
So to have that visibility into how you take a user story to production is a key outcome that we deliver with our technology. But it's also a key outcome that you should be able to surface yourselves through any technology that you use. But really helps you orchestrate and understand the business impact of the pipeline of value to your customer.
Continuous testing. So continuous testing is an industry word. What is continuous testing? Continuous testing means many things. To me, it's about being able to continuously test across the whole SDLC and discover a defect as soon as it's introduced. It's about testing at the speed of Agile. It's about modern testing. It's about being able to really take some of the testing practices that have existed for many years and have them work at the speed of Agile.
Now, I do want to say that, as a software vendor, everything we talk about here, culture and organizational design is a key factor. I'm going to focus here on the tooling aspect, but please take that. When we talk about test transformation, testing at the speed of Agile, that's a change in the ways of working which tools help deliver, that new, innovative ways of working.
Model-driven releases. So when we talked about cloud migration, and I made a point about the multi-cloud scenario, where organizations that have invested are definitely looking at it. But the data shows that it's an area over the next 12 months we expect to explode.
Now, you do see some, I'd say some standard implementations of multi-cloud might be where someone's invested in AWS, and maybe they have Azure for apps. So I've personally come across that in many organizations. But this move towards a multi-cloud, don't put all your eggs in one basket, to look at cost optimization and so forth, that if you do go down that model, it does require that you have a focus around model-driven releases.
And this is about decoupling and having an abstraction layer so that you can deliver the same service without being hindered by the underlying infrastructure. So it's about having an abstracted, decoupled way of delivering value to your customers.
So we talked about fragmented tools. The point here about open and integrated. Anyone who wants to talk to you about a software delivery, value delivery pipeline to customers, it's an open and integrated toolchain out there. Open in terms of open source. There's a lot of good open source products out there. There's a lot of good niche vendors out there.
So it's really important, as we deliver value to our customers, that we are able to work with this open and integrated toolchain. So from a CA perspective, if you were to look at our stand, you'd see that we, like the other vendors you may be aware of, that it's really important that you have plugins that support the likes of Jenkins all the way through to configuration management tools.
So these type of software delivery tools are not focused on removing the likes of Jenkins. They're talking about taking an artifact, taking it through all the phases it needs to go to get it fully built, tested, deployed, and then delivered to a customer.
Mainframe to microservices. So I touched on mainframe earlier, and I know mainframe is probably not the number-one topic at a DevOps conference, but I think it's the reality when you've heard from some large banks speaking at this event, the mainframe does exist. And I mentioned about how five billion new lines of COBOL go into a mainframe each year.
So organizations are transforming the way they do development on a mainframe. So it's unlikely to find an enterprise who has a mainframe that has adopted Agile, DevOps, is moving towards continuous delivery, and that mainframe is part of their system of record, that they haven't adopted new ways of working around a mainframe. Which means they need to be able to orchestrate from the IDE all the way across the mainframe and ensure that they're working in the right ways to deliver value to their customers.
Visibility and analytics. I touched on that as being one of the problems. I think there's a lot of benefits in being able to see really the value stream in terms of, this is a user story. Whether that's coming out of a tool like Jira or Rally, I can see this user story, I can see all the phases it's gone through, and I can ultimately ensure that I can optimize that through understanding of my SDLC.
So what my cycle time is, where I'm spending my manual time. It's another piece around visibility and analytics, which does link into security and compliance, is also about trust and transparency.
So I've worked with an organization, and they had a large organization, and they'd adopted AWS. I'd say they had a very full-on Agile team. They were working in sprints, a couple of Scrum teams. And the challenge they had was when they came up against some of the more traditional ways of working in an organization.
So they told us that they're working in two-week sprints, but they have to give two weeks' notice to the change advisory board of a change, and then there's a one-week SLA. Now, I think Nicole talked about change advisory boards yesterday. I don't think she's a massive fan. But I think the reason she brings it up, because they do exist and there is a clash, especially in the enterprise.
So how that organization was able to solve that problem was to work on a couple of things. Automation to not only scale, control, and accelerate the SDLC, but how could they extend that automation into the wider enterprise, which then led to trust and transparency, because the change team could see the flow of a user story or a change, in their world, of how that had gone through, who tested it, segregation of duty, audit and compliance capability.
So that enabled them to then expedite those changes and basically halve their time to market. But that also talks to when I talk about the value stream. So that came out when we looked at a value stream for this customer, where this bit's great and this bit less so.
And I've spent some time with customers where I could say, and I hark back to John Smart's presentation earlier, where the only thing that's Agile in the delivery of value to their customer is the sprints. They still have issues around production. They still have issues around finance and approval. So that whole move from projects to products is a change for organizations. So that visibility and views does start to help and give that trust and transparency.
And then compliance and security. If you heard from NatWest, Jenny, she would've talked, and John touched on it, that compliance and security in the world of financials is an extremely important topic. So they need to have the velocity whilst also ensuring that they're securing their customers' data, whether you get into the world of GDPR or not. But also, how do we handle this from an audit perspective?
So what they're able to do is they need to ensure that they're able to have full audit and compliance, make sure that their applications are running through the security scans, shift left, and so forth.
So really, these are the capabilities that help enable continuous delivery. And as I mentioned, there will be the aspect of absolutely, we're in a cultural movement, for sure. And organizational design is a key outcome from that. But tools alone won't solve the problems we've got or remove the constraints we have, but tools will help deliver new ways of working and deliver new innovation. So they are a key. It's back to people, process, technology again.
So if I said that continuous delivery was a maturity model, I would expect you to throw rotten tomatoes at me, because that's not really the way that we should be looking at these things, right? Continuous delivery is a journey.
Let me talk to you about an example of someone we know and how they went on a journey. They presented here last year at DevOps 2017, so ING. ING, I'd say, is a DevOps leader when it comes to the financial, from digital banking. And that's really the guys, there's good ways of working on the ground, but a lot of that was led by their CEO, Ralph Hamers.
So in 2014, he said, "People need banking, but they don't need banks." And I think that's a great cultural example of leadership driving a change in the way they work. And they've proven it because they were the first bank in Europe to deliver voice-activated mobile banking.
But how did they do that? And they quite openly talk about their journey, and we help power that journey. But their journey really started with, they were doing DevOps and Agile. But what they were doing was some of many of the challenges that I talked about earlier. They were working with fragmented teams and tooling who are really focused on process and the tools rather than delivering value to their customer.
So one of the challenges they had is when their CEO says, "We're going to be the number-one bank. People don't need banks, but they need banking." Part of that journey was going to be driven through their mobile app. Now, the problem with their mobile app was they had delays in terms of feedback and getting that information, customer feedback, into the development lifecycle and being able to drive that through to a change.
So what the team did was they were doing Agile and DevOps in pockets. They read the Continuous Delivery book by Mr. Humble and Farley, et cetera, which they call the Bible, and they built continuous delivery as a pipeline. So continuous delivery as a service.
So they don't predicate every single tool that people should use, but they do have an approach that these are the tools that we recommend, and don't duplicate work. And by driving that pipeline, they're able to deliver on being the first bank to deliver voice-activated mobile banking. Also to be able to increase that mobile app, which had a one-star rating, which is never good for the CEO, all the way up to a five-star, and then be seen as a leader.
And it is a journey for them, because they have 750 out of about 1,500 applications that are going through this pipeline. So it's a journey. It doesn't happen overnight. And where we would help organizations like this is to remove those islands of automation, help them orchestrate that toolchain, and then go to be fully automated and optimized.
But like I said, this is a journey. Technology changes. I talked about on the first slide about how the state of applications will continue to change. So this journey can't stop when you're customer-obsessed. Like Amazon would say, their goal is to be the most customer-obsessed organization on the planet. Customer needs to be at the centricity of what we do, and what we want to do is put the right products at the right time in the hands of our customers. So that's why this is going to continue to be a journey.
So I thought I'd put a slide in here with loads of bullet points so you can all read it, because I went to presentation school, and it said, "Put as many bullet points on as possible." No, actually, that's British humor.
What I want to do is I want to talk here about an airline. Another scenario, another customer outcome.
Imagine this scenario that you're just heading out on holiday. You've got your family with you, mixture of excitement and stress as it would be. And you get dropped off at the terminal, and as you go into the check-in, you see that there's a lot of queues. And you think, "Uh-oh, what's going on here?" So you check around, find an assistant. The assistant tells you, what do they tell you? What do you think they're going to tell you why there's big queues?
There's been a problem.
Yeah, there's been an IT failure. So that's terrible in terms of brand, in terms of revenue. There's a big problem.
So then picture, imagine that you're in the war room of this leading airline that's just had a major outage that grounds their planes. So you can imagine a lot of finger-pointing. Ultimately, they had to change the way they were working. They had to focus on a couple of core issues, which was around configuration drift caused by silos of automation and scripting.
So they looked at how they could replicate what the likes of INGs of the world have done. And also, they run into this, the configuration drift also was related to their test data. So they didn't have credible test data.
So they went on a journey of generating data, synthetic data that they could inject and virtualize across the lifecycle, which meant they could expedite the delivery of new services, but underpin that with automation across all the different islands to then remedy this problem. So you can imagine that there was a couple of heroes and there were a couple of late nights involved in that.
So with that said, I want to finish, just to really point out that we've got a great book that you can pick up from our stand, and I know that we all like something decent to read. So this is actually a good book. Aruna, Kiran, and Peter, I know them personally. Aruna was voted one of the most influential people in DevOps. She's done many events with Gene Kim and Nicole and so forth.
So this is a good book about digital leaders and how to embrace DevOps on that journey. So you can download it, in the interest of not wasting books. You can obviously download that from our website, or please go to the booth, try and win a guitar, pick up a free book.
With that said, I thank you very much. I think we're almost at time. Thank you very much for your time.
Thank you.