Building Better - The Next Step in Software Delivery
Welcome to the software economy, where developers unleash disruptive innovation at an unprecedented rate. If you are a non-trivial software business (and pretty much every non-trivial business is) then guess what? You're now competing with every other software business in the world.
Research tells us that organizations that successfully implement DevOps practices and efficient CI/CD pipelines are able to deploy faster, fail less frequently and recover more quickly from downtime. (source: DORA 2018).
Cheryl Razzell (Global Head of Digital Platform Operations, Digital Platform, at HSBC) is undergoing such a digital transformation to increase their release cadence and quality, while also improving governance and system stability. But as she eliminates bottlenecks, more appear on the horizon! In many enterprise environments, islands of information and automation still exist, adding opacity, friction, cost and risk to the software delivery process.
Some of the problems we hear include:
- Engineering not knowing how their latest change impacted system performance or user behavior
- Customer success teams viewing the wrong version of a feature flags documentation
- Ops, marketing and sales having trouble keeping up with and effectively leveraging smaller, more frequent releases
- Senior management unable to prioritize releases based on the value they contain.
In this talk, Cheryl Razzell (Global Head of Digital Platform Operations, Digital Platform, HSBC) and Sacha Labourey (CEO, CloudBees) will discuss the 5 key things organizations should consider as they work to connect their software development efforts to the rest of the business and allow the organization to achieve a harmonious state of "Continuous Everything."
Sacha was born in Neuchtel, Switzerland and graduated in 1999 from EPFL. It was during Sacha's studies at EPFL that he started his first consulting business - Cogito Informatique. In 2001, he joined Marc Fleury's JBoss project as a core contributor and implemented JBoss? original clustering features. In 2003, Sacha founded the European headquarters for JBoss and, as GM for Europe, led the strategy and partnerships that helped fuel the company's growth in that region. While in this position, he led the recruitment of some of JBoss? key talent and acquisition of key technology. In 2005, he was appointed CTO of JBoss, Inc. and oversaw all of JBoss engineering. In June 2006, JBoss, Inc. was acquired by Red Hat (NYSE:RHT). After the acquisition, Sacha remained JBoss CTO and played a crucial role in integrating and productizing JBoss software with Red Hat offerings. In 2007, Sacha became co-General Manager of Red Hat's middleware division. He ultimately left Red Hat and founded CloudBees in April 2010.
Cheryl has worked within many large organisations like the BBC, Apple, News international, Microsoft, HSBC. Over her career, she has undertaken many roles from Desktop support to project management and Global IT Director. Cheryl has more recently shifted her focus within her career in technology to move away from traditional IT and into the world of engineering where she has in her past 2 roles lead DevOps teams to build and support the CI/CD journeys for both Yammer and more recently for the past 2 years HSBC. In her current role at HSBC Cheryl has built a team and function globally from scratch to support the new Digital Platform for HSBC retail and wealth and banking consumer products. As an advocate for agile devolvement, she is working within the business to move away from monolithic development practices and to adopt a mircoservice and API environment. As well as supporting the on premise CI/CD platform, she is also in the process of building out a cloud-based platform that will allow growth with the use of automation and chatops.
Chapters
Full transcript
The complete talk, organized by section.
Sacha Labourey
Hi, everybody. I'm very happy to be here.
I'm especially happy because Electric Cloud was one of the founding members of this conference, and the team joined CloudBees a few months ago, so that makes it a very special reason.
So today, first I'd like to welcome Cheryl Razzell on stage. She has a title with lots of words, I have to say. I couldn't learn that: Global Head of Digital Platform Operations at HSBC. Welcome, Cheryl.
Cheryl Razzell
Thank you, Sacha.
Sacha Labourey
Yeah. Thank you.
Cheryl Razzell
We rehearsed this really well, by the way.
Sacha Labourey
Yes, we rehearsed that. Actually, we've been on stage quite a few times.
Cheryl Razzell
Yeah.
Sacha Labourey
And last year we were at the keynote at DevOps World | Jenkins World.
Cheryl Razzell
That's right, yeah.
Sacha Labourey
So we're not going to do the same presentation, don't worry.
Cheryl Razzell
No.
Sacha Labourey
Not the same presentation at all.
So Cheryl is going to talk first about the digital journey at HSBC. You'll see it's an amazing project, a massive project, so it's very important, very interesting. And then we'll talk about some of the challenges faced in any digital transformation and the one at HSBC, obviously. And then I'll say a few words about how we see the strategy going forward at CloudBees and how we solve those problems. All right?
Cheryl Razzell
Absolutely. Okay. Perfect.
Sacha Labourey
The floor is yours. Thank you, Cheryl.
Cheryl Razzell
Fantastic.
Sacha Labourey
Do you want me to hit clicker?
Cheryl Razzell
Yep. Is it this one here?
Sacha Labourey
That's it. Yep.
Cheryl Razzell
Oh. Sorry.
Hi. So I'm Cheryl Razzell. I've actually changed my job title to have more words in it just to confuse you, Sacha. So I'm now Head of Digital Tooling, Monitoring and Engineering, and I look after the CI/CD platform for digital.
So let me talk a little bit about the digital transformation at HSBC. I've got the slides up. Yep. Okay, perfect.
I want to talk about the journey of HSBC and our journey to integrating new digital platforms into the organization. Just let me talk about the scale of HSBC to give you an idea of the complex situations that we're trying to resolve. We have around 3,900 offices, 67 countries worldwide. We have turnover about 21 billion in profits. We have around 2.52 trillion worth in assets. We've got 229,000 employees across the globe, and around 38 million customers.
So about five years ago, we set up a team within the retail bank called HSBC Digital. This was our start of our transformation journey, where we were looking to bring the IT teams and the business together to create true cross-functional teams.
The journey was about the transformation of the bank, and consumerization of IT has changed for our customers. As a bank, we need to evolve our requirements and our needs to provide those applications to our customers. So we wanted to increase our delivery velocity of our customer journeys. We wanted to leverage the data and the value of the insights of those customer journeys, and we wanted to test, build, and iterate more quickly on our product that we were delivering to HSBC customers. And we wanted also to deliver more engaging experiences to our customers.
It was never going to be easy to reconstruct the way that we work, considering how large the organization. Some of the complexities that we've had: it's difficult enough with legacy banking platforms, widely dispersed geographical teams, and operating and innovating in a highly regulated environment. And of course, our customers' needs are evolving rapidly, and the demand on our capabilities from us every day are changing, and we need to keep up with those demands. So not only do we need to meet our customers' demands, but we also need to exceed them. And where IT used to be part of the business, now IT has become the business.
So this took a rethink within the organization looking at our infrastructure. We needed to start breaking down our large monolithic websites and applications and turn those into microservices and APIs.
To give you an idea of how far we've come as a digital organization, prior to 2016, we were delivering around 10 features per year. When I talk about features, that could be a new mobile app for our HSBC customers. It could be a new platform on one of our web platforms. So these are quite large features that we were delivering to our customers.
In 2017, we went to about 108 over the course of that year. And to date, we've just released, I think, 500th feature to our customers this year using our digital platforms.
So hands up, anybody in the room is a HSBC customer. Okay. So hopefully you'll be more familiar with one of our recent applications, which is our mobile application, which is our internet banking application for HSBC UK, Canada, and Hong Kong customers.
This application was built on our new digital platforms using CI/CD and using our new digital platforms hosting our microservices and APIs. This year, it was successfully deployed to the UK and Canada and Hong Kong. It was written by the team and delivered on our platforms, and it ranked very highly on the App Store. So we're very proud of the accomplishment, and I think if you're a HSBC customer, you'll see the value of the new application where things like paying in a check, they're now simple. Where you had to go to branch to do that, you can do that on your mobile phone. So we're listening to our customers, and we're evolving our products around our customers and trying to deliver faster to them to meet their requirements.
HSBC mobile app, I'm using this as a good example for the way that we've applied our agile ways of working, releasing new products to our customers. To give you kind of scale of the application, we have over seven and a half million browser sessions globally in a 24-hour period. We have 3.5 million via computer and around 4 million on mobile devices. Seven million mobile sessions over a 24-hour period. This just shows you the demand on our products and how we have to keep our platforms up and running, and also how we have to iterate and try to use CI/CD to the best that we can and the best of our ability within the bank. We would have only been successful in doing this had we adopted agile ways of working. And that's the use of CI/CD.
So we're now four years into our journey at HSBC Digital. We're starting to look around the capabilities of CI/CD and how do we evolve our CI/CD offering. We've now started to build and develop and push our products to our customers. But agile ways of working is a constant iteration. How do you improve things? How do you make things better? Not just for our end customers, but also for our customers internally, which is our developers.
As an organization, we need to think about many things about governance and compliance and security and process. And we need to make sure that we bake those into our pipelines and that we bake those into the development cycles within our teams. This then takes us onto the next question around metrics and how do we measure our software delivery cycles. Where do we understand where the blockers are? Where do we understand where our pain points are within the organization? And then at a higher level, how does the organization start to look at our software delivery cycles, and how do we understand where teams need a little bit more support, where we have bigger maturity areas within the organization, and where we have pockets of teams that we need to work on their software delivery, and we need to improve our process to support them?
So this brings me down to CI/CD and working with CloudBees. There's a few things that we need to take into consideration with the evolution of our CI/CD. One of those is risk and traceability. How do we take the control and risk of CI/CD without introducing new risks to the business? And how do we track accountability and adherence to governance without losing autonomy, which is really important for our developers and the way that they deliver?
Also, measurement of success. How do we measure the success of our deployment teams? How do we identify blockers, as I spoke about earlier? And how do we see how teams are performing across the estate in an organization that I have said how large HSBC really is?
And then visibility. Visibility gives us greater insight into our digital program. How are we delivering? How are we reporting back to our stakeholders on how we're delivering as a program, and how this initiative could be taken out to the further parts of the bank and selling and showcasing the way that we're working within digital, and how can we roll that out to the rest of the organization?
So I've got some questions for Sacha around the evolution of... I'm hoping that I can read this. Because these are questions that daily, me and my team, we have to deal with these sort of scenarios. So we're trying to think about how we evolve the CI/CD platform. How do we incorporate some of the things that I've talked about, the traceability, accountability, compliance, and security?
So how do we cater for different capabilities, but at the same time provide a compliant framework which allows us to add value but doesn't restrict the flow into production, allowing developers autonomy but in a controlled way? Point one.
Point two, source control. How do we, with the evolution of GitOps, with the added complexity of not only managing just application sources, but infrastructure as code, pipelines as code. You name it, everything as code.
Policy management. So policy management, how do we centrally apply policies on our CI/CD environment across the estate in a controlled manner? Again, going back to the autonomy of our teams, we don't want to control them, but we want to make sure that we have the right guardrails in place to help them on their way to keep them compliant.
Artifact management. A central way of recording artifacts deployed into production with traceability and also control.
And then alignment of tooling. How do we bring a central solution with an array of tools that is used across the estate?
There's my problem statements that I tackle daily, Sacha. I'd love to say that you guys at CloudBees are working to resolve some of these issues. So I'll hand it back to you.
Sacha Labourey
Yeah, thanks. Well, actually, stay around because if you're okay, I'd love to ask you to go a bit deeper in some of those issues you're facing, because I think they're interesting.
So there is this notion of compliance that's very important in the banking industry, obviously. And on the other hand, you were talking about autonomy. You want developers to have autonomy, creativity, improve efficiency. You moved from 10 features being released to more than 500 this year, year to date. So how do you see that problem? How do you perceive it in the day-to-day operations as being a problem?
Cheryl Razzell
So it's a difficult situation that we're trying to resolve. So we want the developers to be as effective as possible. We want to give them the agility and ability to develop when they need to. But we need to make sure that we safeguard our production environments and our HSBC customers. We've got some solutions that we've been looking at internally in-house to kind of tackle this solution. But it'd be great to hear from CloudBees how you're looking to resolve this on a greater scale for other organizations as well as HSBC.
Sacha Labourey
Okay, perfect. So I'll maybe show you what we have in mind.
Cheryl Razzell
Sounds like a plan.
Sacha Labourey
All right. Thanks a lot, Cheryl.
Cheryl Razzell
Awesome. Thanks, Sacha. Thank you.
Sacha Labourey
All right. So I'm always amazed by the size of those digital transformations because, well, you've seen the numbers. We've all been in a company, I guess, that's smaller in size at some point. And you already see how changing anything is hard. And so when you have those massive ships where you have to refocus and change the way governance takes place, it's pretty amazing.
All right. So the digital native companies have really changed the way companies are operating. And companies now know that they need to be software-first organizations. And software can't just be a side activity. It needs to be a core business function.
And so very quickly the question becomes, how do I get my time to value to be as small as possible? Because we see that in lots of organizations, they work for a long time on fancy features, nothing gets released, and it's as smart as if you were building chairs and tables and not selling anything. IKEA building a million chairs and then saying, "Let's try to sell them," right? You want to test the market much faster. So you need to iterate much faster.
That's really what the data tells us. If you look at the DORA report, it's very clear on that. Once you build that muscle of delivering, deploying more frequently, good things happen. You're first going to get the time from code to deploy to shrink and become much faster, which means you further increase the amount of times you can actually deploy to production, so one reinforces the other. And then you will fail less, which is amazing because it's counter-intuitive to lots of people, where if they feel like they go too fast, they're going to fail more. Actually, they fail less and they're actually going to recover faster.
And one of the things I discussed with Cheryl, for example, is in the UK, in terms of compliance, if your application goes down for more than 30 minutes, you need to have a complete investigation take place as to why it took place. So the recovering fast is very important. Anything that flies under the radar of 30 minutes, you're pretty much safe, right? Nobody wants to see that happen. But when you see an application going down and you have to go through a full investigation, that's a big problem.
And so we've talked to lots of organizations, and we've been sponsoring the Jenkins project for at least a decade at CloudBees, and the feedback is always the same. You start from the left, you start automating. It typically starts in development. You go further to the right, you start moving to continuous delivery, and great things happen. However, at some point, once you get enough maturity, you realize that you want more. It's not enough.
And you start hearing comments such as the one Cheryl was making. So I'm going to go through a few additional typical comments we receive, would be, for example, "I'm a developer and I'm being asked to develop features. Well, it's nice, but I don't care about features. I care about impact. I want to know what's the impact. I want to see what's in that feedback loop so I can activate changes much faster so I can increase my impact." If you're being asked to develop a feature, but you can't see that impact, why do you care?
Ops people, as Cheryl was saying, you want developers to have creativity, to have velocity, to go faster. Ops are getting fine with this, but at some point they need to satisfy compliance. So how do you satisfy this need for a safety peace of mind with this desire to go always faster?
Engineering managers. Engineering managers, they're focused on one project, but they probably depend on another project that might depend on other frameworks. How do you get visibility on all of this, especially as you need to satisfy strict deadlines? You need to deliver something by one date, but hey, you depend on so many people and you don't even know who those people are in many cases. So it's a complicated world out there. That's the bottom line.
And what we have today are kind of like microorganisms. You have teams here and there. They might be using slightly different tools, slightly different processes, sometimes the same, sometimes not. They're completely separated. And they're like microorganisms. They try to optimize things locally because the only thing that matters is to succeed locally at the project level, at the product level.
But if you wanted to query that environment and ask for data to have some insights, it gets very hard first. But then even if you were able to do so, nothing tells you that they're using the same vocabulary, right? It's a Babel tower. Saying A in one team doesn't mean that it's the same A in a different team.
Cheryl was talking about features. Well, I'm sure that other companies might talk about product or initiatives or projects. So what do you really query when you want to know how things are going? And if you cannot have data as a solid foundation, then nothing gets real. You can't create common processes across your world, right? You want to do compliance, centralized compliance, that applies to all of your project, fine. But what do you use as a baseline to make this happen?
So what we really need is to upgrade this set of microorganisms to a real brain. We want to connect all of those things together so that they work at the same drumbeat, so that the left hand knows what the right hand is doing. So at all time you can start working as a united company, as an organization, and not just as a set of distinct and disconnected organisms.
So how do we achieve that? Well, at CloudBees, we created this new software category that we called Software Delivery Management, or SDM. You can see that as the new category for software delivery, the same way as CRM would be a category for sales and marketing, or ERP would be a category for finance, for example.
And it's actually pretty simple. At the foundation of this, you need data, common data. And so if you are first able to align all of the data across your environment, but not just the data, the data model as well. You want to make sure that A means A for everybody, B means B for everybody. So you need to neutralize, to standardize everything on a common definition, on a common vocabulary.
So it's a DevOps data model, essentially. You want to feed the DevOps data model, no matter where the information is coming from. Once you've got that, you can start building value on top of that. You can get universal insights, insights that really apply to your entire universe. You get to start connecting processes. You want to have regulatory embedded into your processes? Well, make it happen. Now you can, because you can measure and detect situations equally everywhere in your universe.
And you get to have those insights, and it's really enabling collaboration in the end. But true collaboration, the one that amplifies the value that's being created, and so that's really what SDM is about.
So sometimes we get the question, but so is SDM against or a replacement of CI/CD? No, not quite. CI/CD is a fantastic layer. We're not negating this, and you don't need to change anything. Keep everything you have. It's great. It's built upon CI/CD, and it collects the data from CI/CD, and it's going to start managing the delivery process and help manage the delivery process within those tools.
So another way to visualize this is imagine you have to the left of that screen a bunch of those tools. Not every company, not every team will use all of those tools. Some team might use one of them, several of them. And you have your system of record where you want to feed your data model. Essentially, you want that type data model, this DevOps data model, to start normalizing this data and make it a source of truth so that any team in the organization can start either creating tools or applications on top of this platform so that you can query for specific things.
Maybe some engineering managers would want to know what team is having the most efficiency, or as Cheryl was saying, what teams need more help in getting faster and unlock, unleash some value that they're facing. Maybe a development organization will want to navigate through some problems that have been described through user interaction because they have to develop new features. And yes, product management has done some work, but they really want to go to the source of the comments that have been made by the customers to implement that feature.
So the use cases are really unlimited, and it's not just about the tools that a CloudBees would provide. It's the tools that you could build on top of this data model once you have it fed by the data you've captured. And so in some way, this is going much beyond CI/CD. It goes essentially from the research project where you need to capture all of the research data down to the support documentation. You want everything to be aligned. You want to make sure, for example, when you release, that the right documentation is being pushed online at the same time. You're never not in sync.
And CloudBees is implementing this as part of what we call CloudBees v3. It's going to be the next iteration of our offering, and it is implementing the first SDM on the market.
So I'd like to put this slide well in mind because I think you're going to hear a lot about SDM in the next few years. And SDM is bound to become really a new category that a lot of vendors will start implementing because it makes a lot of sense. And much like today, you couldn't imagine a company that would not be using a CRM or would not be using an ERP to conduct their business. In a few years, it's going to make no sense to imagine a company having islands of project and team working in an unsynchronized fashion with no common data. Everybody will be using an SDM.
So if you want to talk more about the SDM, we have a team at the booth you can talk about. Or if you want to join us for the launch of this SDM, you can come at Jenkins World, DevOps World in August, in six weeks, about six weeks in San Francisco. And if you want to learn about that in Europe, we're going to have the same session, the same event in Lisbon, Portugal, in December, so a few months after, where we'll talk a lot about those things.
Thanks a lot, everybody.