BMK Lakshminarayanan

[00:00:02.420] Hello, and good afternoon. I sincerely appreciate every one of you here in the room because I know there is a Disney magic happening there. And thank you very much for coming along.

[00:00:16.600] My talk is on Bank of New Zealand's DevOps journey. If you wonder where is Bank of New Zealand, and where is New Zealand, there you go. Okay. We are very small, at the very bottom, and that's where we are.

[00:00:38.900] About Bank of New Zealand, we are a 150-year-old bank, and we have 180 branches throughout New Zealand, and we have 5,000-plus staff members, 2.5 billion in revenue, and our purpose is enabling high-achieving New Zealand, and our mission is to help customers be good with money.

[00:01:06.160] We have a full set of products ranging from personal banking, business banking, institutional, and private banking. Our purpose, helping New Zealanders be good with money. We make that by enabling digital banking for our customers. We help our communities. We run Scheme Savvy, we run Community Finance, we run other local initiatives, and help New Zealand small businesses, and also help New Zealand businesses to grow.

[00:01:44.800] Banking in New Zealand, I would like to give you a context, because everybody's talking about banking, everybody's talking about DevOps. And DevOps and banking is actually for a long time in this DevOps Enterprise Summit. But the market that we are operating in, and what is the competition for us in New Zealand in terms of our context? New Zealand is pretty small, actually. We are 4.5 million in terms of our population. And there are totally 26 registered banks. And there are 21 non-bank institutions. And out of the registered banks, four of the major banks are owned by Australia, including us. Bank of New Zealand is owned by National Australia Bank, headquartered in Melbourne.

[00:02:32.700] About me, my name is BMK. I'm following this DevOps community since long time. And I would like to give you a bit of a quick story about how I was attracted. I didn't come to the conference. I did not read about DevOps. But I was involved in one of the production rollouts for lending applications to do the back office drawdowns. I was a lead developer and architect for the program. And we scheduled our production rollout on Friday evening, 7:00, thinking that, "Okay, I'm a developer. I've done a number of production rollouts. I have ops team. Six of us on board. We can finish all this maybe in a couple of hours and go to bed early."

[00:03:15.420] Friday evening, it started at 7:00. And we finished our rollout on Monday morning, 3:00. And the Monday morning 3:00 is very critical. The reason: because Monday at 9:00, the application went online. It means the business users started using it. And that two and a half days of marathon effort to get our application to production opened my eyes. That was even before I came to know about the DevOps term. I went back Monday morning to my work. I started writing down all this information, and then started investigating more what could we do better. That's how I started my journey.

[00:03:56.340] Then I introduced, actually, a deployment tool. I did through the evaluation process. Oh, we could automate a lot of things. We used to write Word documents and wondering, can we do something different? 2014, June, this is what I'm talking about. And I went to the operations team, and I told them, "Hey, guys, I would like to do an automated deployment." The infra guy says, "Hey, mate, who does that?" After five years now, the developer goes to the ops guy and says, "Hey, my application is fully automated deployed, but I have to configure manually couple of things." The same ops guy says, "Hey, mate, who does that?" So that's the success for me in terms of changing the culture, changing the team, changing the adoption, changing the people's mindset about this.

[00:04:46.680] And my community: I'm involved in DevOpsDays New Zealand. I'm one of the core organizers. I co-chair the Cloud Native Summit Wellington, and I'm one of the CNCF Ambassadors. I run Cloud Native meetups. I run two meetups in New Zealand, Wellington-based. One is Cloud Native, and another one is Future of ICT. Future of ICT is to help students, women, people who are looking for career change to come into IT, and then help anybody who's returning to work after a break. So the whole purpose of the meetup is to help community. My mission: make your life better, make my life better, make others' life better, make the world better. And I'm a community man. I would like to draw a lot of inspiration from the community.

[00:05:34.960] Right. History about Bank of New Zealand. 150 years means it's not easy because we inherited a lot of systems, applications, culture, and whatnot. Still some of our systems are 40 years of code that has not been changed. 150 years means we have done throughout the years changes in the form of programs or smaller projects to improve certain areas, certain values, not looking at the holistic view of the entire experience of the customer. So we've improved in one area, what The Goal book talks about as local optimization. So we did really fantastic. We built, actually, a lending drawdown system better, but we failed to make the lending request system that great. So we had those kind of scenarios.

[00:06:25.824] Our business challenges, as usual, most of you have heard since last three days. We have problems in terms of business looking after... Well, we have a waterfall process. Business starts--their thinking is that IT systems are slow to change, because business, they think the speed of light, and the IT systems are not moving as fast as they wanted.

[00:06:49.284] Competing priorities and timelines, because technology teams are bombarded by the initiatives: from new initiatives for launching onboarding experience, lending experience, digital experience, mobile apps. And the other area, we have compliance, regulatory requirements to meet. And other areas, we have to look after the BAU projects, because the team that I used to be in used to support 70-odd applications, and we are maybe a dozen developers in the team, supported by a few contractors. But that's what the case is.

[00:07:25.324] And one of my friends told me very jokingly, "The ability to keep up with the regulatory changes is the biggest challenge, and forget about introducing new features." Because that's what happens if you're in a highly regulated financial industry. You need to meet a deadline and delivery line to be compliant, not to lose your banking license.

[00:07:48.144] The bankers who are helping the customers in the frontline, they have their own struggle, because they have to switch between systems to help and service the customers. One of the classic examples we had in a lending journey is that we had 21 systems. The bankers, frontline people, they have to help switch between these systems to service one lending request for one customer. That's really frustrating for bankers. But some of them carried on, some of them with the bank for 10 years, 15 years, 30 years, because a small country, and we have a lot of veterans in our bank.

[00:08:26.004] And these processes are time-consuming. And these systems, though I talked about 21 systems, if you think they are all interconnected in a modern engineering and modern architectural practice, they are not. These systems, if you have to copy and paste, open one window, copy the data, open another system, paste there. So this is what, actually, the bankers have to do. And the challenge is that they are always busy in copy-pasting. They're always busy in context switching between multiple systems. And the challenge is spending quality time with customers, quality engagement with customers. They struggle with that.

[00:09:03.224] So what does it mean for the bankers? Multiple processes, multiple systems, multiple people. It is insane complexity, and it's not easy. And there is no way that bankers can give the feedback directly to the technology teams, and there are no avenues. They have to come through the business, and the business has to come through the portfolio teams, and the portfolio teams have project managers, and the project managers have to engage the technology teams. So this is how the whole cycle is, and probably it might resonate with a lot of you if you're coming from that background.

[00:09:35.104] Technology challenges are a different side. I come from a developer background. So the challenges for developers are like this. We have to deal with legacy systems. Now we have mainframes because we are a bank. We have databases which are 20 years, 30 years. We have some of the giant monolith products and databases and systems. We have to get the data out of it. We have to move the data between multiple systems using batch files, CSVs, overnight process, whatnot.

[00:10:03.684] Deal with heavyweight processes and documentation and release calendar. Probably, again, this is something that most of you heard since last three days or may be hearing again and again. Centralized data modeling, operations, and security. I have to create a Word document, hand over to ops team. The ops team will then go and deploy it. And the Word document becomes actually the Bible for them to follow. And even if you want to go and change something, no, you have prepared the document. That's what we are following. Manual testing and deployments, dependency on other teams, times for upskilling, learning, and growing is another challenge. I think the bottom one is very critical if you want to be innovative transforming the organization.

[00:10:43.724] The technical service owners who own the system from technology, these business systems, they have their own challenges. The complexity in code, monolith systems, because the system is getting grown. Every time when somebody is coming, we keep adding a Band-Aid or plaster, saying, "Oh, let's do it. Add a new feature." Or, "You need to add a new broker capability. You need to add a new lending capability. You need to add a document upload capability." So we keep adding all those plasters in the system. It becomes gigantic in nature. The monolith, when we say we don't understand the complexity, but the reality is that your monolith systems, they are not easy. And everybody is now in journey, monolith to microservices, as we do.

[00:11:25.714] Juggling priorities and timelines, yes, because we have a small developer team. We have to always switch because if I am busy in delivering a new feature for a new system, somebody else has to come. If they say that I have a requirement for compliance, I drop what I'm doing. I start looking after the compliance project because that takes the priority, and because I have to meet a deadline.

[00:11:50.284] Then people have to switch between multiple systems, and it becomes easy. Projects to BAU. Forget about what we are talking now, because that's modern, projects to product, but this is actually projects to BAU. The technology teams, every time somebody comes, they form a project team, they develop a system, they throw to the BAU, and they disappear. And what happens? The team with 10 systems started maintaining 20, 30, 40, and now, the team when I left, actually, I'm in a different team now, but 70 applications.

[00:12:22.576] The technical service owner and the operations team challenge is all the same, because the ops team is sharing resources, and we have rippling incident effect. It means if something goes wrong in one server due to one application, and we take down other few set of applications. And priorities of patching, projects, BAU, always challenging, and then manual operations.

[00:12:49.436] What did we change? And this is one of our--it's really a busy slide, but I would like to focus only on the middle portion of it. This is the industrialization. Okay? It's similar to what Gene showed this morning. And what we are doing is to achieve our One BNZ plan, we need to make our system simple, and we need right skills, and we need to fund for industrialization, because you cannot throw the legacy systems and start introducing new systems. And you have to keep supporting them. You need to keep fueling them. You need to keep watering them. Some of the systems are the trees to grow. At the same time, you also introduce new capabilities, build new systems as you migrate on. And that's our tech strategy.

[00:13:33.836] And business and technology, we came together, we said that the best way to introduce new technologies, best way to introduce new applications, is to introduce new ways of working. We have to work together to improve the customer experience.

[00:13:50.076] What did we change? We talked about a lot of teams, they talked about here the Spotify model. We talked about projects to product. We also have been through the same journey. Three years before, we started this, and we formed different squads, different tribes, and we had dedicated product owner. The product owner is not from technology. The product owner is from the business. More collaboration. Now business is sitting with technology. They are able to understand the challenges. What kind of problems we have in technology, why they can't make certain changes as we are requesting them to do. And they see and appreciate the progress. And they are not seeing 80% of project report complete or 90% project complete. This is the burndown of our finance, budget, timeline. Nothing like that. They are sitting with the team. They're trying to understand that, oh, this is the complexity because you guys have to deal with multiple systems, getting the customer data from mainframe, getting your loan and security data from a different system. So the challenges are real. So they started appreciating that.

[00:14:55.496] In terms of enabling the technology, the technology team changed. How did we change? Our delivery teams, we are no longer in one team where the pool of resources that we go out and we start working on different initiatives, and we come back. We started forming delivery squads, and these squads are full-stack product and stream teams. They are co-located. They have a single goal, and they are domain-aligned. And this is similar to the projects-to-product approach. The domain, when I say, I'm not talking about domain of anything. This is about pure domain-driven design concepts, and we have bounded context. We have to agree on what is our boundary and what services we are building within them. And this also resonates with the talk from Team Topologies, where they said, reducing the cognitive load, because I am not worrying about what I'm doing with the lending securities, what I'm doing with unsecured lending, because the teams are only focused on that particular problem.

[00:15:52.216] Platform teams become enablers. And upskilling, we run experimentation, training programs, and we have guild and certification assistance.

[00:16:01.816] Now, these are some of the tools overview. Now, I don't have a pipeline view or anything like that because most of us, we are in the CI/CD journey, and we know how the pipeline looks. But this is how we started. We are one of the first in New Zealand to start with the container journey in terms of in a bank. We started with Red Hat OpenShift, and we lead, and we spent a lot of time, and the team have struggled to--the steep learning curve to understand the product, how it works, how to containerize, how to help the developer productivity, developer experience. These all matter.

[00:16:38.056] Sharing our stories. Now, if you're doing in a small team, okay, the team knows that, oh, this is what you're doing. Okay, you're using Jenkins. Your applications are getting deployed fine, and fantastic. Now, how do you make an impact in the rest of the organization? The one way that we did was we used to run internal BNZ DevOpsDays. And I started the momentum in 2016. And I used to invite people, whoever they come over for a conference from States to New Zealand, and some of the industry people from government organizations who are going through the same journey. I invite them. I bring them as a guest speaker. I ask them to share the story. So this is one way that we call now internal tech conferences, right? There is a book on that as well. But this is I started doing 2016.

[00:17:24.876] And we also started running architecture and design for continuous delivery because your DevOps journey and delivering business value is not just about Jenkins and containers. Your system need to be designed, architected, including your database, your communication between your systems, your API contracts, your events. We have to think about it, put a lot of thoughts around it, and then make the system. Because there is no value in delivering your monolith crap using a very fast pipeline. And you need to deliver them in smaller batch sizes, and again, connecting to the cognitive load.

[00:18:01.716] Internal forums, chapter meetings, brown bag sessions, and one of the things that we have done--and I would like to also quote on this--NAB, National Australia Bank, is the second number of highest AWS-certified people working for a bank in the entire world, second to AWS themselves. It means they launched a program called Cloud Guild, and they started training the people on the cloud. The interesting approach on this is actually when everybody is working like that, they all start understanding ubiquitously the same language. They understand the same aspects of it. I'm not talking about Glacier and somebody is talking about a tape drive. No. We have same kind of an understanding what technology is talking about, what the operations team is talking about, or what the businesses they want.

[00:18:51.708] And we haven't done it. Customer feedback, I don't want to read the points, but I would like to give you an example. Think about the banking experience like this. Mom, with a six-month baby in the hand, goes to a supermarket. She buys formula for the baby, walks to the checkout, wants to pay for the money for the formula using a card or whatever the system that they have, POS machine, we call it POS there. Let's say that when you tap your card, if the banking system doesn't work, if that says, "Sorry, we are experiencing some technical trouble. We can't help you." Think about that situation. The baby goes to sleep empty stomach. You don't want that experience for your customers. And everybody, we need to think about that.

[00:19:46.968] Think about ourself. I bank only with one, BNZ, myself. And if your primary bank and you have a card, you're hungry, you're taking your friend for lunch, you're standing in the queue, and when it comes to pay, and you couldn't able to pay using your mobile device or your card, then your situation is really bad. It's not about you going hungry, but the situation that you are in. And this all matters to improve the customer experience. This all matters for technology. I am a technical developer, but I see value when you put smile on the customer face. That matters. Not how many process you have, how many documents you write, how many lines of code you do, how many containers you're running. Those don't matter. What matters at the end of the day is the smile.

[00:20:32.728] Achievements. We have built useful functionality and usable functionality to users. We have done that. And one of the small business customers, it's a real story. One small business customer, 3:00 he woke up, he needs to pay wages for the employees. He logged on to our system. He applied for an overdraft. In 15 minutes, the fund is in his account. And that tells the story itself, how much the team has struggled to bring that value to the business. Because 3:00, the small business owner is in a pressure because he needs to have a fund to pay wages, and you are helping that individual in that time, in that moment, and that matters a lot.

[00:21:14.768] DevOps challenges. Okay, this is interesting thing for me. I want to talk a couple of minutes about this. Risk and security, staying compliant, audits, and RBAC. RBAC, most of you might be knowing, role-based access security. This is the biggest area for us. And I will probably, when I am summarizing, I'll summarize with one slide, which will help you.

[00:21:35.368] And the database challenges: changing the data management team to adapt to modern engineering practices to unlock the data. It's not easy. And we are not yet successful. We are working with them. And the data management team is helping us in achieving that. After 40 years of same naming standards and consistent conventions they used, we were the first, my team were the first to break that, and we introduced we could make more meaningful names. We had six characters, eight characters, 10 characters kind of naming conventions because we came from an old DB2 and the mainframe systems, but we changed that. And not only that, in 150 years of history, last year, we were successfully able to deploy database changes automated way. It's not just application, but also actually the database.

[00:22:32.428] And external dependence is our biggest thing because, if your vendor is not ready, and if your other teams are not cooperating, if other teams they say, like, "Oh, you have to create a ticket for us." I call funnily, it's not TDD. TDD stands for test-driven development. It's not TDD, it is TBD: ticket-based development. If you want somebody to do something, you have to raise a ticket. If you have that culture, probably that is not going to help you. So you need to work with them, facilitate, make them understand how we can collaborate and how we can be successful.

[00:23:07.388] Our learnings. Somebody was talking about establishing a common standards language. That is very critical. If I'm talking A and if you're talking B, it's not going to help us. We need to understand A is for A. Ubiquitous. Problems are not unique. If you think that, oh, in the entire world, we have a unique challenge, that's not the case. Reach out. There are a lot of people who have gone through the same experience. And the one important thing that I keep coming back to this community, DevOps community and DevOps Enterprise Summit, is that it amplifies your learning. You hear those stories. Those are your inspirations. Spread and share, because if you have success to shout out, do that, because that will encourage the team. And don't give up, keep challenging.

[00:23:55.128] To go faster, you need a discipline, and if a Formula 1 driver is on the track, if he needs to go fast, he needs a better control on his system. Okay? It doesn't mean he has a control, he's going slow. This is a quote from Adrian Cockcroft, who's with Amazon at the moment. And you need to have a sensible architecture as well because don't think just having Jenkins, as I said, and containers and cloud is going to make you go faster. No, these are enablers, but your system needs to be designed for doing that.

[00:24:28.516] Other confessions. This is another important thing. If you are embarking on more extensive technology changes at the same time, you will have to pay the price for it, because it will slow you down, because you're introducing a new API gateway, you are changing your enterprise database system, you're introducing a new integration platform, and you are also introducing a new streaming platform. All those changes: don't start to do that. You think about it. I don't have a solution for it, but think about it.

[00:24:59.456] Solving puzzles, not problems. This is important. You might ask, what is the difference? Do you want your team member to go and sit and fiddle with the Jenkins scripts or with the Kubernetes YAML file and do all the glory, add a lot of bells and whistles for that? Or do you want them to sit and solve the problems, what the business is after? Think about it. And please make use of the platform teams. Make use of the different teams who are capable of doing it. Gene was talking about this particular point on this improvement of daily work in one of the ideals. So focus on enablers.

[00:25:37.316] This is very critical for us because if you don't have--this is again, the quote of Satya Nadella this morning that Gene showed in the back. Developer productivity is very important for you. Just introducing containers doesn't help. What do you have to do? You have to give them self-service capabilities. Do they have to use those systems? Those are very critical. Understand, 15,000 people working for developer productivity in a big enterprise, and you are size of 5,000 staff members, 700 people in technology, and you need to have quality investment in helping those developers to deliver the business value.

[00:26:11.316] Help that I'm after. Decentralized governance. How do you guys do it? And if you have answer, I would like to talk more about it. Applying product mindset to platforms. Because, okay, now I have domain teams and stream teams, how we are developing capabilities and introducing new features. How do you do that with the platform teams? Sourcing and procurement. Sourcing and procurement is another area. Probably, I'm expecting another forum paper maybe in a year or two from IT Revolution books, because sourcing and procurement in a highly regulated industry takes months. If you want some capability, and if you evaluate and if you want to buy that today, sorry, forget about it. It takes ages to get there because we have to meet a lot of obligations. No compliance. Waterfall funding, agile funding. And ask this question: are your vendors that you're working with, the products in your suite, are they DevOps ready? Do they well connect with you? Do they have APIs to handshake? How do you do it? If you're introducing new vendor tools and disconnected, it's not going to help.

[00:27:14.896] My sincere advice, I'm not that experienced, but keep learning, keep challenging, and keep fighting. One day you will make the change. And I'm still working with teams since last two, three years in this journey, but we started in 2014, some of these areas. And don't give up. And one day you will make the change and you will cherish the benefits.

[00:27:45.756] Closing thoughts. There's a Maori proverb. The question goes like this: what is the most important thing in the world? The answer is, "He tangata, he tangata, he tangata." It's about the people. If you are doing a transformation, take the people with you in the journey. Don't forget about them. And the important thing that you need to be successful is your own people, and help them out.

[00:28:15.456] And this is a quote from a Sanskrit Rigveda. It says, "Sangachchatvam sambadatvam samvo manamsi janata." It means you all move in harmony. Speak in one voice. Let our minds be in agreement.

[00:28:33.316] With this, thank you very much for the opportunity. And as I started, sincerely appreciate every one of you in this room for foregoing the Disney magic and being in my session. Thank you very much for that.