Shaun Norris

Good morning, everyone. I'm Shaun Norris, and thank you, Gene, for the nice introduction. I lead public cloud database products for JPMorgan Chase.

To introduce myself quickly, I've been a veteran of the IT industry for the last 25-odd years. I started my career in Canada, but quickly moved to the UK earlier in my career. Since then, I've lived in Switzerland, Argentina, South Africa, Singapore, and now I'm back in the UK. So I think I was a digital nomad before it was a thing.

JPMorgan Chase, though, of the companies I've worked for, and those companies include places like AWS and Pivotal, startups like Lastminute.com, is actually the only firm that I've ever returned to. So I'm at my second stint at JPMorgan Chase, and this is my sixth DevOps Enterprise Summit in some capacity, and the fourth time speaking.

You're going to see a nautical theme through this talk. I have imposter syndrome whenever I come into this room, so if you're up after me, you're talking, don't worry. I have the same. But I've actually realized from doing a bit of boat racing that if you're actually at the back of the fleet, it's not imposter syndrome. You're just last.

Through my career, I started out in ops, so I really identify with the system administrator of old. I migrated into software engineering roles, and now I really identify as a technology leader, helping teams build great products.

I live in Lymington in the UK, on the south coast, with my wife, Mags, and two crazy dogs, Odie and Helix.

But enough about me. Let's talk about this boat. The boat you're looking at on the screen here is a boat named Madcap. Now, this is not just any boat. This is what's known as an X One Design, and it's only found in the Solent, which is that famous body of water between the Isle of Wight and mainland England. And if you're familiar, that's where Nelson's fleet was based. It's where the first America's Cup race was held. It wasn't called the America's Cup then, until America won it. And I'm lucky enough to own this boat in that body of water.

But this is not just any X One Design boat. This is actually the oldest one still sailing. It was built in 1911 out of wood, and it's been restored a couple times in its lifetime.

But I thought, as I was putting this talk together, this is a great analogy for heritage technology. Heritage is great for the weekends, but really, when it comes to day-to-day business operation, we often need something more modern.

So this really is a talk about databases. It's not about boats, but I thought the boats made a better background and better photos. And so we're going to have some nautical pictures accompanying this talk about databases.

A couple of reminders. First, while I work for JPMorgan Chase, my opinions here today are my own and don't necessarily reflect my employer. And if I mention any technologies, please don't take that as an endorsement. It is not.

Let's talk about the scale of JPMorgan Chase. Gene introduced it to some extent, but JPMorgan has over 290,000 employees and over 43,000 developers. In 2023, technology spend is estimated to be in the neighborhood of $15.3 billion US. To run that size of estate, we run tens of thousands of database instances. Thousands of application workloads are in the public cloud. And we, as you might imagine, have a large estate of relational and NoSQL databases.

We've made a heavy investment into DevOps practices as a firm. And I think maybe one of the best ways to illustrate that is Chase.com, which you're likely all familiar with as one of the largest online banking platforms in the world. We now release code to that, on average, 15 times per week. And just a few years ago, that would've been unusual.

Now, databases are a growth market. You have one, and suddenly you have them all over the place.

When we think about databases at JPMorgan Chase, the demand for database tech is high, and it's growing on top of an already high base. And unlike this particular day a few weeks ago, where we had no wind, when it comes to databases as a firm, the wind is really in our sails. Production databases across the firm, in the last 12 months, have grown on an instance basis by over 28%.

Now, that's great. It's great to hear me talk about the great technology we're building. But this is from one of our customers. Our customers are internal. We provide technology platforms to application teams building applications that all of you and the rest of our customers use.

This is from Harsh, who's the managing director. He's the global head of engineering for our digital document services. And they did a migration. They migrated to a new modern database platform. They migrated millions of documents as part of this effort. And this was a joint effort between the application team and our database platform team.

The outcome of that was, when they were finished working with us to do this migration, their search performance was 85% better, and transaction update time was 50% better. Harsh here mentions how key the partnership was to getting this done. This was not just a lift and shift onto a new database. And also, it was not purely an application effort. It was a really good example of working together to get to a great outcome.

Now, when we think of where we run our databases, it's really a combination of all three of these areas. If you think of everything below the line as heritage, those would be traditionally built databases, your artisan, handcrafted database, as it were. Whereas everything above the line, we think of as modern.

We're going to get into and talk about modern versus heritage a little bit. And you can see a picture of Madcap on the left and a current-day, state-of-the-art technology of sailing on the right, and they couldn't be more different. And it's actually similar, and I think this is a good analogy for thinking about how we deliver databases to the firm and to the firm's application builders.

It wasn't always the way in the testimonial we heard from Harsh. In the past, if you wanted a new database, it could take months. Well, why? There were a lot of tickets involved, and a lot of different teams had to work together and answer different tickets. And if you didn't fill it in correctly, it might get sent back to you. There were lots of handoffs.

They were bespoke. Everyone was typically manually configured. There would be custom patch versions, custom engine version levels. You'd need a separate database administration team, a whole team of database experts to run these databases for you.

And then even once you got the database, often months later, you needed a separate team to actually do your ongoing operations of that database. And that level of effort, incidentally, also applies to owning a classic boat. There's a very high ratio of working on the boat to actually sailing and enjoying the boat.

Now, when we think about modern database tech as we provision it today, it's very different. When an application team orders a database, typically it's available in minutes. That's done self-service through an API. It will typically have a standard config. It'll have consistent patch and engine versions across the fleet, and it will be self-managed by the application team to a large degree.

And the ongoing operations involved with that database are typically done through self-service, and they're integrated into our CI/CD pipeline tools.

So why does database modernization matter? Well, this is, I think, another serious outcome. This is why it's really not optional for us anymore to get to modern technology platforms and modern database platforms. Installing a database engine is only about 10% of the job of provisioning and getting a database suitable for use within the firm.

Hundreds of controls apply in a highly regulated firm such as ours. These controls used to be largely manual, and now they're increasingly automated. But the metrics we have coming back in this case here, that for teams migrating from one of our most commonly used databases, if they went from our heritage platform to our modern cloud platform, same database engine, mind you, so you're not changing the type of database you're running. You're just changing from the heritage platform to the modern one. You were 82% less likely to have a database-related major incident.

This has been a huge, at-scale success. This is over thousands of instances.

One of the ways that we're looking at it when it comes to helping application teams manage their own technology is really looking at a split between container and content. Just as a boat without a crew is not really much use, a database engine without some data in it, or without some content, is also not much use.

This has been one of the helpful mental models to think about: how do you provision stable, ready, fit-for-purpose technology at scale in a firm as big as ours, and yet make it suitable for use and fit for purpose? So this container-content split is one of the ways we think about that.

When you look at container, well, that's, I have the engine installed, I have the right infrastructure, I have CPU, disk, and network, I have my operating system and engine patches. I have replication and resilience and failover set up. And hopefully, those are all done through automation with API-driven control planes.

And then when it comes to content, well, that's my schema and the data in that schema. How are users authenticating and authorizing? Are those apps authorizing? Are they human users, SRE, et cetera? Are you publishing your schema to a catalog? Are you enforcing all the relevant controls?

But we then typically like to provide standard administration capabilities through curated self-service, rather than just giving people the ability to log on and do whatever they like.

So it's worth now a look at controls. You might find this a bit of an odd photo, but I chose this one not because of the massive ropes and the cam clutches and the cleats here, but if you look carefully, like on the red rope to the right, there's a little yellow piece of twine. And what we do when racing is that's actually a baseline marker.

So once we figure out how to make the boat go fast, and full confession, I haven't quite figured out how to do that yet, but when we do, you want to mark the rope so you can get back to those settings.

So I thought this is a great visual analogy of what we try and do, particularly with controls. We want to try and make sure that the systems are well built and in control.

Some of the controls we have to look after over and above just the basic database are things like backup reporting. Are backups happening? Are they on the right schedule? Have you tested them? Privileged access management: who logged into a database, with what privileges? If those were elevated, what did they do? And was it appropriate?

Security drift monitoring: was the database set up with the right security baselines? And has it drifted over time away from that? And if so, who's putting it back?

Things like SOC 1, PCI compliance, highly confidential data certification, resilience testing. The list goes on.

Running databases at scale is not without its challenges. One of the challenges of sailing an old wooden boat is it has no flotation in it. So my friend Simon here on the left, the angle he's at, if he was another 10 degrees over, chances are he would've sunk because he would've taken on too much water into the open cockpit. So this is a little scarier than it looks. This was earlier this year at Cowes Week.

But when it comes to running databases, particularly when you put public cloud into the mix, cloud is of great benefit, but it's also not without its challenges. How do you get the right balance for app teams between just refactoring or replatforming? To what extent should I have to think about reworking how my app works when I want to go to the cloud?

Because in a lot of cases, or in some cases, the technology that teams are comfortable with and have been using for years on premises is actually not available in the public cloud.

Migrations as well are a big challenge. How do you migrate from when you have 10 or more different source databases and a small set of target databases? How do you make sure anyone can migrate from any to any safely, repeatedly, with rollback, et cetera?

Public cloud migration has been not without its challenges, but we are moving quickly in that direction.

So let's talk a little bit about, then, we've talked about migration. Now let's talk about new applications. We also want to take a longer-horizon view. If you're coming as a new application builder, what sort of things should you be considering when you choose what is the right data store for your application?

So we put together this, and we've kind of put it in the form of the Agile Manifesto. I won't read the whole thing, but to go through this, we don't want this to be binary, that you should only ever use open source and never use commercial databases, for example.

When it comes to open source, we are bullish on open source. We think it's really important as a firm. We already run thousands of open source database instances. But on the balance side, commercial databases are still important to us. We will still run and support large numbers of those into the future.

And then convenience versus portability, that's another area. Cross-cloud. Some folks like to chuckle about multi-cloud in our industry. It's a reality. Cross-cloud, in my opinion, is going to grow in importance. And so if you have coded your application to be tightly embedded and will only work in one cloud, if you, for whatever reason, need to migrate away, how are you going to do that? What level of difficulty and what timescale is that going to be on?

One of the mitigations that we are looking at, and we're quite bullish on, is the idea of using Postgres compatibility as a mitigation.

And other considerations. If you code heavily against the identity and access management substrate of one cloud, and then you need to move to another, how much work are you going to need to redo? If all of your developers are skilled up and only know the specific skills that work on one cloud, how portable is that going to be if you decide to onboard to another?

These are all open questions still.

So let's move on and wrap things up and look at, well, what's worked so far in our environment? First of all, a contribution model. When we're building a cloud platform and the databases on that cloud platform, often there are specific capabilities that a certain team will want but are not coming soon enough to meet their migration needs. And we're rapidly embracing a model that allows those teams to come work with us and contribute features more quickly than they would be available otherwise.

We have successfully invested in a firm-wide CI/CD toolchain. So whatever line of business you're in, whatever development team you work in, it's a consistent set of tools so that you can build tasks, deploy your code with confidence.

Visible work. I want to give a shout-out to Dominica DeGrandis, who I met briefly earlier. We've successfully made a lot of our work visible. We have one central clearinghouse for raising a cloud feature request. If you need something to migrate your application, we've successfully got that into a one-stop shop where you can raise that and ask for it. So all those things have worked for us so far.

If we talk about where we started, well, Python. We had a lot of database administrators who were keen to retrain as software engineers. So we've used Python as a training vehicle internally with a lot of success. We have trained hundreds of DBAs and retrained them as cloud and software engineers.

And cloud skills training, we've doubled down on. We've focused on both open source certifications like CKAD, as well as cloud-specific public cloud certifications. And we've done that at scale, with thousands of certifications across the firm.

And if we look at help we're looking for, well, there's a need for more community and industry standards. I mentioned things like security configuration baselines. Well, those are bespoke and firm-specific today. I believe there's a need to come together as an industry and actually open source some of these ideas, and actually collaborate for greater cross-collaboration across the industry.

And there's a need to shift compliance further left. We've started, and we need to go further. We need to get compliance built into our products and our applications right up front as part of it. And we need to go more in that direction.

And then the last thing we need is really feedback. If you've enjoyed this, if it's sparked any conversation ideas, if you want to chat more, you can engage with me on LinkedIn. You can chat with me at the conference.

And lastly, as we wrap this up, it's time to drop our sails on this talk and to take the boat and the talk back to the mooring. But before I go, a few words of thanks. First of all, to Gene and the organizers. Thanks for organizing an amazing conference.

To Emily and the rest of the media relations team at JPMorgan Chase, it's a huge effort to be able to come and share at a public event like this. So they did a fantastic job in helping me get here and share this with you.

To my wife, Mags, for taking most of the photos. And most of all, to all of you for making this an amazing community where I learn a ton every year.

Thank you very much.