Log in to watch

Log in or create a free account to watch this video.

Log in
Las Vegas 2024
Share
Download slides

Practical Modern Enterprise Architecture: Modernising Architecture & Ways of Working at Saxo Bank (SAXO BANK)

Simon Rohrer takes you on a whistle-stop tour of all that we’ve achieved in the last 5 years at Saxo Bank combining multiple modern ways of working under a hands-on Modern Enterprise Architecture approach for sociotechnical transformation:- Using cloud class technologies - Docker, Kubernetes, serverless, and more - as a lever to drive autonomy and alignment around a Team Topologies and BVSSH model- Introducing event-driven architectures and an operational and analytical Data Mesh on Kafka as a foundation for a data-driven culture, an improvement in resilience, and a platform for Generative AI and machine learning- Autonomation everywhere - implementing build and deployment pipelines in Azure DevOps inspired by Tapabrata Pal’s CapitalOne pipeline model and “Better Governance – Banking on Continuous Delivery” talk at DOES18 in London; automating Change Advisory Board and Change Requests; moving deliveries to production from once every two weeks to multiple times a day.- Communication structures - using chat infrastructure like Microsoft Teams as an information radiator across the entire technology organisation- Know Your Architecture - using ServiceNow, the Configuration Management Database and Common Service Data Model, and Spotify’s Backstage, as well as observability tools like Elastic and Grafana, as a foundation for governance, understanding, and automation- Authoring regulator-required in-house formal technology and ways-of-working policies and procedures and using them as a driver for modernisationAs a contributor to Jonathan Smart’s Sooner Safer Happier, and with 30 years of experience across Software Engineering, Enterprise Architecture and Ways of Working Transformation, Simon has brought all of his experience to bear on improving customer and colleague experience, increasing speed to market, and reducing cost and complexity in Saxo Bank - and learned much more from his colleagues and experiences along the way.

Chapters

Full transcript

The complete talk, organized by section.

Simon Rohrer

Good afternoon folks. It is a minute past, and I have a million slides to get through in 30 minutes, so I apologize if I make a start right now.

I'm going to talk about practical modern enterprise architecture. I've given a version of this talk before that is a lot more theoretical, but I'm going to talk about what we've been doing for the last few years at Saxo Bank in the enterprise architecture space. Thanks so much to Rashmi for the last talk — I recognized a lot there. For those who are here for Rashmi's talk, I'm going to say some of the same things in a slightly different order, and some other things too.

Saxo Bank — who we are

Saxo Bank. We are a Danish company that offers an investment and trading platform to our direct clients and to third parties as white labels. We're 32 years old now, with 25 years of experience in FinTech. We have a lot of legacy, a lot of phases that we've gone through, but we're now in a phase of constant improvement. We have a purpose: we get curious people invested in the world.

What do we do? Like I said, we are a facilitator for markets — to people, traders, investments, and wholesale direct and institutional. We have one tech stack. That's it. In terms of numbers, we have about 800 billion Danish Kroner — about $120 billion US — under our platform, and 1.2 million clients. Not quite as many as Nubank as was being talked about before, but still quite a few.

We have that tech stack. This is our architecture on a page. We have about 1,200 configuration items in our configuration management database — that's mostly homegrown software. That's quite a lot.

Our organization

We have gone on an organizational journey. We have broken down what used to be silos of technology and business — various departments — into what we call client journeys. This is our name for value streams. We have 10 different value streams and a few sort of cross-client-journey streams, but mainly we try and organize into those 10 value streams. That's about 2,500 people across the business, and 900 in tech. We don't have a separate tech department — all of the tech teams are in those value streams, or cross them.

We have a tech strategy. It's there to support our business strategy of continuous growth. We have a few pillars there that I'm going to talk about. I'm going to mainly focus on architecture and engineering modernization — what we've been doing there and how we as enterprise architects have facilitated that.

A little about my 30-year journey

I did an intro to myself at the top: I've been in technology for 30 years now. Started as a developer. I've been an enterprise architect, working with Jon Smart at Barclays. I started off as an agile transformation lead. For those who have heard Jon's journey and our journey at Barclays — well, we started off doing agile transformation. We then decided it wasn't transformation, it was adoption. Then we decided it wasn't just agile, it was ways of working.

Then I was asked to contribute to Sooner Safer Happier, Jon's book, as well. I wrote the more techie chapter. And then Brexit happened in the UK, and I decided to escape the UK and head to Denmark, which is a slightly more sane country — only slightly more. I'm now Head of Enterprise Technology Architecture and Ways of Working at Saxo Bank.

Over those 30 years that I've been working in software development, the main thing that has affected me is this big paradigm shift from what Dan North calls a process that was based on the civil engineering metaphor. Basically: you plan something for several months, then you code it for several months, then test it for several months, then spend a few weeks waiting to release it, and then somebody has to operate it forever. We learned a lot in the last 30 years — agile, DevOps, continuous delivery, cloud, a whole bunch of things helped us get to a very different way of working. Now things that used to take months or years take days or hours, and we do things in cycles as opposed to just going left to right. Instead of working in silos, we hopefully have a product team — a team that develops and operates IT products and business products.

Our problems now, in the 2020s, are mainly around complexity. Some of those project management techniques seem to work their way back in because of coordination. I'll talk a lot more about that.

Modern Enterprise — and the ABCDE framework

So I'm talking about modern enterprise architecture. What do I mean when I talk about a modern enterprise? Teams of teams of teams, typically more than 150 employees. You interact with your customers digitally. Change is a constant. It's a heterogeneous environment — you've got older and newer systems, slow and fast, monolith and distributed, vendors and internal. Systems of systems of systems. That's certainly where we are at Saxo Bank.

I'm going to talk about the A, B, C, D, E of modern enterprise architecture, where: - A is aligning value, people, and technology - B is better value, sooner, safer, happier — architecting for outcomes - C is continuous governance - D is DevOps at enterprise scale - E is evolutionary architecture

A — Aligning People, Value and Technology

So aligning people, value and technology first. Conway's Law. (Sorry, I'm going to really try and get this clicker working… this clicker is a little weird. It's doing two clicks at once. Techie blaming the equipment — yes, indeed. Integration problem. It's always an integration problem. Maybe a time when two steps forward, one back. Still doing two clicks at a time. I am going to have to use my laptop. Jon, can you do that? You did this for Dan North at one point — a human clicker in place. We've at least got there.)

Rashmi talked about Conway's Law. I'll go a bit more about this. Very roughly, Conway's Law says organizational architecture — the communication structure in an organization — drives system architecture.

Ruth Malan puts this well: "If the architecture of the system and the architecture of the organization are at odds, the architecture of the organization is going to win."

Allen Kelly says sort of the opposite — it's a great YouTube talk. He says: where all hell breaks loose is when management tries to reorganize things. If you try and change the organization, the software won't let it happen. So now system architecture is constraining the organizational architecture. How does that work?

Well, obviously it's a feedback loop. Unsurprisingly, for those of us who like systems thinking, there are constraints that happen both ways, and we've got to work with both of them.

Ruth Malan again says: look, systems architects — who we call architects — and business and organization architects — who we call managers — shouldn't work as if one has no impact on the other. So we've got to work together.

Business architecture as well, of course. What we now sort of call value streams often drives organizational architecture. And in typical organization architectures, it's the organization architects (a.k.a. the managers) who are doing that architecting.

But actually what we've got is this triplet of systems, organizations, and business architecture, where there are constraints in all of these. You have to work deliberately and consciously to try and align all three. And if you're working in opposition, that's not going to happen.

Fractal alignment — team / team-of-teams / team-of-team-of-teams

For sustainable flow, we have to align all of these things, and we have to deliberately and consciously align value, organization, and technology and processes at all levels — fractally, which I'll talk about.

Level zero fractally is what Team Topologies is talking about. Martin Fowler's article on Team Topologies is brilliant. He talks about effectively the full enchilada. This is an ideal two-pizza, two-pizza-sized, eight-to-12 (maybe a little more) team, and they own some value. Maybe less than 20, depends who you're talking to. Dan North talks about software that fits inside your head, and Team Topologies extends that to software that fits inside the team's head. You're going around that entire loop, and you own probably some data, some business, and some front end. You're communicating with other teams and other systems. Cool.

But then you've got to think up a level as well. No software is an island. Certainly at modern enterprise scale, you're never going to be totally uncoupled. You are loosely coupled, you're lightly coupled, but there is always some degree of coupling to deliver higher degrees of value.

Next level up — an ideal team of teams. What's going on here? You are also delivering value. You're delivering next-level KPIs and OKRs here. You've got a team of teams. We talk about the Dunbar number here — the sort of number of people that can know each other roughly and work together. It's a system of systems here. The domain here really needs to fit inside the tribe's head, or the value stream's head, or this team of teams' head. They need to know what they're working on. Nested domain-driven design here, typically some kind of shared or overlapping data model, probably eventually consistent.

Next level — an ideal business line. Again, this sort of fractal nature. It's a team of team of teams. This is certainly where Saxo Bank is. It's a system of system of systems. Back to the Dan North metaphor though — does this fit in anybody's head? Can anybody hold this entire set of systems and value and team structure in their head? Very doubtful. So you have to use other techniques here.

Saxo's structure — 89 product teams

This is us. This is Saxo. We have 89 product teams, roughly 900 technologists. 55 of those teams are aligned to client journeys, and 34 are cross-journey — they do platform engineering, things like finance systems, HR systems, email systems.

Gene and Steve in Wiring the Winning Organization talk about simplification through modularity. That is very much what we're trying to follow here. It is modularizing all of those three levels. We talked about the team owning modules of value and systems. The team of teams owns a module at the next level. And ultimately we are delivering just one system to our clients — that's a module as well.

This is also really interesting: as I was reading Steve's book, I've been reading recently about cybernetics. Dan Davies, who has recently written a book on cybernetics, says "carcinization" is the tendency of many animals to evolve toward the form of crabs. Dan Davies says that a lot of writing about control theory and management eventually evolves into writing about cybernetics. I think a lot of what Gene and Steve are writing about in Wiring the Winning Organization has a lot of overlap with cybernetics.

This is control theory. It's Stafford Beer's Viable Systems Model, and Ashby's Law of Requisite Variety sits under this. "Only variety can absorb variety." And when I talk about that fractal model — not every level of detail can fit in everybody's head. Certainly not when you've got a thousand people and 1,200 components. But you've got to attenuate at various levels, and above those levels — at those different fractal levels — there is only so much you can absorb, but it's the right amount. Awesome book on that is called The Fractal Organization.

At Saxo Bank, this is what we have implemented reasonably consciously. We have that alignment of those value streams — the client journeys. They're directly aligned to 55 of those teams, who in turn own individual modules within our system of systems of systems.

B — Better Value Sooner Safer Happier (architecting for outcomes)

So: better value, sooner, safer, and happier — architecting for outcomes. Jon, you know this slide well. Better is quality. Sooner is flow. Safer is "agile, not fragile," where we're architecting for security, privacy, and minimum viable compliance. Happier — absolutely important. Happier customers, colleagues, citizens, and climate. All wrapped around value, which is the thing that makes you unique.

I'm going to talk mostly about flow here. A quote from Winston Churchill: when the UK Houses of Parliament were bombed in the Second World War, Churchill was asked whether he wanted to re-architect the Houses of Parliament to be more like European parliaments, where it's consensual and in a U shape where people sit together. And he said no — I like the adversarial British politics. I want this rebuilt as it is. He said: "We shape our buildings, and they shape us." I like that adversarial form of politics, and I like the building that helps that happen.

Gene Kim has quoted that, saying: look, we shape our architecture — our technical architecture, our systems, our socio-technical architecture — and then the architecture shapes us. As the systems we build become larger, the coordination increases, and it can grow so large that all of our time and energy is spent coordinating at the expense of creating value.

This is very similar to some of the stuff that Rashmi was just talking about. Look — we can have teams that look like this, or teams of teams where you've got a UI team, an API-layer team, a couple of service teams, and then a back-end team. In order to coordinate that, you need people outside the team to be able to coordinate even solutions to that. And then the tests afterwards. You've got something that comes out roughly looking like that idea, "Hello, world. Hello world. Close enough. Not a showstopper defect. We'll release that one." But that's not ideal for flow. Those coordination activities are not good.

Quote from Gregor (who I see in the middle of the room here — somebody next door is talking about exactly the same theme, there's so much overlap going on): layering is sometimes considered harmful, because it can reduce flow.

An interesting one — great quote from Johanna Rothman: release trains are iterative and incremental, but ultimately they're not agile. They're not agile at team level, because you are coupling teams together.

So, as Rashmi was saying, we can refactor — or at least restructure — organizations and architecture. That's the two things at once: the socio-technical refactoring. We have to say, "Hey, team — this may be slightly higher cognitive overhead, but for the organization's flow, which is important, you know all of these technologies and you can make all of the architecture and test this all yourselves, and you can flow at a much greater pace with much less coordination."

Two years ago, Scott Prugh did a great mathematical (or accounting) proof of this that said: this isn't just about flow — ultimately this reduces your costs as well, and increases your chances of success.

For us, how are we doing? We've got 55 client-journey-aligned teams. We have some that we've managed to refactor to full stack. Some are still work in progress. But we have written in our IT strategy — signed off by our board — that we do want to work towards consolidating full-stack teams aligned to those journeys.

C — Continuous Governance: conversational and automated

C is about continuous governance, conversational and automated. This is all about governing complexity.

Architecture governance to the rescue, maybe — but an anti-pattern. After that paradigm shift, the anti-pattern we've seen is architecture review boards. We used to govern architecture like this in that old civil-engineering-style world, where we used to do an initiate phase and then a plan phase and then execute and close — which aligned to the waterfall steps of analysis, design, build and test, and release. We used to have an architecture review board. That happened maybe twice. In between those two steps, we could review something on paper — review a design.

How does this work even if you're working in project mode with agile? You've got analysis starting early but continuing all the way through the test, design, and build phase and multiple releases. Where does the architecture board review a design or a proposed architecture and sign it off and rubber-stamp it? It probably can't. Certainly when you're in product mode, this is just really not feasible.

Black-box architecture governance. What we've been implementing the last few years is black-box architecture governance. We govern realized architectures, not designs on paper. So: "I think I need a new service. This adds potential complexity to the landscape. You might need it, you might not — you might be able to put this functionality in another service." So in EA we say: "Great, you can have a new service, but tell us why. Tell us what it's going to do." Literally you cannot get a new service in the landscape without that. Connections — connections add complexity. "I think my service needs to access data." We approve this. Ideally, we're starting to look at, at least for some patterns, asynchronous by default. But you might have a good reason to do synchronous. Again, we will approve that, or we'll discuss it.

Enterprise architecture as GitOps

We're trying to move towards enterprise architecture as GitOps. Eduardo da Silva, following a little tweet by Nat Pryce, has written this beautiful article about the difference between informational and aspirational architecture. Informational architecture tells you how your architecture is today. Aspirational is where you want it to be. Of course, it's in this feedback loop.

How have we done that? Our informational architecture is in a combination of our slightly more old-school configuration management database and ServiceNow. But then we're moving towards Spotify's Backstage and using the service catalog to keep our architecture as a file — as infrastructure-as-code, architecture-as-code. For the aspirational, for the long term, we just look at sketches, we talk to people, and we have this conversation.

But if you want to change the as-is architecture, you submit a pull request on the service catalog. You say, "I want a new API, I want a new service." The right people review it — that's typically enterprise architecture, but also we've got security if that's needed. And then that's used to automate access control, identities, Kubernetes provisioning, and soon even firewalls will come out of this architecture-as-code.

Enterprise architecture inside platform strategy

I see Charles in the audience here as well — sorry for quoting everybody in the audience today. Superb article from just a few weeks ago from Charles [Betz] that says: enterprise architecture used to be high-handed and bureaucratic, and wouldn't support rapid innovation. And that, honestly, is that sort of enterprise architecture — still thinks things in the old civil-engineering way, or not. But in a platform strategy that's properly executed, these checks that enterprise architecture and other stakeholders like security need to take place — they've happened already. They're in the platform, and anything you build on it is compliant.

This is a compromise. It's a trade-off, as with many architectural things. The stakeholders have to accept a level of standardization. That's typically dev teams needing to say, "Well, look — I'd like to use this, but because actually I'm paid to deliver business and not experiment with weird technologies, I'm happy with that, because I can get through my compliance and governance requirements in seconds rather than weeks."

For us, this means that we have regulations like the upcoming (unfortunately also called) DORA — the Digital Operational Resilience Act, confusingly named in the EU for banks. But we now have governance and compliance baked into our platform. This is a slide that we showed our regulators when they came in to talk to us a few weeks ago. This is how we work with our enterprise foundation teams, including us in enterprise architecture, our DevOps platforms teams, and our security architects. We author the standards. We design controls. This is inspired by Topo Pal's presentation, I think about six or seven years back, about building pipelines for governance. We collaborate strongly with our security architects for this as well. Ultimately that's governed by our technology committee at the top of that fractal organization, who have the final say.

D — DevOps at Enterprise Scale

D is DevOps at enterprise scale. This is the continuous, conversational governance around the DevOps loop. We have a small architecture center of excellence. We have federated architects out in the teams. But we have collaborators as well who also work at enterprise scale. We work super closely and we align the direction of service management, security, and our team of SREs.

This is what I've been working with our delivery teams. We have delivery team and development town halls as well, which we in EA collaborate with.

This is the old world. This is pre-DevOps, when the developer used to work in the top-left quadrant only, and they had a happy world because someone else was doing stuff for them. Then the first iteration of "you build it, you run it" happened — and they got thrown in at the deep end in all four quadrants of application, infrastructure, build, and run. This is too much. It's too much from a compliance perspective. It's too much from a cognitive overload perspective.

So where are we at? We've got platforms — and the platform strategy we talked about doesn't just help us with compliance, it helps us with cognitive load as well. The line is not exactly in the middle. The developers have to know a little about what they're running on, because some of that is going to be very specific to their own business problems. I had a great chat with Justin from John Deere in the break about how hard it is to work out where that line goes — but it's not in the middle. It's a little below it.

E — Evolutionary Enterprise Architecture and Organization

E, finally — evolutionary enterprise architecture and organization. Evolving both of them. I'm going to reiterate some of the stuff that Rashmi said last time.

Of course this is inspired by Neil [Ford], Rebecca [Parsons], Patrick [Kua], and Pramod [Sadalage]'s awesome book Building Evolutionary Architectures. What that focuses mainly on, though, is evolutionary architecture and fitness functions at a single-piece-of-software level. It is harder — and there's a great Thoughtworks podcast where Neal Ford talks about this — much harder in a distributed system to start to monitor and govern the metrics of distributed systems.

What we talk about in Sooner Safer Happier is punctuated gradualism of evolution. There are periods of time where it's just continuous improvement, but then periods of time where it is a big bang as well. We've been doing a lot of this. Continuous attention to complexity has to happen all the time. That is a continuous evolution, and that gets continuous funding. Occasionally you're going to want this sort of punctuated evolution where something large happens — but it's got to be both.

Back to Martin Fowler, of course, who has written up about the strangler fig pattern. This is something we've been using a lot to evolve our architectures both continuously and in a punctuated way.

Saxo's renovation progress

We have about 30% of our services modernized, renovated as Rashmi would have put it, over four years. We've been focusing on the ones that give most value. We do have 100% of our services on those pipelines that are compliance — we have to.

So: modern enterprise architecture — A, B, C, D, E. And a little bit of help, if anybody can help out, because this is the theme.

Closing ask — help us decompose a 4,000-table front-office database

One of the things we have been having problems with modernizing is our huge monolithic databases. We have our old front-office database that has about 7,000 stored procedures and about 4,000 tables. It's the last thing that is not in source-code control. But getting it into source-code control seems almost impossible — because how do we do that before we decompose it? But how do we decompose it before we get it into source-code control? If anybody has any experience in this world, please reach out. Thank you so much.