Fate of the Red Delicious Apple - An approach Discover took for one critical shared service
There’s a reason why the Red Delicious apple is no longer the beloved fruit it once was. What does the fruit have to do with one of Discover’s critical shared services? We’ll share how our shift into the product operating model helped place a spotlight on the core of Discover Card so that we can avoid the same fate.
Chapters
Full transcript
The complete talk, organized by section.
George Kraniotis
Good morning. Good morning.
The Red Delicious apple: alluring to some, undesirable to most. Something that, if you got during lunch, you often tried to trade for maybe a snack pack or something like that.
In the 1870s, an Iowa farmer discovered a seedling in his orchard, and shortly thereafter, the Red Delicious apple was born. By the 1920s, growers began to look for ways to pick the fruit earlier and store it longer without compromising its red color. By the 1940s, it became the most popular apple in the United States.
But as each decade passed, consumers became less satisfied with the tough, bitter skin and mushy flesh. Beauty was prioritized over taste. Producers lost sight of what consumers actually cared about. And by the year 2000, over $200 million in sales were lost.
I'll get back to the Red Delicious apple in a second.
Let's talk about Discover Financial Services. Our vision is to be the leading digital bank and payments partner. People in the United States typically think about Discover, and they think about the Discover Card, the credit card.
We're actually much more than that. In addition to the card, we have a growing deposits business, having savings, debit accounts, checking accounts, personal and student loans, as well as a proprietary network that drives every single Discover Card transaction, debit transactions across the United States, as well as global transactions with our Diners Club International business, all supported by about 20,000 employees at Discover.
Now, Erin and I come from the card side of the business, specifically the card portfolio organization.
Last year I presented this slide to really share our story around project to product, but it's a logical grouping of product families. The orange boxes represent all the different customer journey families that we have oriented around the customer, from setting up their accounts, to maybe doing a balance transfer at some point in time in their lifecycle, to maybe adding their Discover Card to that new Apple or Google device that they got.
But supporting all of those journeys is the portfolio enablement product family. That's what Erin and I have the privilege to support.
So about us: Erin is on the product side. I'm on the engineering side, and over 30 years of experience. Today we support about 20 product teams in portfolio enablement doing a whole bunch of different stuff, from actually creating the card that folks get in the mail, to the statement that they get digitally or physically, because people still do that. They get statements in the mail.
To what we're going to talk about today: card posting and billing, CPB for short.
CPB is a critical system for Discover Card. It's responsible for things like posting transactions, calculating interest and fees. We are the source of truth, the system of record for Discover Card, and there are dozens of other capabilities that would be too long to list on a slide, but just want to give you a little bit of a taste.
Over 100 engineers support CPB, close to 6,000 batch jobs, and well over 60 components. To put it as Steve Smith did on Tuesday, not just a zoo of APIs, but it's a zoo of multiple systems ranging from things on mainframe all the way into the cloud and everything in between. That's what makes up CPB.
And the last major change in the architecture was in the year 2000. So put that in perspective. It's becoming more and more expensive to support.
Back to the Red Delicious apple. What in the world does it have to do with Discover? Well, what's made Discover successful up until now isn't going to get us where we want to be in the future. We can easily become the Red Delicious apple.
What Erin and I are here with you today is to share how we're working to avoid a similar path of demise for CPB and for Discover.
So what challenges did our teams face? A lot of them, but we had to distill it down.
Two of them are interrelated. It's a lot easier to add business logic in a big zoo than it is to try to clean things up. And a lot of the knowledge is centralized. When you have systems from mainframe to cloud and everything in between, you have a lot of diverse skill sets. I haven't found a unicorn engineer that can do COBOL in the morning and then in the afternoon go and push stuff to the cloud.
And so as part of that, the knowledge has become centralized over the years, over the past few decades. This is really the core of our first two problems.
The second is around unclear business ownership. Business ownership, again, was a theme last year when I presented. Same thing. CPB supports the entire card organization, and in some cases we know who cares the most about a capability. But in most cases, we may have an idea of an area, or maybe a couple people, or we may not know at all. And so this is something that we knew we had to address.
The hypothesis that we took on as part of this is, how do we solve these problems? If we treat CPB like a product, we think we'd be in a good spot in the long run. Easier said than done, but I'll turn it over to Erin to talk about our journey.
Erin Daugherty
All right. Thank you, George.
As mentioned, if we were going to truly begin treating this shared service like a product, that was a paradigm shift that wasn't going to happen overnight. It was going to take a mindset that needed to grow and to mature over time, but we had to start somewhere.
The first step in our journey was to stop treating CPB like a back-end platform alone. We needed to establish that clear business ownership. And so we named a product owner, and she, with a jumbo team of over 30 engineers, began to understand their work, their customers, groom together, make prioritization decisions, and create a roadmap.
I say a roadmap because it wasn't really a strategic roadmap as much as it was just a backlog of work as received by internal customers.
What did emerge was how complex CPB is as a platform. And so our second iteration was to embark on a three-month event storming exercise. We spent 90 days bringing in individuals from the business, from architecture, engineers, and other subject matter experts in order to do a 90-day technical deep dive so that we could truly understand this platform.
We identified capabilities in the dozens, and the aha moment was we identified capabilities that some of our most tenured and senior engineers did not even realize were bolted on to this critical platform.
Once we had all of our capabilities mapped out, we then began to leverage bounded contexts to group those capabilities into customer-driven domains.
And this is what the outcome of that exercise looked like: 13 distinct, unique domains comprised of 65 capabilities. And it became apparent one team could not possibly handle all of that work, all that variety, and all of the cognitive load.
And so version 2.1, we had to start somewhere. We needed to start forming differentiated product teams. And so to stop operating like a feature factory, we needed to rethink our entire operating model.
That started by understanding our internal customers and understanding where we were the bottleneck to bringing new features to market. We really leaned into our existing work, identifying where those pain points and issues were already happening today, and began carving out persistent domain teams.
And so we essentially conducted a test, a proof of concept of sorts, to validate our hypothesis. Historically, CPB had operated as a single team that would swarm around work. Our hypothesis was if we could protect resources and they could focus on a specific domain, that could unlock business value faster.
And so here you see APR Management and Delinquency were those first two domain teams. Their mission was simplification. To begin pulling away from the monolith, we wanted them to focus on driving toward true objectives and key results that could then shape a strategic roadmap.
The result that George and I were looking for were truly empowered product teams.
So where are we today? While we have made tremendous progress, we have definitely not yet arrived. We continue to evolve and improve as we learn more. Today, we have scaled to eight dedicated, persistent product teams. However, we acknowledge we still may not be right-sized, and that's okay. We'll continue to make changes as we identify them.
For example, we acknowledge that there was a need for a shared platform team in this space. Similarly, we continue to lean into knowledge sharing and cross-training. We're trying to propagate knowledge so that we eliminate those silos and single points of failure.
When the business can name an engineer by name as the person they go to to fix a problem, that is a problem not just for George and I, but for our entire card organization.
And so the message from the beginning has been consistent: progress over perfection. If we're going to truly avoid the fate of the Red Delicious apple, that's not going to simply happen with one change. We will make mistakes, but we will learn and we will grow.
And so what does success look like? For us, success looks like dedicated teams that are knowledgeable and truly experts in their domain. Those teams, in turn, have to be persistently funded. They can't be surged on a project-by-project basis. We need that stability where knowledge and experience is retained.
And in a large organization, we need to address common needs across our CPB teams and solve problems once. And lastly, we're driving towards transparency so we don't continue to operate like a black box for our customers or for ourselves. One great example is driving towards human-readable business logic that everyone can see.
And so there have been many wins along the way, some big and some small, but two proud moments that we wanted to share.
First is that we are finally positioned as a strategic priority for the business. We are no longer looked at as a back-end technology only, a necessary evil that you have to go through if you want to deliver an initiative, but as an intentional center of investment for the organization.
And secondly, while we never want to introduce defects into production, issues happen. And in the past, that would have not only taken us days, but more likely weeks or even months to correct. And with one of our first two domain teams, we did identify an issue, but the team was able to fix it, test it, and release it into production in three days.
And so to recap: highlight the burning platform and agree across technology and business that there's a problem to be solved, and treat that burning platform like a product. Enabling products drive tremendous value for the organization, so don't treat it like a feature factory. Be intentional about driving product strategy that balances the needs of your customers internally with the needs of your product itself.
Map it out. If you're in a highly regulated industry like ours, you need to remove risk by formally documenting end-to-end value streams, including people, processes, and technology.
Start small and iterate, and know it is a journey.
And lastly, prioritize knowledge sharing. Share knowledge generously and explicitly remove silos and single points of failure.
So where do we need your help? If you have had success developing objectives and key results from platform teams, we would love to hear from you. I think this is an area that we still struggle with, as our platform product owners and product teams don't necessarily see themselves the same way as a traditional customer-facing journey does.
And secondly, if you have found great resources to help upskill product engineering teams about owning shared services, please reach out.
And so thank you for joining us this morning, bright and early on day three.
I also wanted to say a thank you to George. I truly believe that technology and business are two sides of the same coin. And whether it is a challenge or an opportunity, we best solution in partnership.
So thank you, and enjoy day three of the conference.