Log in to watch

Log in or create a free account to watch this video.

Log in
Las Vegas 2023
Share
Download slides

Optionality & the Science of Happy Accidents

As a "sneak preview" of our upcoming IT Revolution book Unbundling the Enterprise - APIs, Optionality & the Science of Happy Accidents, this session will unpack and explain how optionality works and lay out the path for business teams, app-dev and operations can align to create the conditions for big value events (AKA "happy accidents") to occur in their organizations.

Chapters

Full transcript

The complete talk, organized by section.

Stephen Fishman

Welcome, everyone. Thank you so much for attending our talk.

We're really excited to be at the DevOps Enterprise Summit in 2023. My name is Stephen Fishman. I'm the Global Practice Lead for MuleSoft's Success Architecture, which is part of Salesforce. That's an integration platform as a service.

Matt and I have been working on a book for IT Revolution for the last year, and this is the first time we're here to give a little bit of a sneak preview about our subject. The subtitle of our book is the subject of today's talk: APIs, optionality, and the science of happy accidents.

Matthew McLarty

As Stephen said, my name is Matt McLarty. I'm the Chief Technology Officer for Boomi. Steve and I have been in the industry for quite a while, and in the industry over the years, through the many eras that we've lived through, we've been observing the patterns of success in the digital economy.

That could be patterns based on web pioneers right up to established enterprises. And that's why, working together, we felt it was pretty relevant to bring those findings in the form of a book to the public. And as Stephen said, this is our first time on stage doing that.

We're not going to have a chance to go into the depths of the book. We're not going to be able to plumb all the depths of what we found. But we do want to share with you what we're calling this science of happy accidents.

So let's set sail. I know we're in the middle of the desert, so embrace this flowing water here.

In the digital seas, there are opportunities everywhere. Probably everyone is hyped on generative AI. I think everyone came to this DevOps conference, and there's been a lot of discussion around AI. There's all this excitement that's out there, but there's all sorts of digital opportunities that are out there. Everyone's trying to figure out, how can I capitalize on that opportunity? How can I find digital treasure with these new things?

So there's buried treasure all over our digital island, but maybe the most valuable treasure of all is the treasure we don't even know about today. The stuff where it may not even exist today, because it's going to be built on the innovations of the future, bringing these innovations together.

How can we find all this treasure that's out there?

Well, pirates know how to find treasure. Here's Blackbeard. Blackbeard is maybe one of the most famous pirates of all. If you took the Blackbeard approach to finding treasure, what would you need? What's the first thing you need? Anyone?

A map, right? And pirate maps: very well-articulated, very specific, maybe cryptic instructions there. But of course, X marks the spot. You would know exactly where the treasure was that you're looking for.

Once you had that map, you know where the treasure is buried. You get your crew, you go to the island, and you start digging in that spot.

Now, maybe this approach would find treasure, but maybe it wouldn't find the treasure even that you're looking for. You might be digging and it could be 10 feet away, and you wouldn't even realize it.

But what we do know for sure is that you would not find the treasure that you're not looking for. You wouldn't be able to find the treasure that isn't anticipated, isn't part of your map, isn't part of your plans.

So Blackbeard and his pirate crew would need to get used to disappointment. All that time they invest in hoping for the luck and providence of the map, and maybe they'd come up empty-handed. But maybe there's a different way, a different way to find treasure.

So we think that there's a new brand of pirate out there who might say, "I'm not so sure I need a map." And they might go even a little bit further to say, "I don't want a map."

Inconceivable.

Stephen Fishman

You might think about that. That's not possible. But let's play it out for a second.

If you have a mission to find some treasure, and you set sail to the seas to find an island, what if you had a big enough crew that you could actually dig in every spot on the island at once?

Wouldn't that be interesting? You wouldn't have to worry about finding which place to dig, because by definition, you're digging everywhere.

So what if your pirate crew, instead of humans, was a set of 3D-printed drones that you could print cheaply on demand? And the more pirates you had digging across the entire island, the more treasure you might find. It might not be the treasure you set out to find, but there's unexpected treasures, like the ones that Matt is talking about, that you might not have known existed.

Clearly distributed drones in the thousands would find more treasure than the old-fashioned way of following the map. And it turns out this is actually an apt metaphor for how a lot of leading companies are thriving in the new digital economy.

The companies that try to predict the future with the top-down genius who's got this strategy and says, "This is how everything is going to work," the phrase is, that's like a highway without exits, because everybody's focused on the plan and you're digging in one spot.

But nimble corporations who plan with the idea that, "I'm not necessarily sure what I'm going to find, but I'm going to commit to a different approach to allowing opportunities and treasure to find me," that's the ones who are succeeding in the new economy, where the treasure is in a place where nothing marks the spot.

But this begs the question, what about those robots and shovels? Not everybody's got a 3D-printed army of robot drones. So let's go one step further.

Matthew McLarty

In 2011, I don't know if anyone remembers this happening, there was a Google engineer named Steve Yegge. I wonder if he's here. Remember Google Plus, everyone? The social network that was around for a very short period of time?

He was posting to Google Plus, thinking he was posting internally to Google Plus, going on this big rant about how Google needed to learn from Amazon and how to build platforms. Well, it went public and it went viral.

In that rant, he talked about what is now known as the famous Jeff Bezos mandate, or the API mandate. Anyone familiar with the API mandate? A few people.

That's where it came from, actually. It's not like somebody had the memo and published it. It was in the Yegge rant. He listed five or so principles. I think the sixth one was a bit of a joke. But it was saying everything must be built behind a service interface. It has to be built to be externalizable. That was an important point: we have to assume that people are going to use it from outside Amazon.

Teams were going to communicate through the APIs that they had there, these service interfaces, not just machines. So it was this very explicit set of instructions that was sent out to the company, and by all accounts was absorbed and taken forward.

You ended up with this whole new approach to building systems, very modularized, very constrained, which might sound like it would weigh down on innovation and creativity. But if you look at the results since then, Amazon, you can look at this whole point around 2002, when the mandate went out, from that point forward, all these different dimensions of expansion: going into universal retail, personalization, payments, logistics, reinventing what a book is, getting into all these different areas.

Amazon was able to quickly go into those spaces because they compartmentalized and built these API-based services.

Now, those ones that I list there, you could say somebody in a boardroom might have said, "Yes, this is the plan. We're going to do personalization and payments and all that." So maybe they could have done that with a map.

But the thing that they never would have found with a map, or even written a map for, was AWS. What happened with AWS was around that same time, Bezos had the mandate, and he had Tim O'Reilly in his ear talking about, "This is where the API idea came from. This is the future. You're going to have Web 2.0. You're going to have all these APIs connecting things. So when you build stuff, you should be ready to put it to the outside world."

Amazon had been dabbling with external APIs around product catalogs and other things. At the same time, they're building all the services down to the guts of the infrastructure using APIs. Combining that work that was happening to externalize some APIs with these services that had built and provision infrastructure in a very rapid way was the happy accident that led to AWS forming.

Stephen Fishman

If we're going a little bit further with the Amazon example here, they were able to find treasure in many places thanks to their many robot pirates. Those robots that we're talking about, those are options in the form of digital business capabilities, software functionality, and data, all packaged in a standardized interface that anyone can mix, remix, and compose at will.

You could think of these APIs for capabilities as the shovels, because it is only through API enablement that those capabilities can be easily combined, mixed and remixed, and externalized to the outside world with a very low cost of coordination. That's when new business value and new business models emerge.

Amazon is not the only one who's been successful with this approach, not the only API, right? How did Best Buy survive? How did they thrive when every other retail organization was suffering in the wake of Amazon?

Whether it's Best Buy or Coca-Cola or Cox Automotive or Capital One, we're going to get into all these examples a little bit later. They all focused on this idea of how to actually build composable infrastructures in the same way that we're virtualizing and not having to care about the infrastructure. We're hoping that there's another way of being able to allow that sort of recomposition at the application layer, even if they weren't doing it consciously. It led to a bunch of happy accidents.

Matthew McLarty

Happy accidents. We have to kind of accept, if we accept that the most valuable treasure that's out there is the treasure that we don't even know about, or that doesn't exist yet because it has to be invented, how do you aim for that? How do you build a strategy around that?

This is the focus of our book, Unbundling the Enterprise, that will be coming out later next year. Quick promo.

Our science of happy accidents breaks down into these three "oops."

Number one: create optionality. You're probably hearing a lot about optionality. We'll go into details on that and how APIs relate to optionality.

Number two is, within this context of going after the stuff you don't know about yet, how do you identify the opportunities?

And number three, when you have the opportunities and when you've been able to understand the landscape of opportunities, it's to optimize value.

Those are the three "oops." Now let's go into more detail.

Stephen Fishman

The first "oops": optionality.

Optionality may be easier to understand outside of the technical arena. As you approach your own lives, do you guys try to commit to the first option that's available to you, or do you wait and wait until the scene plays out and find and allow that concept that Gene calls slowification, allow more information to happen, slow down your reaction time?

As you approach your own life, if you're seeking a new job, or you're looking for a date online, or you're helping your kids through the application process, do you commit early or do you create the options and then commit late?

That's what happens when you slow down in the context of an API. You're not planning and locking yourself into a singular context of use. You're leaving yourself open for all the ways that people might actually consume it.

One of the greatest examples in the space is Google Maps. Google Maps was originally conceived not as an API, but instead as just a web app that would support their advertising business. But lo and behold, the happy accident happened.

Google actually never even decided to charge for it. It was only as a response to what customers asked for. The developers who were coming and actually pulling the Maps API by themselves, they went to Google and said, "We need to pay for this. Please take our money, because we want to build a business model on top of this to go search for our own treasure."

One of the ideas that we're going to explain in detail in the book is the idea of choosing for a decomposed architecture of discrete capabilities is not specifically a technical choice. It's actually a financial one, where you can give yourself the option to consume that capability in a near-infinite number of contexts.

To follow in the spirit of DevOps, when we talk about unicorns and heritage companies, we're going to mix a bunch of them. We're not just talking about the digital natives like Amazon and Google. We've got tons of examples from outside that universe, whether it's Cox Automotive.

Cox Automotive, you may not be familiar with. That's Autotrader, Kelley Blue Book, about 50 other brands, and a fairly large amount of revenue. They've enabled cross-brand consumption of everything through having these decomposed capabilities.

If you were a car dealer and you went to an automotive auction at Manheim, that's one of the world's largest automotive auctions set up in multiple locations around the country and internationally, before the gavel even bangs on your purchase of the particular used car that's in the lane, and before the car leaves the lane, you could already buy revenue-generating services from across all 50 of those brands.

Whether it's transporting the car to your dealership, or not even going to the dealership because you can even start to sell the car, including all the imagery and all the description and all the other stuff about that car, just through the click of a button on an app that is all API-powered.

Capital One, who is a frequent presenter here at DOES, has long been committed to leveraging technology as a value driver rather than a cost saver. This commitment to packaging business capabilities with APIs first emerged in 2016 with Capital One's DevExchange.

Capital One beat the EU by four years. Before any regulatory thing needed to be happening, Capital One actually made data portability through the DevExchange to allow other people to build on the Capital One platform. And they've even taken the next step, which is taking these capabilities inside that were once conceived as cost centers and actually rolling them out as SaaS opportunities, and they spun out to a full-on SaaS company.

Matthew McLarty

Thank you, Stephen.

I think that the optionality idea is definitely out there, and people are probably familiar with the power of optionality in lots of different contexts. But I think to put it into business context, one of the parts of the book, for sure, we can't get into the depths of this, it's an area we call value dynamics.

Value dynamics is about showing where these options and APIs fit in a business value context.

If you want to drive value in your organization, you want to be as close to the value exchanges in your business model, the business model of your organization. And business models, like everything else, are fractal. You can kind of go down, down.

At the top level, value dynamics gives you a way of visually depicting business models. Alex Osterwalder is the creator of the Business Model Canvas, and he defines a business model as the way an organization creates, captures, and delivers value.

What this value dynamics does, and it's heavily influenced by two Dutch gentlemen, Roel Wieringa and Jaap Gordijn, who actually have a book coming out exclusively on this sort of value exchange mapping, is it allows you to break down visually how business models work in very clean terms.

You can see the constituents in a value network, which can be depicted by objects, arrows showing where the value is flowing from delivery to capture, and then what we're calling value currencies. This is fundamental because you might think, obviously, value is money, value is products and services. But also you've got intangible types of value like reach, providing access to an audience, or time savings is value. Risk reduction is value. And of course, in a digital economy, data is extremely important as a form of value.

Just to show how this works with a couple of examples, because this is how you take your landscape of options and capabilities and start to put them into a framework of where you can actually have a business model or augment your existing business model.

We mentioned a couple of examples off the top. Coca-Cola, we had a great interview with our former CIO, Esat, where he went into depth about all the innovation that they were able to introduce by having this approach that was API-heavy.

But one particular example you might not realize is those Freestyle machines, where you can select new flavors and create crazy, undrinkable flavors of Coke products. That's all API-powered.

In a sense, you could look at that and say, "Okay, well, we have a new way of getting soda into these cups for our consumers," which would be just a typical, okay, Coke provides the beverages to the retailers, and here are the units that you use to dispense the sodas, and away you go. It's just another product-for-money exchange.

I guess an add-on to that, and you can see depicted here, is that there's this new service on top of that, which is beverage customization. That's a new value add on the top.

But even that doesn't tell the whole story, because through that whole framework, what Coke was able to do was create new points of data exchange. They were able to collect data back from the machines, which would help tune and optimize the operations of the machines themselves.

But also, in order to provide that value-added service to consumers, to say, "Hey, customize your own beverages," they had Bluetooth technology, or have Bluetooth technology, to help use a mobile app where you select the flavors.

Well, now, once you've got the mobile app, now you've created the direct connect between the consumers back to Coke as the product company, which opens up a whole new channel for delivering more value around loyalty programs and so on.

It's a very simple diagram when you draw it out this way, though you can start to ideate very quickly on where new opportunities are to add more value to existing channels, where you might be missing channels and want to add channels of value, and where you can even shut down channels where it's an imbalanced value exchange, and so on.

Another example here with Best Buy: they had a program when the mobile boom exploded where they were already doing a lot of work. They were doing the optionality work. They created a common platform to API-enable all these different services.

Through that, and through the thinking, they were able to say their Geek Squad service, which was a massive business around electronic device repair, was always in need of new inventory. At the same time, the retail outlets were wanting to sell all these new mobile phones. And of course, consumers were just dying to get their hands on mobile phones.

They set up a program which connected all these value streams together, where you could go into the store when you buy a new phone, they would promise you a buyback price to say, "Hey, we'll give you money back when you're out of your term so you can come in and get a new phone."

That's a win for the consumer because they're going to get some money back for their phone. It's a win for Best Buy because they're going to get a recurring revenue stream, or a pretty reliable one. But the magic in all that was the devices that were bought back were fed back into the Geek Squad business, which was able to then turn that into augmenting their parts inventory.

Were these companies using exactly this approach when they were coming up with these innovative ideas? Probably not. But when you have these options and when you're able to almost create a game board for strategy around delivering value, it's amazing how creative you can be with this and come up with more and more ideas to either just do some tweaking or come up with whole new business models.

Stephen Fishman

Just to follow on to something Matt said there, if anybody watched John Willis's talk a couple days ago, when he talked about serendipity as a fundamental thing inside the process in Japan, that's what we're talking about: being able to actually create an ecosystem that mass-produces these serendipitous accidents that lead to large-scale opportunities.

The third "oops" is optimization.

If you're a fan of Gene and the third wave of DevOps, you're probably familiar with the OODA loop that's up there, another golden treasure that came out of the military. You're likely to be present to the path to the value, paving the path to ever-accelerating insight that you can institutionalize for your own market advantage.

Each of the three different "oops" tactics works together to lower the cost of running trials and experiments.

We talked earlier about that top-down idea, that everybody's digging in one place, and that the only way to be able to dig in every place at once is to lower the cost of what it takes to dig a hole. The lower coordination cost of APIs is one part. Being able to at least do some targeting towards people where you're likely to be able to find value is the other part.

But the third key part in order to really lower that cost of experimentation is paving the feedback loops. It's not just enough to go and ask people things and get feedback in and then have meetings and figure out that stuff.

It's also important to have four key things. If you were a watcher of Etsy over the last bunch of years, they've been masters at multivariate testing through feature flags; ramps, so being able to channel audiences in low-risk environments with small percentages of audience; being able to visualize the results of tests real time, even within every second.

And at the same point, it's not just having all those tools. You need a staff that is capable of actually reading statistics and having statistical literacy. Because if you can't actually discern what the stats mean and what they don't, you're not going to be able to do the things that you need to, like doubling down or knowing when to fold.

When all three of the "oops" techniques are used in harmony, you set up this institutionalized way to get that accelerating path to beneficial insight, knowing when to double down, knowing when to fold, and you accelerate the harvesting of these uncapped gains that come from systematizing innovation. Those are the happy accidents.

Digital pioneers like Amazon and Google have a head start here, and it's easy to see the path in retrospect. Amazon, which now earns close to $100 billion in annualized revenue from AWS, never started out as an intent. That emerged from a system of experiments that utilized each of these three "oops" techniques: creating optionality via an explicit API strategy, understanding how their internal capabilities could be leveraged by a worldwide audience, and capped off by being ruthless and automating the end-to-end process of making experiments cheap.

It's an economic truism that when you lower the cost of running experiments, more experiments will be run. So that's how Amazon lived their day-one philosophy, where they catalog negative results by running lots of experiments in the hopes of finding the winner and ferreting out that winning option that allows them to double down, double down, double down via serial optionality.

For those who remember back in 2004 when Amazon launched AWS, it only had three products: S3, SQS, EC2. It is only over the last 20 years of consistently doubling down on the winning option that they created this thing that we now see as the backbone of all these application development opportunities out for the world.

Google, like we talked about earlier, built the web app, and it just ended up that people found it and used it to build other things.

Two things before we leave here. One, the three tactics are not specific to the digital landscape at all. The world is full of all these accidents in digital land and in other lands as well. Whether it's Coca-Cola, the actual drink product, Silly Putty, microwave ovens, penicillin, Ozempic, dynamite, all these things were discovered by accident. And then only in hindsight did they actually figure that out.

The one counterexample: I know some of the people from LaunchDarkly are here at DOES. Remember, Etsy provided all that basic functionality 10 years before LaunchDarkly, Optimizely, ExactTarget came out with that capability that now are accruing millions, if not hundreds of millions, of dollars. Etsy gave it away to the community, which is a good thing, but did they pass up an opportunity to actually strike gold with that unexpected treasure?

Matthew McLarty

Very quickly on this: we're not up here saying just go copy Amazon. That's why we deliberately went under the covers and distilled down the patterns that can be applied in any company's context. That's going to be the thrust of the book. This isn't just a case of, go and copy the ceremony and the surface value. We're going deep.

Stephen Fishman

You might be wondering what we're doing at a DevOps conference talking with this business strategy talk.

Let's consider these pirates again. There's a real alignment with another popular DevOps metaphor, which is the pets-to-cattle conversation. In the same way that pets to cattle abstracted infrastructure and allowed for a new layer of agility and new value to come out of internal capabilities and infrastructure, there's an even bigger treasure out there: being able to abstract and compose at the business logic layer of applications.

We've seen this mindset take off in the DevOps community, and we're hoping that we can help enterprises around the world see that there's that same level of value, but at a different exponential rate with the shift to APIs and optionality. There are cost and efficiency gains, but there's so much history out there of the gains actually being convertible in the top line.

We've gone over at a surface level what constitutes about a little bit around half of the book. Most of the book will take the science of happy accidents, as we've discussed here, and show how it can be applied in patterns of innovation for companies.

We've listed the four areas of our book here. What we're hoping is that people around the community can actually look at these things, whether it's exchange optimization, distributed innovation, capitalization on internal capabilities, and aggregating value through different value networks, and look at the examples that are in their hindsight and bring them to us. We'd love to talk with you if you've seen any of these patterns in the wild.

Matthew McLarty

To be fair, we will find digital ways of sharing those patterns because we didn't get to even cover that part, which is the main thrust of the book, after we get through explaining happy accidents.

So big thanks. Remember your "oops" as we leave today. This is the science of happy accidents. This is what we've seen empirically through examples we've shown and many more examples of organizations we've talked to, experiences that we've had personally in working with companies.

Make sure you're driving optionality, exposing business capabilities in a modular way through APIs. Make sure that you come up with this new way of looking at the opportunities created by those options. And then continuously optimizing the value exchanges through your automated feedback loops.

Stephen Fishman

So take away one thing. Remember, lots of small, cheap bets. Casinos don't gamble. Casinos work the math. That's what we're talking about. Thank you so much for your time.

Matthew McLarty

Thank you so much. Keep your eyes out for Unbundling the Enterprise, which will be coming out next year. We'll be sharing more information with more about what we talked about today online, and look forward to seeing you at the event and online.

Thank you. Thank you, everyone.