Building a Start-Up to Compete with Ourselves
Working for a company as old as Morse Code that is trying to make a wholesale pivot to digital education services in a market ripe for disruption from new entrants is a recipe for an exciting challenge.
In this talk we discuss Pearson's approach to managing this shift in focus and how we have taken the start-up mentality to heart building a new team with a new approach to challenge and drive change from within.
We will touch on how we see our emerging DevOps capabilities scaling in a global company of over 40,000 people and what a seismic shift in technology does to the varied silos of a large distributed enterprise. We will share what has worked for us to create an opportunity to drive change and where we see our next challenges as we launch our first production services.
Chapters
Full transcript
The complete talk, organized by section.
Chris Jackson
Good afternoon, everyone.
We're going through a transformation, so I'll drop all the words in, like "transformation" and "journey" and those things now. Agile. Agile. Yep, thank you. If you're filling in the business card as you go, the little bingo, just please shout out if you've got a full house or a line.
But I think putting some sort of story behind it helps people to conceptualize what it is you're trying to do. Some people will talk about their journey as a marathon or some sort of big challenge to overcome. Particular to how I think me and my team are working inside Pearson, we've really told the startup story.
Prior to Pearson, I worked at Rackspace, and I worked with a number of startups. I think there's a lot of similarities between the evolution of a startup and how we're trying to approach a different way of doing technology inside Pearson to position them for the future.
So this is the two sides of Chris Jackson. The one on the right tends to come through a little bit more than the one on the left.
I've spent an awful lot of time working in technical roles through Rackspace, doing all sorts of different bits and pieces with those. I was employee 200, just before they went really, really big. I've always thought of myself as a technical person, and the more technical people I worked with, the more I realized that I wasn't particularly technical, and I couldn't really hold a candle to those people at all.
So I pivoted myself and said, "Well, how can I make it my job to enable technical people to work better, faster, smarter?" And got myself into the leadership gig, where I get my kicks now from watching magnificent technical people do their work through a format and a bubble that I've tried to build for them.
So this is the story of me building a bubble at Pearson to try and do something a little bit different.
I think lots of people probably have heard of Pearson as a brand, and you're probably now thinking Financial Times or something, right? The story of Pearson started in 1844. I was hoping we'd be one of the oldest here, but then Thomas Cook have just smashed me on that one.
Pearson started in construction. They actually built the Blackwall Tunnel. So Pearson's story is one of adaptation.
At one point in our history, we also got into media. We were one of the first investors in BSkyB. We owned the production company behind Baywatch. Sure. Something about assets.
We were also a huge publishing house, and that publishing house led us towards education in particular. You've probably all read a Pearson textbook at some point in your development. The goal then became, how do we become a digital education company? So it's that ISBN to FQDN transformation we're trying to make right now.
It's 2,000 applications, 400 dev teams, locations all over the world. This is a super-sized challenge. But I come in every day because it's a job with fairly high moral fiber. If I feel like I've improved the quality of education at the end of my day, I can go home knowing I've probably lined some shareholder pockets, but I've also made the world hopefully a little bit better place by using that digital channel to try and democratize access to high-quality education.
So that's my company.
The organizational context, which I think is really key for these kind of talks: you could argue that every little bit of white space there is a silo or a wall in some way. It's actually worse than that. Every line break is also a silo.
We have a very broad C-level owner for the technology group, which is great because we've got one throat to choke around all the initiatives we need to really guide a DevOps initiative.
But if you're playing along, it's Conway's Law. Tick. That one's done now as well.
We've got groups here that are factioned even inside their own teams. Pearson has been a company of acquisitions for an awfully long time, with rather little M&A and integration work. So those brands and those identities still sit quite deeply inside the fabric of the organization, even though the org structure looks completely different.
What we see there is an operations team, and when I used to run the operations team last year, I had seven different operational models to try and harmonize because they came from seven different parts of different brands that we'd acquired.
So my job here is to try and build a bubble inside that to allow a new wave of highly talented engineers to do their best work, and stop the sharp and pointy things in Pearson from popping it.
This is the brief history, and you could probably insert any enterprise name here. It's that story of internal IT, which is bureaucratic and slow, exacerbated by those silos of brands, a very clear lack of standards.
Lots of very smart people in places like QA and security have got some very interesting ideas about how to go drive those things forward, but not able to translate that policy paper into an actual outcome.
Developers running a lot of their own tooling. I call that shadow IT in the light. We know about it because we're paying for it, but we can't really influence it.
In the challenge of Pearson, a company the size of who we are, with the business headwinds increasing, there's a huge transformation happening within higher education in some of our heartland markets like North America and the UK. New leadership coming in. There's that recognition we're making that transformation.
But really a long horizon for roadmaps. So we think we want to be this company in the future, but that point is so far out, we don't really know whether that's the right place to be targeting. This all drives towards this need to be able to do things a little bit faster. Because if we can't get to our end state within three years, we're just saying we're wide open for disruption from an external partner to come in and say, "Actually, I want to go and do the Uber of education." And if that happened to Pearson right now, that'd be quite catastrophic because we can't respond fast enough.
So the goal was to do something inside the company that would be as disruptive, but in a way that was ultimately friendly to the shareholders and to the company itself.
This is our inception point. This is where it all started. On the precipice of perpetually failing in technology operations. We were having customers move to shadow IT clouds. They were leaving behind the traditional operations groups. And if we'd have lost one more customer, it's a very valid question to say, "Well, who are you supporting in the business? What's left?"
But also, everything we'd done to try and solve it was a huge waterfall program, which is the antithesis of developer engagement.
So we did a big end-to-end review of tech strategy, and we realized we wanted to do something different. And like all good ideas, this started from a monologue. It was my monologue around why we were not going to go and spend a whole bunch more time doing developer workshops and customer advisory councils to talk about things they might want to do.
We should just stop that and start doing something. It doesn't really matter what it is. If we do it in agile enough iterative chunks, we'll get to a thing that's useful. But I felt that we weren't doing enough to respect the developers' desire for actual action rather than more talking.
So that gave us Byte Size. The word Byte Size comes from how do we eat an elephant, because it was a huge problem for us to go and solve. And this was the first commit that I actually made to the Byte Size repository, which was a code of conduct.
That's what we used to go reach out to different parts of the business and say, "We are trying to do something different. We want you to look at this and try and put some of your Pearson baggage at the door."
I don't think I really understood how impactful that was to people until I spoke to one of my engineers about two weeks ago and he said, "You know what, Chris? I was ready to just leave Pearson. And then I read your code of conduct, and I read that piece about leaving baggage at the door, and I realized that I had an awful lot of baggage that I had to go drop. And actually, I've really enjoyed what I've done since then because we felt able to go challenge the status quo inside the business and the things that all the people on the frontline can see are broken but feel powerless to fix."
So let's get back to this startup story. I've grown to love Silicon Valley, so this is our story.
We were founded in August 2015 on the back of that monologue. We did everything we could to pull our first customer in because you haven't really got a business until you've got at least one customer. And we called in a few favors to get one person to say, "Yeah, I'll try and get my application running on your platform just to prove it'll run an application."
That was a very tumultuous period because between August and December 2015, there were a number of different platform initiatives flying around inside Pearson. And we were in a battle for survival to be the one that the business was going to put their weight and their muscle behind.
We'll go into why that happened. It was more luck, I think, than real talent, but that's open to debate.
But once we got that angel investor VC onboarded, which is really just us being selected as the preferred approach for delivering platform, some more customers followed, the engagement started to build, which led to our A series funding happening at the end of April, start of May.
I've got funding now to go build a team, to go deliver a thing with a set of adoption milestones and commitments. We've just taken our first customer live on our production platform last week, actually. It's fully containerized. It runs Kubernetes on public cloud, and I can talk more about the architecture a little bit later.
So that sounds like quite a journey. How hard can it have been? It's only Iceland.
I'm going to walk you through a couple of really key problems. These aren't the only problems that we've had, but they're the ones that shout out to me the most.
The first one is, how do you fix DevOps? I don't know if this is the same for other enterprises, but I get this sense that every enterprise gets the chance to have a go at a word once. And if you break that word, all the people that try and use it behind them are going to have that same trouble. "Oh, we did that thing. It didn't work."
So we'd already done DevOps, except we hadn't. We had a DevOps team with DevOpers. And they had Jenkins. I joke you not. That is actually one of the quotes. It's in brackets for a reason.
There were some pockets of people inside Pearson that were doing really advanced, really mature DevOps work, something you would look at and say, "Yeah, that's really cool." But there was a huge bunch of people, the noisy majority, that were a little bit behind where they needed to be in terms of their maturity point.
So we'd almost turned DevOps into another silo. A dev team got a DevOp, and then they didn't have to talk to operations anymore.
So we had to go and do this, not on the grounds of we're better at DevOps than you. It was about credibility. If I talk about the previous waterfall delivery projects and the big enterprise IT stick that says, "You will come to us and you will use this thing we spent three years making and not talk to you about," we had to go rebuild that trust and that credibility.
That really started by just sitting down in a room with a small group of developers and saying, "What can we help with?" And at first it was, "Well, you can do all this stuff we don't want to do." But as that started to move and mature, we found that there was a set of requirements in there that almost gave us the fabric and the foundation of a product or a service that a developer might want to consume.
That set us off on that journey that said, "Well, maybe we can build a thing that will take a lot of headache and heartache out of being a developer in Pearson." Because whilst they've got the keys to the kingdom, they've got root access to Amazon, and they can do anything they want, the reality was they didn't really want to. They were kind of stuck with it because operations were so chronically failing them that they had no other alternative.
So we had to then work out, well, what is our product? What does a developer want? Is it the Airfix kit they can build themselves, or do I give them a ready-made plane?
What we found here was that it's a little bit of both. This is the difference, for me, between a product and a platform. A product is a shiny object. I've taken all the surfaces and I've polished them, and I've made them exactly how the user story says it should be. And I've handed that over and I've said, "Cool, here you go. Have it."
But the developer, from our experience, wants something a little bit different. They want something they can stick their own widgets on, and they can add their own little bits and pieces to.
For me, this is the secret to real developer engagement when you come at this from a kind of technology perspective: how do I let someone pave their own garden path? I'm going to build the road up to the outside of their house, but they're going to make their own garden path.
That ability to feel like you own a little bit of this builds that kind of cohesive, collaborative piece that says, "I really enjoy this because I own a little bit, and I've made it work in a way that's really good for me."
A developer can express their creativity, but they're still using a huge body of standards that gives us the ability to run really efficiently and maximize and leverage that scale.
So we're pivoting. Although my title is Cloud Product Engineering, we're really thinking about ourselves as a cloud platform. We hand off an API and some really awesome docs. We build a rich developer experience, and we let the developers go nuts with whatever widgets, plug-ins, and UIs they want to build on top of that.
Problem three: governance. This is my quote. I will claim it every day.
We deal with situations that are driven by governance, and that's where you get things like water-scrum-fall, and huge project management officers, and capital asset programs, and all that kind of stuff.
The challenge I have with this is, I respect the need for governance because in companies the scale and size of Pearson, there needs to be checks and balances. But how do we recognize that those checks and balances must evolve in the same way the products and technologies are?
So for me, take the example of a capital asset, right? I'm funded on the grounds that I am building an asset for Pearson. An asset, by its very nature, will depreciate in value over normally a three- to five-year period. If I'm doing two-week iterative sprints to build more features and functionality, how is my asset ever depreciating? It's arguably increasing in value or at least maintaining value across that entire life cycle.
So we've got a discrepancy here between the way that we manage and govern investment in Pearson and the kinds of technology and deliverables we're outputting at the side.
So whilst the big challenges on day one are getting people bought in and getting things moving in the same direction, the piece that everyone's kicking the can down the road on right now is how do you get finance and compliance and audit behind these things, so that you're not violating some GAAP principle and, in three years' time, going to jail over it?
So what do we do? We can look at this from a number of different angles, but I think the first one is probably some of the tech.
We were working with a set of developers in Pearson who had already, in their minds, done the cloud, right? They'd got Amazon. They had built some infrastructure automation capabilities. They had an API. They were doing stuff every day with a level of automation.
We couldn't go in there and say, "We can do cloud better than you," because it's just going to create conflict and tension around not invented here, no, this was my idea, et cetera. We had to make the decision to go ahead of the demand curve.
So we bypassed IaaS, just went straight to containers, and said, "Look, there's very few developer groups in Pearson right now building on containers. Let's build the best possible platform to run containerized applications, knowing that a good percentage of our development workforce are currently looking at exploring those solutions."
So rather than building the house as they get there, they get to walk in the front door. They can pick up a coffee. They can enjoy themselves in this ecosystem that we started to curate, and we invite them in as early as possible.
So this is where we get our investment in open source technology. With the exception of about two vendors, we are 100% open source because open source, for me, has a direct affinity with what we do in education. It's the sharing of code, the sharing of knowledge. There's a real good story here about how Pearson have a responsibility, being in education, to also help share the code that we're outputting.
These are real commits from members in my team who have built things for Kubernetes. They've built Python clients. They're working right there in the community, and we're doing it as much as we can in the open. My engineers blog. We had one of our guys talk at KubeCon back in March. We're trying to make sure we share this stuff as much as possible because we get the benefit then of everyone else following us. We don't have to own all that tech debt. It's not just our problem.
The obligatory marketing slide. Like I said, the goal here is to get simple, consistent delivery of code to a cloud-agnostic platform.
We want leverage over vendors that sell cloud. We want to be able to evacuate workload and move from Amazon to Google to Azure to OpenStack based on cost, based on performance, based on whatever we can think of, right?
It's a very difficult thing, as a unique example, to get services into China. Not every cloud provider is equal in their delivery of services to China. So having that portability means we can exploit best of breed without really becoming too beholden.
And this is all wrapped up in three files, three files that we call the Byte Size manifest, and it's really like a DSL for deployment of applications inside Pearson.
It's consistent to all the apps that use the platform, which means there's a lot of portability and reuse. And that reuse is key if we're thinking about building an education platform on top of this.
We don't want 15 ways of doing learning management systems or three different ways of doing testing and assessment. We need to be able to combine those core education components together to deliver different services to different markets and different age ranges at different points.
It's almost like that kind of Salesforce model of they've got one platform, and they sell it to a mom-and-pop shop with one employee, but they also sell it to Oracle, right? That breadth all comes from a single set of components under the hood.
And as we move this piece on and think about that platform, I think we might have become accidentally serverless. If you read anything I've tweeted or blogged about, I'm not a huge fan of this idea of serverless. I think it's akin to saying, well, in DevOps of old, it was about chucking code over a wall. Now we're just chucking it over an API, and that makes it okay.
What I think we're doing is we're putting a trust model in place that says there are things that sit beneath this API that you used to care about, but as long as we've solved that higher-order value conversation about the output or thing, as a developer, that you want or you need, it's not really about how it gets delivered, it's about how quickly you can get it back.
And if we can automate the majority of that, we can reinvest our people in the human-enabling functions of sitting together and working through problems, discussing things, and building what I'd call community governance. Take the open source model and apply it inside a large organization. It's a self-governing community around development standards, around the approach to building services, APIs, interaction models.
And we're using this platform approach as a vehicle for other enabling functions. Technology operations is just one. We also have a security group, we have a network group, we have a testing group. So rather than trying to break all those silos down, how can we package them inside our platform such that the developer sees the de-siloed view of it, but we still manage that piece on the back end to put some level of sense into our roles?
What we do is, within the workflow automation that we're building, a CISO group can drop their security testing payload into our workflow on demand via a Git repo. So they load their test into a Git repo, we pull it in, we extend our test suite with that stuff they've added, and we can do that based on any number of teams that want to try and package their view of testing or quality into the delivery workflow, such that then production can be completely immutable.
And that's the engagement piece, actually, that the groups were looking for. That's how we got their buy-in, because they were struggling to conquer the developer groups on their own as well. We gave them a vehicle to say, "Get your opinions into this. This isn't just DevSecOps, this is Dev, Sec, QA, you name it, Ops." There is no limit to what we can drop in there.
And then we can then go focus the people on a global technical community that sits around that. So if it's underpinned by a next-generation application platform and we put a small, lightweight investment in things like community engagement and developer experience to curate an ecosystem inside Pearson, there's 2,000-odd, 4,000-odd developers in Pearson. That's more than enough people to have a very active and very healthy community, who could be responsible for things like best practices and standards for new technology adoption.
Technology operations supports a thing like Apache. It's been around since time began, and we've got a pretty clear view of how we do that. Now, a developer might come out of left field and say, "I know it's been around a while, but I want to use CouchDB." And we say, "Well, we can't really support that."
But if we've exposed the framework for how we support Apache, you're just dropping your own Docker image into that same harness. It's inheriting that sustainability matrix, and we're allowing developers to move forward at the speed they need to with new tools and new technology without giving them a barrier to say, "No, you've got to wait till Ops has approved that."
We just say, "Look, here's the framework. Drop it in. If it passes the test for production, you can run it in prod. You own that." And if everyone in the company over the time uses it, we might choose to incubate it back into our core functions again.
Not to mention here hackathons and brown bags and meetups. How do we get that knowledge-sharing stuff going? Because we're trying to break down these branded silos inside Pearson and get a group of developers and a group of operators to think globally and to share experiences.
There's a huge amount to be gained from the huge number of smart people that work inside Pearson. Facilitating those interactions right now is our number one blocker.
And things like hackathons will make it fun. This stuff should be fun. It should be that we can innovate faster, and we might find that something we cobbled together in 12 hours in a really crazy coffee-induced hackathon becomes our next big thing. But if we don't have them, we'll never know.
The other area to this is where we're taking operations. If we think about our ability to orchestrate workflows, we use an open source tool for workflow orchestration called StackStorm.
StackStorm was recently acquired by Brocade, but we're still using it and persisting with it because it's a really valuable architecture. Above their API, you have the ability to define YAML-based workflows for a thing you want to have happen, but those things can traverse any number of tools that are in your ecosystem, based on your ability to write a small pack that connects their Python SDK with that tool's RESTful API.
So most of the major DevOps-centered tools are out there in that space already, which means that we can write a workflow that says, "If this..." Whatever you want. We could do very, very complex orchestration across a number of discrete tools to abstract all the different nuance of those tools into one higher-order value conversation.
And if we can then trigger those off of a webhook from a monitoring system or a log analytics platform, we've taken operations out of that pathway completely. We've said, "Trust the monitoring systems to have the right level of awareness and know then that we run these workflows when these things happen."
We're building self-healing systems. And if we take that one step further, think about where we're going with cloud-based AI and cognitive systems. If you can also provide an authoritative source of metadata, I know Gareth's going to think, really, that metadata is the one thing where we could do more, having more tagging against instances, against infrastructure, against applications.
It gives a combination of log and monitoring data with runbooks some contextual awareness. If we're building large-scale distributed systems, we want to know about things like cascading failure, because those tend to be the most damaging things that can happen.
If we've got contextual awareness of how all these things sit together and we can simulate how these workflows will play out, we can potentially predict cascading failures before they happen.
So we're not just talking about automating operations out of a job. We're talking about a sentient managed service here, something that can learn how to run this stuff better than we can, as long as we keep pumping it with the right information.
And that is how we're going to scale DevOps inside Pearson. It's not going to be dropping DevOp people into every single team, because we'll get no consistency and we'll get no scale. This is about investing those people into a community that is self-governing around a set of technology tools and workflows that we can effectively crowdsource. Because we've got the scale to make a huge amount of progress on this in the space of a couple of months if we can really harness the power of all that workforce towards a common goal.
So where do we need help? Pearson's motto is "Always learning." So I'm always open to opportunities to learn.
I think if we go back to our startup example, I'm thinking about how does my startup now manage hypergrowth? If I've got my funding, I've got some customers, we're getting interest, how does a startup now manage hypergrowth? How do we get through that difficult phase where everyone wants a piece of you and you're having to scale everything on the fly, not just in terms of tech, but in terms of people and process and maturity?
When does DevOps stop being DevOps? At what point have you touched a part of the business which is non-technical? You can't really call it DevOps at that point. Is it just common sense then? Is it just empathy? Is it just giving a shit about other people's jobs for a while?
We've been joking internally about the concept of executive DevOps, right? That's when SVPs in your company come together and realize actually they've got common goals and they should work better together. It comes with some special key to a bathroom or something.
And then around that governance side, how do we recognize productivity gains rather than cost savings? So going to a finance team saying, "I'm going to save you 20,000 developer hours," says, "Cool, how many developers can I get rid of for that?" You've missed the point. How can I reinvest that development stuff into other projects to build new revenue streams?
So having to have that conversation that a gain is not necessarily an opportunity for a saving is not really sold for us just yet. We need to do some more work.
Pearson have got some really passionate veterans, people who have got 10, 15 more years of tenure at Pearson. They've seen every transformation under the sun. But I'd argue they've been under-invested in, in things like cloud.
So we're finding that there's these people who have got this passion for Pearson, which is undiminished, even through all the hard years that they've gone through. We have to be able to harness that, but we have to be able to try and transform their skill sets.
I know Lee was talking about there's still jobs for administrators. There absolutely is. But we're thinking about how do we start to manage operations teams like engineering teams who are working like developers.
And then setting the maturity bar. I've spoken to a number of different groups, and I was very interested in what Ticketmaster were talking about yesterday around defining application maturity standards.
Bottom line, although we've got some great applications in Pearson, we've also got some really bad ones, and if those bad applications go onto a great platform, it will become a bad platform. It's only a matter of time.
And telling people their children aren't pretty without data is difficult. So how do we establish a framework for maturing applications to say, "Yes, you are a good fit, you are a bad fit, and you in the middle, little bit of work and you'll be right there. Here's the potential cost and the business case for whether that's worth it versus green-fielding the entire application."
So I talked about open source. We are going to be putting more and more work on our GitHub pages. If you don't catch me through the rest of the day, happy to tweet me. We can have a debate online.
And as with all good startups, we're hiring.
Thanks very much.