This is Not Just Any Transformation; This is an M&S Transformation
Experience report sharing M&S experience and lessons learned from accelerating their shift to Product & DevOps ways of working within their Digital & Technology Teams over the last two years.
Chapters
Full transcript
The complete talk, organized by section.
Host Intro (Gene Kim)
Marks & Spencer is the famous UK retailer with annual revenue of over 10 billion British pounds and over 75,000 employees.
I am excited about this presentation because it describes in gruesome detail what it takes to transform the business processes and the business-critical systems of a large, complex e-commerce system in a large, complex retailer.
For those of you who attended the half-day workshop that Dr. Steven Spear and I ran two days ago, you heard how excited I was to learn about the tipping point that forced Amazon to refactor its Obidos code base in 2002. It was when they shipped their clothing apparel product category.
The reason was that the SKUs, or stock-keeping units, for clothing apparel were so much more complex than for books and music. This created the need to create so many database schema changes. Each one of them could often take down chunks of Amazon globally.
Up next are Richard Day, Software Development Manager, and Lee Wilson, Staff Software Engineer. They are going to tell the amazing story of what Marks & Spencer has done to modernize. When you hear them mention the words bee cushion, think about how many problems Amazon had with clothing apparel back in 2002. Here are Richard and Lee.
Lee Wilson
Good afternoon everyone. I am Lee Wilson, and I am a Staff Software Engineer with M&S. I have been with Marks & Spencer for about 10 months now, and in about 10 months it has been an incredible journey. So much has changed in such a short space of time.
Before that, I worked across the industry, from startups in fintech to scale-up and medtech, and now, of course, global e-commerce at Marks & Spencer.
Richard Day
Hi, my name is Rich Day. I am an Engineering Manager at Marks & Spencer. I have been with M&S for about a year and a half, but I have been working in the software industry since 1999, mostly in fintech and financial services.
M&S is my first foray into the retail environment and the first time I am working with an organization of this size. My passion is team culture, helping teams be the best they can be, and supporting engineers, empowering engineers to help build their careers.
Lee Wilson
M&S is a British institution, established in 1884 as a penny bazaar by Michael Marks. He was joined by Tom Spencer in 1894 to create the famous name that we have today. We have come a long way from the first market stall in Leeds, and we are now a multinational retailer with a presence in many countries.
Our core business covers clothing, home, beauty, and food, but we also offer other services too, such as banking and opticians. We are proudly an own-label retailer, but we do offer carefully curated products from third-party brands.
In 2007, we launched Plan A, our industry-leading sustainability program, and by 2040 we will be carbon neutral across our entire supply chain and store estate.
As an organization, we cover probably 65,000 colleagues. That is quite a lot. In our little part of the world, we have about 4,000 colleagues in digital and technology, and they comprise software engineers, QA, design, and all of the people that you would expect.
Before we go any further, it is time to share some love. If you live in the UK, anyone from the UK in here today, give us a wave. Loads of you. Fabulous. I am pretty sure that you have shopped in M&S recently. I can pretty much guarantee that you have got something from M&S in your house right now. You might even be wearing something.
According to a recent documentary, one in four UK men wear M&S underwear, so at least some of you are wearing M&S pants right now. For the ones that are not, you are missing out.
We have some famous UK brands as part of our portfolio. They have people like Jaeger, Per Una, Autograph, Blue Harbour. They are all well known and well loved amongst the UK public. We have Goodmove, our activewear range, that has seen rave reviews across the UK press for price, performance, quality, and affordability. There is a lot of stuff out there.
YouGov polls suggest M&S is popular across Generation X, boomers, and millennials, and I think there is a reason for this. What strikes me about M&S beyond any other brand in the UK is that everyone has a story about us. These can sometimes be intensely personal stories, loaded with emotions, because they relate to pivotal points in people's lives.
For example, children going to school for the first time will often get their first school uniform from us. Young women will have their first bra fitting with us. Young men and women will often get their first suit for their first job interview from us. People trust us. The UK trusts us, and we are with people in the UK throughout their lives.
Personally, for me, working at M&S is a source of an immense amount of pride, both personally and professionally. It is the highlight of my career so far. You see this in our colleagues across the organization too. We love us more than anybody else loves us, I think. And our 20% staff discount helps as well.
I want to take you back to the start of our DevOps journey, two years back to 2021. We had all just experienced COVID, and it resulted in unprecedented economic downturn. Brick-and-mortar shops up and down the country had struggled as the pandemic further accelerated consumers' utilization of digital channels.
Fundamentally, it became clear that only one thing is certain in this life, and that is change. We knew we could harness it, but two years ago, technology change within M&S was difficult, it was slow, and when it did occur, it did not always have the desired impact.
We did not understand the why. We had a traditional project mindset, focusing on features over outcomes, putting too high an emphasis on cost and return rather than the impact we were having. As a result, we did not dedicate enough time to tech debt, tech improvement, and did not invest in the long-term health of our systems.
We had siloed our devs and our operational teams, which meant our processes and structures were diametrically opposed: stability fighting against progress.
We had outsourced 95% of our people within digital and tech, resulting in large project teams of 40 to 50 individuals with unclear accountability, unclear ownership, and a lack of autonomy.
We had a legacy tech stack based on proprietary in-house frameworks, which were becoming increasingly difficult to hire for. This led to frustration at every level of our organization. We had so many amazing people doing amazing things and finding themselves stuck in the mud.
We knew if we continued like this, we could not compete, we could not stay relevant, and we could not be pioneers in an ever-changing market. We knew we would go the way of the dinosaurs, and we knew things had to change.
We set ourselves a mission. We were going to use technology to enable everyone in the business to help our customers, drive great experiences, and be pioneers in our field.
We had a vision to be agile, flexible, quick to change; to be driven by data, not by opinions; to be iterative, relying on small feedback loops; to be willing to experiment, willing to stumble, but most importantly willing to learn; to be outcome-focused, not output-focused; to have everyone care about the metrics we are trying to change and why we are trying to change them; and, most importantly, to put people at the heart of our transformation, empowering them to enable the business.
So we had a decision to make. How do we start out on this journey?
We had a big ambition, but we knew we needed to start small, test the waters, and establish patterns and processes which would spearhead us onto bigger and better things. We needed a project that was self-contained, one that did not suffer from the legacy of the past and would give us a chance to learn.
We chose Jaeger. It is a British fashion brand that Marks & Spencer acquired in January 2021, which we wanted to bring into the M&S tech fold. We needed to create a new homepage, one that would link back to our existing product pages.
We decided on a static hard-coded microsite, leveraging modern technologies like Next.js, TypeScript, and Azure, learning and understanding and evolving our approach. We built a platform to support this, which we called Onyx, and created a new in-house team empowered to use those technologies and make the right decisions, with DevOps best practices and continuous improvement baked in from the beginning.
It worked. By starting small, it enabled us to test and learn, to get something into production quickly so we could understand how our customers were using it. By doing so, it confirmed our approach. It created the foundations and a team that could seed those ideas as we scaled. It created a product showcase that helped grow momentum inside our company and get others around the business excited in what we were doing as well.
Our humble bumble. We had dipped our toe in. We had created an autonomous, empowered team to create an amazing homepage for Jaeger. We had trialed and tested technologies and brought the specialist knowledge we needed in-house.
The next easiest step would be a simple product page: no sizes, no colors, nothing but an add-to-bag button and some hard-coded product information. But what would that give us that we did not already know?
Easy is not always right. We knew if this transformation was going to be a success, we needed to test its mettle. We needed enough complexity: complexity to test our approach and enough components and infrastructure to accelerate our future product migrations.
We needed a product range that had enough orders and customer traffic so we could use the data to prove what we had set out to achieve: improve customer experience, increase conversion, and deliver change at a faster pace.
So we chose our humble bee cushion. It might not look like a complex product, but under the hood there are a lot of moving parts. It required the building of dozens of complex front-end components that would form the foundation of a library that all of our web-based applications would use going forward.
It integrated into our existing basket and add-to-bag flows, allowing for multiple types of product, each with our own unique SKU. Our existing APIs just were not fit for purpose, so we spun up a GraphQL layer to create a flexible, scalable approach to our data needs.
We were ambitious. We did not want this project to take years, so we created the teams to make such a dream a reality. Within six months, we went from nothing to 1% traffic. Within weeks, we were up to 20%, and we kept going. We were outperforming our legacy page on Core Web Vitals and conversion, resulting in increased revenue.
We had embraced a truly DevSecOps model by allowing our teams to own the full lifecycle of their product. But it was not all plain sailing. The waters were choppy at times, but we embraced and mitigated the uncertainty and the risk by choosing a single product. We reduced the blast radius for migration. If something went wrong, only this one product would be impacted.
We focused on automated testing and finding and resolving issues before they made it into production. We empowered our teams with the right tools and training to monitor their own application, meaning if an issue did make it to production, it was identified and fixed early on.
All of this gave our stakeholders the confidence they needed, and the teams the experience they required, to shift towards a DevOps-first mindset.
Richard Day
At this point, we were gaining some traction, but we all know culture eats strategy for breakfast, and we knew we needed to get our culture right.
Let's take you back to something Lee mentioned earlier. One of our challenges: it is 2021 and digital and technology had outsourced 95% of its people. We knew this is not where we wanted to be. We could not be.
We wanted to create outcome-focused teams, empowered to make their own decisions following the strategy of the business. We wanted to create an environment where those outcome-focused, high-performing teams could thrive.
A place where data, innovation, and experimentation were at the heart of every decision that we made. Where DevOps was not just a phrase, but a truly lived experience. Where metrics like DORA helped us drive incremental improvements. Where psychological safety was inherent.
Psychological safety, in my opinion, is one of those bedrocks of a high-performing team. Where our leaders acted as coaches, helping everyone to be their best. That is through things like thought leadership, fair challenge, collaboration, and not just technically. We wanted really amazing people where everyone felt that they were supported, felt they had mastery, autonomy, and purpose.
We knew we needed to get the right people in: talented, passionate people who wanted to do the right thing. We have a multi-stage recruitment process with a strong focus on cultural fit. In the last two years, we have interviewed a lot of candidates. I have done nearly 200 interviews in about 18 months, and I did 15 in March. I am not top of the list of interviewers. We have about 60 people that do it. I am knocked off that list, and that is really annoying.
We have grown the number of permanent engineers from 25 to 120 within two years, and we have still got more positions to fill. If you are in the UK and you are a front-end or back-end engineer and you want to work for an amazing organization, come and talk to me afterwards, but only if you are not with your boss.
Seriously, we are really looking for some good people. You are welcome to come and talk to me. We have created a career framework that is going to help us grow and promote our existing engineers. We have embedded agile and product coaches in every team, so that really helps everyone embrace that DevOps mindset.
We have created space for our engineers to upskill so that everyone can be a master of their craft. We recognize every team's journey with DevOps is going to be different, so we have created frameworks, and they are focused on continuous improvement, prioritizing progression and not perfection.
All of this while we are creating multifunctional teams that own, develop, and operate their own solutions to break down the silos between developers and operations. We are working hard to become outcome, not output, focused, embedding our OKR process across all 210 products within the digital and technology estate. The teams own the needles that they are trying to move, maintaining cohesion with the company's strategic direction.
Lee Wilson
We have gone a long way. We have gone from a pilot to a more realistic, complex project, and we created successes on that way. We have proven the approach and have buy-in at multiple levels of our organization.
But now the real challenge begins. We need to scale it. We needed the right guardrails in place, processes that could act as a compass for our teams, helping them as they made their own journeys.
We redefined our product model and team structure using principles like domain-driven design and inverse Conway maneuver. We created frameworks around product and DevOps maturity, like this one focusing on product health.
We changed our funding model from a traditional business-case one to one based on value delivered. We did this, and we did so much more, because we knew for this to succeed it could not be done in isolation. These changes would impact broadly, and so we worked with different groups around the business to help us define and create the changes we needed. By doing so, we brought people on the journey and got them invested in this transformation as well.
Now it is time to turn it into a reality. We continued and accelerated our work on the platforms, integrating security by design and creating a one-stop deployment process and an ability to create rapid services.
We started to create our new domain APIs, onboarded more and more of our applications onto our new Onyx front-end framework, and kept growing our permanent engineering force. But it did not go entirely as planned. We have had our setbacks, and as our teams gained experience in their domains, new discoveries were made, new ways of working were created, and our processes, like everything else, had to be improved.
This, above all else, is at the heart of our transformation: we cannot predict the future, so we focus on creating small feedback loops to help us iterate quickly, driving us on to success.
Six months on from when our humble bee cushion went live, we have now migrated 100% of our traffic for all our major product categories, meaning 70% of our website revenue is now on our new and improved tech stack within six months. We are not stopping there.
We have 13 additional projects being built on our Onyx platform, from our loyalty program to our payment services. To support this, five domain teams have already been created, and we are planning for 25 in total.
This is possible because we are enabling our developers, making sure a new engineer can be up and running quickly, contributing code in their first day. We are enabling the business by making sure experiments are quick and easy to set up, and supporting it all with metrics like Core Web Vitals, where we have seen an immediate improvement for every app that has undergone a migration.
Richard Day
Gene asked us to provide some testimonials, and this is a quote from a lady called Krista Nordlund. Krista is our most senior stakeholder. She is Chief Digital Product Officer, so she is outside of the exec committee. She is our most senior stakeholder.
Our transformation has helped us improve the performance of our web experience for our customers and our colleagues working on this platform. It enabled a step-change improvement in how we work, going from one or two releases every fortnight with no standards to multiple releases per day and industry-standard security and safety built in. Our time to introduce change to this code was reduced by 76%.
We all know DevOps is a journey, not a destination, and we still have further to go. Our processes need to continue maturing. Soon our funding model will focus on total cost of ownership, utilizing run, change, and cloud costs, creating the visibility we need to enable better decisions at every level.
We are going to break our reliance on huge monolithic technologies such as SAP, our warehouse and order management system, and continue with the great strides we have made in domain-driven design, helping us reduce the cognitive complexity for teams while enabling a faster pace of change.
Of course, we are going to keep improving our platforms, our frameworks, and our applications, making sure they grow and evolve as our needs do. We want more of our teams operating at DORA's elite level, using metrics to drive our improvements and create an amazing developer experience every step of the way.
We need to build on a rich culture we have created, creating a company that not only accepts DevOps but embraces and loves it as much as we do, helping us break down existing silos within our business, allowing us to organize our value streams and create even greater opportunities. All of this while continuing to hire amazing people, bringing even more fantastic ideas to the table, helping us do what is most important: creating an exceptional experience for our customers.
We would love to hear from you about anything. Come talk to us afterwards or hit us up on LinkedIn. We love talking, as you can imagine, and we cannot wait to hear from you about your own experiences.
In particular, we would love to hear from anyone who has migrated large monolithic applications like SAP. We are especially interested in how you decoupled, de-risked, scoped the changes, and how you made it a gradual transition rather than a big-bang approach.
We are incredibly interested to hear about your own recruitment processes and strategies that you have had, how you have managed to focus on getting the right talent, and reducing the lead time and complexity for getting those people in.
Finally, we want to hear about your experiences on how you created team and organization structures following inverse Conway maneuvers, the challenges and successes that you have had whilst you are enabling streamlined teams across complex horizontal and vertical product portfolios.
Lee Wilson
The tongue twister. I bet. Good work.
We would like to leave you with one thought, and this is a total vanity moment for us. We give you a quote to take away: DevOps is not a thing that you do. It is woven into the fabric of your culture.
Richard Day
This has not been a transformation. It has been an M&S transformation. Thank you very much.