Bala Madhusoodhanan

Hello. Greetings to all who have taken your valuable time and joined this event. I would like to thank Gene and his team for organizing such a wonderful event. As every year, I'm sure this year will also provide nuggets of learning and sharing from peers in the industry.

My name is Bala. I work as a principal architect with an off-price retailer, and I'm responsible for providing solution direction for supply chain and AI-related products.

This talk is all about something called continuous everything, which means that we made our journey a learning stage. Every time we did something, we picked up something that was good. We looked into things that weren't working for us, and we invested time to correct that or improve that. Hence, the title of the talk is Journey to Continuous Everything.

Our journey begins somewhere during the start of the pandemic. The biggest challenges that we had were shifting from an office four-wall workspace to a home or hybrid workspace, which meant the working culture of the team was completely changed. There were other challenges, like enabling secure Teams meetings or having a collaboration platform enabled for all the team, and things like that.

The focus basically changed from business innovation on products and services to business continuity. With the change in priority came the big challenge with the change of mindset. Also, the other thing that we were trying to tackle during that time was catching up with technology.

A lot of our systems supported a lot of our business processes. But having said that, we had processes not supported by systems. What do I mean by that? It was basically a bunch of Excel and Access workbooks, which were used by the business as part of their reporting and trying to make decisions based on them. Our journey was to modernize decision-making, reducing the tech debt, or removing the business processes that ran on an older technology stack.

What it meant was we had to rethink the whole process. Given that now everyone is hybrid, how would we deliver software products and services with a remote mindset? There was always a fear of failure because of the new ways of working. The other thing was how to adapt to a product mindset, having delivered for so many years with the waterfall model. The next one was how to upskill or unlearn the software delivery process and relearn to deliver it in an agile methodology.

This meant that there was a break in the way we delivered product. We had to rethink the process. We had to respond quickly and adapt at a very fast rate. Also, the timing couldn't have gone so bad. We had to shift our workloads from an on-prem data center to cloud offering.

Having been wired in the old ways of working, new business ideas and products must continue to evolve without longer lead time. What I mean by that is, typically for waterfall projects, all the big decisions were made up front. The documentation was very heavy. Changes to the agreed design were very intense. We had to involve change management. There was lack of investment to evaluate and try technology seamlessly, which resulted in lack of innovation. There were silos between the organizations, which were causing barriers to changes into production.

We needed to run fast just to stay in place. If we wished to go anywhere, we must run twice as fast as that. As you can see, Bolt is laughing at his fellow marathoners, and he's running probably quicker than everyone else. The analogy here is that as the pandemic hit, there were a lot of changes, and there were concerns with security and vulnerability and things like that. The technology was evolving a lot. A lot of products were bringing in machine learning and AI. We had to scale up to understand them, to test them, if they worked for us, try them out, have a faster innovation cycle, and then implement the product or services.

What this meant was we had to break through the mindset of typical waterfall, which means, oh, I need to know all the requirements way ahead. Not going to happen. So we had to invest in learning faster, testing faster, delivering value faster, and failing faster.

If we wouldn't have done that, what would have happened is we would have been playing catch-up, and our competitor would have found an opportunity. They would have made a product or a service, and the consumer would have gone to them rather than us. The whole thought process that the team had was: instead of constantly catching up with the change, what if we try to adapt as the change evolves and implement cutting-edge technology and patterns so that we drive value to the business?

What that turned out to be is, at one side, you had architecture with really good vision of what things should look like within a product team, and on the other end, we wanted the delivery team to deliver those things. What that meant was solution delivery needed continuous improvement. They had to get better every time, and if they failed to improve on the current work, they would basically fall behind. The whole idea was how to adapt faster, how to learn from our failures and implement them and improve every time as a team.

The analogy that I would like to give here is the mindset that Blockbuster had. They were video rental stores, but the way Netflix disrupted that literally made Blockbuster out of business. It is to do with the mindset. Execution of ideas is very key, and it's always hard to catch up with changing technology.

The team had to go through a different thought process. The old ways of working were typical waterfall, but the new habits were to work in a very collaborative manner with a hybrid or remote working style, and implementing or evangelizing the shift-left development mindset, which means they had to break the big block of development puzzles into smaller blocks, perform value adds, and then evolve the product and services in the smallest increments. Testing out new technology, new ideas, and driving value stream was the key for this mindset.

This resulted in a new culture of collaboration. We had to reorganize the whole organization to align with new types of job profiles, which were product managers, product owners, lean portfolio management, and things like that. There was intense collaboration between all these key stakeholders with architecture to drive the product roadmap and to ensure the planning of the product release was efficient.

The other thing that was quite key was, as we started moving away from technical debt, the evaluation of new technology and new platforms needed to be much quicker. We started enabling those technologies through enablers, through technical spikes, and always these things were catered as part of the capacity planning of the product team.

Because it was an architecture working group, the learning from one technology or one product was shared with the other product teams to ensure that it was not reinvented, thereby saving a lot of time. The product owners and product managers spent a lot of time performing empathy mapping of the end user or a customer while designing their product.

We had to shift the way we were delivering products into production, which meant we had to change our software delivery deployment models.

What we started doing was looking into the value stream and identifying pockets of activities that were monotonous but repeatable. That meant we started cataloging opportunities where those things could be solved through automation.

Automation could be exploring the testing automation of any product. We worked with the product managers and product owners to define a core regression pack for certain products or services. Every time there was a change, we started using that core regression pack. Obviously, as you add more features, you revisit the regression pack and evolve that regression pack. That was the first automation.

The second was that we started doing value stream mapping and figured out that a lot of time was invested in scaling up and scaling down environments. So we started automating infrastructure. That was a big value add.

The third and most important thing was implementing Azure DevOps, which enabled a continuous integration, continuous delivery pipeline for all the products and services that we were implementing.

What that meant was there was a slight change in the thought process of the team. The behavior started changing. People started understanding and learning from the previous cycles.

For example, what we did was the team was more focused on technical debt. The team started questioning non-functional requirements as part of their product backlog refinement. We started to assess capability and prioritize based on value.

Fostering innovation was one of the key drivers that each of the product managers invested in. We explored multiple technologies throughout our journey for the same capability. For example, integration: we evaluated between Azure, Dell, IBM, and figured out which one was the best one that suited us. What that meant was it created a learning culture within the product team, and they were able to figure out the best solution for the problem that we were trying to solve.

This changing behavior helped the product team measure how good or bad their product was doing. What we did is we built an application assessment, a self-assessment questionnaire, which the technical lead and the product manager would sit together every IP sprint and go through the questionnaire and fill it.

What that dashboard generally would provide is a RAG status of where the technical debt is and where the product team should focus next. It could be application telemetry, it could be logging, it could be something to do with performance, it could be bugs that need to be prioritized. That was the first thing that came out.

The second one was to do with recovery. When I say recovery, typically what used to happen in a waterfall deployment was you make a change to production. If it works, it's all good. If it doesn't, the immediate thing was, let's revert back to a fixed state. What we tried out with this was we always had a fix-forward mindset. If something goes wrong or something didn't go as per the plan that was thought, the way forward was always find the problem, fix the problem, so that you don't revert back to the old solution.

We did that and there were a couple of challenges. Then we started doing something called lean deployment, which is basically dark releases. We started implementing changes which would enable features through a toggle or a switch. What that meant was the code was running in production in dormant releases even before the actual code was used by the business. This enabled us for a better testing cycle and identifying any core issues that we might have encountered by enabling the capability directly into production.

Throughout a year, year and a half of product development, what we found was gradually the team started adopting the CALMR approach, which was always to do with a continuous learning cycle. The team started to share the responsibility. The culture changed, the mindset changed from waterfall to agile software delivery.

We looked into opportunities where we could automate, whether it is infrastructure, automation of code, or automation of even testing, which enabled shift-left testing practice. We started creating reusable assets to share with the other product teams within the ART, or reusable assets to use within the product itself.

One example of that was authentication of the user. A small module was created by the product team, and as we started rolling out more APIs, the same module or the same code was used across multiple APIs, thereby reducing this delivery time, because essentially the capability was authorizing a user or authenticating a user, and the code was written by one product team and shared across everyone.

Then the other thing was, as I told you, more to do with dark release implementation. We started measuring our product every PI for the application maturity, which would give an understanding of how good the application is, going through the application telemetry, looking for call-outs from the performance aspect, and then factoring those things into our product refinement for the next PI. The dark releases enabled us to release code into production and reduce the risk of failure.

Imagine that a bunch of Lego is provided to a kid. A kid who has already played with Lego would probably sort different Lego by color and size, and probably might end up building a nice block. If the same kid has gone through the exercise of building blocks, or if he is given an instruction sheet of looking into a certain type of brick and building a house, the building-house activity might be much quicker.

What we did through this mindset was, we had multiple product teams within one agile release train, ART. We started evangelizing the engineering mindset, sharing the best practices across different product teams, like doing a health check of collaboration. What that resulted in was improving trust, and the quality of delivery was much better.

A few of the examples I already said were that we shared documentation on patterns and processes, and we always forced all the product teams to go through empathy mapping. The quality of throughput was much better across multiple teams, and we started sharing our learnings across different product teams.

Continuous everything is a mindset. All the tools that we get on a CI/CD market are just tools. They can just help you identify the gaps in your process or can provide feedback on changes that you have done. But if you don't embrace a continuous learning mindset, those tools are of no use.

Let me just talk about what we actually tried to do as part of the product team. We had two product teams. One product team was responsible for building the inbound planning system, and the other product team was responsible to build an outbound planning system. We are a retailer. What we do is we source merchandise from different vendors. It comes to our warehouse, and from our warehouse it is being shipped to the stores.

The first part of the problem was planning of those merchandise into the warehouse, and the second part of the problem was how those products would be shipped from warehouse to the store. Both of them used to be an Excel, Access, and .NET 2.1 system. It was a combination of all these things, which were actually helping the business systemically capture the movement of products between different points in the supply chain.

As part of our modernization, it took a good year, year and a half for us to provide more capabilities and build both the two products. The result was that we were able to move the merchandise from vendors to store in less than 50% of the time. If you think that a particular T-shirt takes 20 weeks for us to move from point A to point B, point A being a vendor and point B being a store, after successfully implementing all the capabilities for both the planning part of the supply chain network, we were able to do that in half of the time, which is 10 weeks. So from 20 weeks to 10 weeks, which meant we were able to provide better products on time, better customer choice to our customers.

Individually, each team could work smarter. Everyone could do empathy mapping. Everyone could adapt to the CALMR technology. Every product team could have automated as much as possible, thought about dark releases, implemented logging and telemetry, thought about performance and scalability and things like that. But if the inbound planning team and the outbound planning team didn't collaborate well, the whole end-to-end process would have been not as efficient if the collaboration was not done.

That's the key. Working harder here, in this context, is understanding the different silos and the different boundary between the product teams, and trying to break those silos to contribute for a bigger goal. That's what the team did.

I would just like to thank Gene and team for providing me an opportunity to talk about our journey. This was our mantra: don't panic, think big, which is have a strategic thinking, start small, start with the smallest offering that you can do, and scale fast. Thank you.