Shift Left! Build it Right! The Discover DevEx Journey
The path to building amazing software starts with empowered and autonomous engineering teams who can build faster. However, the increase in landscape of tools, systems, technologies, compliance, decisions, and tasks that developers must navigate leads to decreased productivity, being forced to make unexpected handoffs and decisions, interrupting the flow of work leading to Developer Fatigue.
Chapters
Full transcript
The complete talk, organized by section.
Andrew Duckett
Hey everybody. Welcome. My name is Andrew Duckett. I'm joined here by Javier Alvarez, and we'd like to share with you today our story about how we use Golden Paths to improve the developer experience at Discover Financial.
You can ask us questions. I think we'll have some time at the end for that. But if you would like to hit us up offline, we'd love to have a conversation with you about how you can use things like Golden Paths and platform engineering to improve developer experience. We're down to talk about that. So you can hit us up on LinkedIn.
Okay, so I'm going to start off with a bit of a story. I'll talk about how we got here, some of the problems that we face at Discover with our developer experience. And then I'm going to hand things over to Javier. He's going to talk about how we use Golden Paths and how we can measure them and determine that they're helping improve our developer experience.
We're going to talk about some of the things that we started with, how we use an open model of working to gather feedback from the thousands of engineers, and how we're able to use that model to scale, quite frankly, a very small engineering team to serve the needs of these thousands of developers. And we're going to talk about some of the things that we learned, some of the paths that we really hadn't even considered until we talked to people. And then we'll talk about how we wrap that up into an experience that we think is pretty impactful, and what we think is next for us.
So before we do that, though, I want to take a step back, and I want to take a minute and think about when we ask a developer to create one new thing. It could be a feature, a bug fix, a story, whatever you use to craft that thing. This is all the things that goes on in their mind every time. So every feature, every fix, every change, they're designing for any number of these things. Sometimes well, sometimes poorly.
In the past, we really leaned on the separation of dev and ops. We leaned on those ops teams to take on some of these responsibilities. So they handled a lot of these non-functional requirements, these concerns with planning and scheduling, and all of the design elements that need to be put in place. And I know it sounds like I'm championing that world. We're a firm believer that creating autonomous DevOps teams is really powerful and important.
But I think one of the things we don't spend a lot of time talking about is, as soon as we do that, we push all the mental load and the cognitive knowledge that they need to have onto that one development team. And so what we did is we created a space where they had to take on all of these concerns. Whether we liked it or not, they do the best that they can, and they try to make their lives a little bit better.
And so to do that, what we noticed at Discover is each little engineering team created their own little knowledge silos. So they had, whether it's a wiki or a SharePoint site or a folder or Confluence, take your pick, it doesn't really matter. They created little knowledge bases that were siloed within those individual teams, and they used them to make themselves more efficient, more effective. And that's great.
One of the things that we noticed as we started combing through all the articles and the content and the documentation the teams were writing is that it was actually pretty good. The teams were documenting things to make their lives better, and that's way more important than just random documentation. And so as we started to comb through it, it wasn't a matter of, "Oh, we need to reinvent that." It was a matter of, "How do we scoop all that knowledge up, sift through all the duplicates and the wasted effort, and really organize that into a central place?"
So at Discover, we created this place called the Discover Technology Academy. It's a collaborative space to share content, articles, tutorials, training, knowledge, learning paths, really anything that you would use inside of Discover to build software at Discover. We think it's a good place to not only collaborate, but you can upskill, you can learn faster, get started faster. And so it's a fantastic place to gather up all that knowledge into one place.
But then we noticed another problem, and what we noticed was that these individual service teams were all documenting their systems very well, but it wasn't really a stitched-together journey. So the developers were still hunting and pecking and searching. And so we took a step back and we thought, "Okay, well, maybe we could use the same place and stitch this documentation together into an end-to-end flow."
So I'm going to hand this over to Javier, and he's going to talk about how we stitch these disparate sets of documentation into end-to-end journeys that a developer can use, quite frankly, to create something new at Discover.
Javier Alvarez
Cool. Thank you, Andrew.
Hi, everyone. My name is Javier Alvarez, and I'm a principal application engineer at Discover. Thank you, everyone, for being here.
So, as Andrew mentioned, DTA is our internal platform where teams can contribute with their documentation. So as our internal developer platform evolved, so did the amount of contributions, as well as the different opinions about building things at Discover.
Having many contributions is amazing, but to have many opinions and different ways to build something could be overwhelming and sometimes not very efficient. So although this presented a challenge, it also offered an opportunity to help reduce the cognitive load of having too many options and help developers in their journey in creating amazing software. Hence, we thought that something like Golden Paths could help.
But what is a Golden Path? Well, as stated by Spotify, a Golden Path is a step-by-step tutorial that will walk you through the opinionated and supported path to build something. So at Discover, our Golden Paths are the source of truth to help developers accomplish a specific engineering task.
So these paths should be to create something or to build something, such as an API or a React or maybe a backend component. So it could be anything that we wanted to create. And so we think of these as higher-order workflows or pipelines consisting of a collection of step-by-step tutorials that we stitch together into comprehensive developer journeys that we call Golden Paths.
Golden Paths can help us in the following ways. They can simplify getting started to minimize the cognitive load, so we can stop reinventing the wheel and let developers focus on using their creativity for higher objectives. They can also turn rumor into recommended, so we can stop rumor-driven development, where the only way to find out how to do something is by asking your colleague.
They can also decrease fragmentation and inconsistencies by making sure that the tools and services that are provided on the path are aligned with the organizational standards and best practices.
So our team has been working on creating Golden Paths since last year. And now that our Golden Paths capture accurately the truth of how the process really works, our team has been actively exploring our opportunities to automate and streamline parts of these processes through the building of apps and ready-to-use templates.
So our team is planning to pilot a proof of concept for a backend component generator that we believe will offer the following advantages. It'll help us to save time, as it'll reduce the time to "Hello, world" to just a few clicks. And it'll also help us to maintain consistency, as it'll make it easier for developers to build components with organizational standards and best practices baked in. So it'll also reduce errors and hopefully boost productivity.
How do we create Golden Paths? Well, our strategy for creating Golden Paths is guided by the open source principles of trust, transparency, and collaboration. So we need developers and the community of developers at Discover to help and participate in the creation of these Golden Paths.
So we lead with transparency. Without transparency, it's really hard to grow a community. So we work to provide meaningful context about where we are in the process, how we got here, and where we are going. Having a consistent Golden Path that provides the materials and information necessary for teams to do their best work enables these teams to collaborate more effectively and build upon each other's ideas, which can lead to better paths.
So as part of building this developer community, we want to empower developers to help themselves and participate. This starts by them following Golden Paths, completing tasks, and realizing if there is something that doesn't work, then submit a fix in the process to make it better.
So we want these Golden Paths to be supported by a consensus of members in the community, where we foster productive dialogue about the best way or tools to complete a task.
As we collaborate with teams and test different Golden Paths, we have been able to identify new requirements and opportunities for creating new Golden Paths. So following is one example of that collaboration.
When teams decided to adopt some of these Golden Paths, specifically for the backend journeys, the community recognized a need for a developer experience or developer journey that would help them to mitigate and resolve open vulnerabilities in their projects. This awareness prompted a collaborative initiative within the community and the cybersecurity team, which resulted in the creation of a new Golden Path that consolidates all these standardized practices in one place that are aiming at effectively managing and mitigating the open vulnerabilities in their projects.
But creating all these Golden Paths by ourselves is quite challenging. As you can imagine, the developer journeys at Discover do involve many different products, and each product team will have their own product page with their own standards and documentation that they maintain.
But we recognized a need for a single source of truth for developer journeys. The challenge was that as the Golden Paths team, we cannot be experts on every area and keep all these paths current and valid. So to solve this, our team created a contribution model that provides a framework for distributing the responsibility that comes with the cycle of Golden Paths.
But to implement this contribution model, we needed buy-in from a leadership team and a sufficient level of technical expertise and experience. And despite these challenges, we were able to secure sponsorship for a Golden Paths team, and through active collaboration with the community, we're driving the evolution of the product.
An example of this collaboration model could be around teams' responsibilities. So you could say that service provider teams, which are the experts in a specific product, they will be responsible for their individual steps of their product, whereas Golden Paths team will be responsible for the overall experience of the journey and any gaps within that documentation.
Golden Paths team will also evaluate and archive Golden Paths that are outdated based on technology strategy, product direction, and community needs. And for critical services such as API management or secret storage, it's important that information is kept up to date by the service provider teams.
Having this contribution model has been really helpful, as it provides clear boundaries of where the responsibility lies within the stakeholders. And it also helps us to provide an information architecture of Golden Paths that enables our product to scale as other teams start to collaborate and add newer capabilities.
As with any other initiative, we wanted to assess the impact of Golden Paths and make sure that the outcome is valuable to our customers. So before Golden Paths were implemented, an engineer looking to build new things at Discover would need to go and visit several product team pages that had different standards and different forms to be submitted. And then they would ask a colleague how to submit those forms. And if any form was submitted incorrectly, it would cause delays in the process.
But now we have launched seven Golden Paths, and we have estimated that an engineer can save up to 10 hours per component created by following these Golden Paths, as they provide meaningful context and detailed information on how to build specific things.
So what's next in our journey? Well, as we move forward, our next step involves implementing an advocacy framework for Golden Paths. This advocacy framework will be aimed at fostering increased adoption and encouraging active contributions within the community.
As part of this advocacy framework, we're planning to engage actively in outreach programs, learning modules, and collaborative partnerships with different teams. We believe that by being able to implement this advocacy framework, we can continue to create awareness of Golden Paths, and we can also drive adoption of Golden Paths. So we can promote a standardization of patterns and hence reduce risk. But we will also be able to know our consumers and be able to work closer with them so that they can also collaborate with us and continue to contribute, so we can keep these paths current and valid.
To summarize, our key takeaways are: reuse and don't reinvent. Use proven solutions as the foundation for developer journeys.
Move towards open models of working. This enables teams to collaborate more effectively towards a shared goal and develop shared expertise, which can lead to long-term sustainability of the path and better collaboration and flow.
Third, leverage the communities. Prioritize the needs of the developer community for creating new developer journeys, but ensure that these ideas are aligned with the organizational strategy.
And finally, start somewhere. Improvement doesn't have to be an all-or-nothing approach. Starting somewhere, even with small changes, can lead to big improvements over time.
I hope that our experience in this journey can inspire others to embrace the power of collaboration through innovative platforms. And thank you for listening. We would like to hear from you about how you're building Golden Paths and your developer experience also.