Today, I want to share with you the journey that we've been on from an IBM perspective. I'd like to kick off my session with a short video with some of my colleagues back at the office, and sharing with you some of the benefits that we've gotten from an IBM perspective by using DevOps practices. So can you hit the video, please? Hello, I'm Scott Kinane, and I'm the director of IBM Service Delivery Application Development Organization.

I'm Carl Krenzel. I'm a distinguished engineer at IBM. I am the director for the Watson Cloud Technology and Support team. My name's Doug Cox.

I'm the vice president of Smarter Cities. Hi, my name's Debby Edwards, and I'm the vice president of development in the Rational Software Group division. As a large organization and one we built through an internal consolidation strategy, we really had three key problems we needed to solve. First was a standardization problem.

We had all the teams off utilizing their own tools and their own processes for how they delivered, and so we needed to change that and get everybody using the same tools and using the same processes. The second was a collaboration and visibility problem. How do we get the teams to collaborate closer with their stakeholders for what they were delivering, and collaborate closer with each other to share best practices? On the management standpoint of visibility, how do we understand what's going on with the technical teams so we can get them the best help that they need?

And then finally, velocity. So how do we go faster? How do we help the teams deliver their changes to the business as quickly as possible, so we can get their feedback and make sure what we're delivering is the most important business values? By systematically going through our processes and eliminating waste, removing manual steps out of the process, we've been able to shift our profile of how we're spending our development resources from being around 58% of our development resources focused on innovation, to now, we have almost 80% of our development resources focused on innovation, which is significant.

First, we've been able to reduce the time of our updates by 94%. We've also been able to shorten our maintenance windows by 50%, which makes a big difference to our existing customers. We don't throw something out there and then get feedback that we deal with in a long requirements process. Six months later, they get the feedback.

The next week, we're rolling a feature out or a change in response. From the standardization standpoint, we've been able to take processes like our boarding process to bring on new engineers and take that from a week timeframe down to under a day. In fact, we just brought 15 new developers onto a program, and we were able to make them productive in a single-day timeframe. In fact, we've had one of the developers came up and told me that with this approach, he actually feels more like he's working at a startup than a large company.

Much of what we've been able to accomplish within Rational and the practices we've been following, we're undergoing an IBM transformation to adopt many of these practices across all of Software Group, as well as our System Technology organization and our Global Business Services team to really bring the value of the speed, quality, traceability to the entire IBM company. Okay, so as you can see, as an IBM corporation, we really are seeing the value of DevOps. And as you heard, themes through each one of those sessions is really about speed. And why is speed so important?

I'm sure many of you guys are seeing the technology shifts and experiencing the technology shifts that we are from an IBM perspective. We call these the CAMS in IBM, cloud, analytics, mobile, social, and security, which is really driving a rate of change that we've not seen in a long time from an industry perspective. And the cool thing about it is that as an IBM company and as many of our clients are embracing DevOps practices as a way to address the speed, quality, and client value profiles that are really needed to address these new shifts in the market. So it's a really great opportunity for us as the IT providers and for you as IT providers within your own companies to really help address business needs.

And I think Jenny said it well. Many of you saw our results that came out from third quarter this week. We are in a major transition as an industry and a major transition as a company. And this is part of what we do.

We're a 100-year-old company as IBM, and we have continued to reinvent ourselves over and over again, and we're in a significant process of doing that, and the key thing is speed. Our definition of DevOps actually is quite similar to the comments that Sam talked about and what I've heard many of you say this week, is that it really is focused on how you bring people and stakeholders end to end in the process together to make things happen and make them happen quickly in a very collaborative way. We really talk about what we call four different entry points. One which is around steering, making sure we're making the right business decisions, putting the team's work in the context of the business, and steering them to successful outcomes.

Focusing maniacally on things like development and test and how we can be super efficient and fast and work together without any silos. Bringing in the operations teams and being able to deploy all the cool stuff that development and test teams are doing into the hands of our clients very quickly so we can get feedback and ensure that we are building the right things and adjusting and changing as we go through the process. And as you can see, Jenny again comments that DevOps is the way in which IBM is going to achieve that, and we see many people in the industry doing that as well. So it's an exciting time to be a development leader in IBM as it is, I'm sure, for many of you as well.So what I'm going to talk about today is I'm going to dig a little deeper now into some of my personal experiences.

As you can see, IBM Software Group is a huge organization. We have over 40,000 engineers spread all over the world, and it's a big task to be able to roll something out to this large of a group. But we're in the process of doing that and in terms of adopting DevOps practices. As a matter of fact, you guys may have seen quite a large IBM group here during the week, and many of those folks are people in our DevOps center of competency that are responsible for rolling out our DevOps practices across this organization as well as the broader IBM, and they were here to learn and get perspectives from all of you.

And it has been very beneficial to them. What I'm going to reflect on is some of my personal experiences that I've been doing because I've been doing and leading software development teams for over 20 years in IBM and in some of my recent experiences as we have transitioned from Agile to DevOps and some of the practices and things that we are following. So first of all, I wanted to sort of start with and give you guys a sense of what we are doing. So in my role, I am responsible for building DevOps tools.

And so one of my key philosophies is we use our tools to build our tools because if we are following good practices and our tools help us to be effective and we're building those tools for those scenarios, they will work effectively for you. And we've been doing this in a journey over a number of years. As you can see, back in 2008 is really when we started this particular effort that I'm talking about, which are our Jazz products, or you might heard of them as our collaborative lifecycle management tools. We were moving at a reasonable pace for the time and that was needed in the market.

We were delivering our products on a yearly basis. And as we saw the changes and shifts in technology, and we recognized that we saw the cloud as becoming a key delivery model for us, we knew that we had to move faster. So we started a set of work to systematically go through and examine all of the work that we were doing within our team, all of our processes about the way that we worked with our business stakeholders, how we were working with our clients, and have systematically gone through and examined each one of these inhibitors and impediments and friction points and eliminated them over a period of time. So over this period, we were able to go from 12-month deliverables down to three-month deliverables, which is significant for an on-premise product within a company the size of IBM with rules and requirements that we have.

And we also now have been able to, through the innovation dollars and efforts that we've freed up to be able to enter the cloud software as a service market, as well as a platform as a service market. Many of you may have heard of something called IBM Bluemix, and our Rational team is a key part of that. We are providing the DevOps services there. So this focus on efficiency and removing waste has demonstrated itself through real business benefit, both as our delivery and part of IBM Bluemix, as well as new invention in other areas as well.

So one of the key things that people talk about a lot is they look at something like this. They say, "Well, Debby, good, you can go faster, but is that really your business metric?" Well, at the end of the day, we're a business and we want to make money. So during this time, not only were we focused on speed, but we had to focus on quality and making sure that we're providing our customers what they want, and therefore increasing revenue and customer deployments as well as customer satisfaction. And those are also other key metrics where we have continued to do well.

The other piece of this that was also very groundbreaking from our perspective in IBM is that when we started this effort, we created a developer community called jazz.net, where we interact with and share all of our work with our clients. And this is all of our work that we're doing. You can see our backlogs. You can see the test work that we're doing.

You can see our dashboards. And we host our usage of the tools in jazz.net as well as share that capability with our clients. So that was a real culture change for us, and we call that transparent development, really based on a lot of the sort of principles of open source in many ways, but done in a commercial way. So now based on all of these experience that we had, I've sort of come up with what I call five best practices that I see that have brought us value within the Rational organization.

And I have a blog where I talk about these and share a lot of details. But what I thought I would do today is just sort of highlight some of the key practices and lessons learned in each one of these and share those with you. And I'll have a link to my blog and stuff at the end of the presentation. So first of all, I would have to say that one of the most significant challenges that we have in a company the size of IBM was making decisions about what we were actually going to do and be able to stick with it.

Okay? We would deliver on a yearly basis. We would literally spend months looking at the market, prioritizing our work, trying to decide what we were going to invest in. And in order for us to move more quickly and be more responsive, we had to change that.

And we did a number of things in terms of looking at our efficiencies and things. But the bottom line, what it was is we brought everyone to the table. We brought our product owners and product managers, our developers, our development leaders and our support folks and our design folks, and we sat down in a room and we decided these are our key business priorities for the next year. We, as the senior leadership team said, we basically agree with that and we are going to allocate X amount percent to this strategy, this strategy, and we're going to allocate X amount of percent to technical debt, which was also a very important aspect of this.

And then we had a framework in which we could work within. And then we proceeded toActually build a groomed backlog based on those business priorities, based on the size of the development work. And we developed a process and created an approach within the development team where instead of committing to work that we didn't know how to do, we actually would create an exploration phase within our sprint so that we could actually go off and understand some of those larger features that we needed to be implemented. So as a result of this, we now are at a point where we have a groomed backlog where we can make decisions at the end of each sprint, and we can pivot if we need to work on something different, if we have a customer need or a business change where we need to shift the focus of the individual teams.

Obviously, that's not something we like to do, but we have the flexibility to do that now, which has been a big benefit for us. I recall just how many times we would just churn over these decisions and remake them and make them again. And pulling together this sort of strategic product committee has really helped us a lot in doing that. Another area is continuously testing and automation and shifting our work to the left.

In a company like IBM, as I'm sure many of you guys, there's a ton of things that we have to be able to do to release our product. We support at least 10 different operating systems and databases and browsers, and we have national language translations and testing, and we have usability and accessibility testing that we have to do and security testing. And we have to maniacally focus on how we can automate that stuff as much as possible. Some can be automated, some cannot.

And it was really a lot of that, what I call the endgame, that was really requiring us to spend a lot of time at the end of our cycle. We show here how, in our yearly cycle, we would do sort of these four-week development iterations. We were doing some testing, but we weren't able to do serious production-level testing in these sprints for a number of reasons. We didn't have the right level of automation in our function test, and our builds were not stable enough, and we did not have the right level of automation in our setup and configuration of our larger system test environments.

So what we have done over the last couple of years is we've really focused on looking at every single thing that was manual within our process. Everything from how our tests are being created. We use things like JUnit, some of our own test tools for performance, but we've also focused a lot on Selenium and using a page object framework to help us because our tool is very UI-based. And so we had to do some investment in a framework that our developers could leverage.

And that framework itself has allowed us to more than double the number of automated test cases that we have. So that was a very significant investment, and we share that with clients. And we have a blog, and I'll help you understand where to go learn more about that if you're interested. So that was a very significant ability to give us the option to do our function verification testing alongside our development.

One of the other most significant aspects is that we used to have our system test done in the following sprint because we just could not keep up with the development team in our system test organization. So what we did is we went through and we created patterns and scripts which represented all of the setup that needed to be done for our more complex customer scenario testing, and we automated it. It literally used to take my team 30 hours to set up an environment to do the testing because we did everything manually, and we would have to set it up, break it down, reset it up. We had all the different databases, configurations, et cetera.

And by using patterns and automation, we're now able to deploy that automatically within three hours, and we're able to spend that 27 hours testing. So it allows the testers to be right in sync with the development organization, which has totally changed the game for us. We're now able to be ready to ship at the end of each one of our iterations because of these improvements in our process. And the other thing that we've done is we've really created a culture now that says everything should be automated, and it's a business decision or an exception if it's not.

Which I think is so critical because I saw a real shift in our team as they started to see the value of this and recognized that doing things manually just was not acceptable and really not sustainable given the level of speed that we were trying to achieve. Obviously, getting to this point was an effort that took a focus on continuous improvement. And one of the things I always share with the team is how critical that is and learning at the end of each one of our sprints and then planning for the next one to eliminate the friction points and the areas where we were less efficient than we should be. This is a view of what our delivery pipeline is.

I thought this might be interesting to share this. We have a global team, so unfortunately or fortunately, we can't just look across the table at our peers. We have to use collaboration tools which are built into how we work to share information, ensure that we're all in sync, and stay in sync.We do hundreds of daily commits and builds and deploys in our environment. We have our developers, as you can see, can launch from their desktop.

We have a build system, which we have completely and totally broken down and rebuilt from scratch to ensure we have appropriate performance of our build. We had to re-engineer our product. I heard Gary talking about that a good bit yesterday in terms of we had to restructure our product, make it more loosely coupled in order to get to the build speeds and a level of parallel development that we needed. We also have an automated BVT environment, and we have these quality gates that are automated as a part of our process.

So when a developer is ready, he's done his pre-testing, he's ready now to integrate the capability into the build, it automatically kicks off the BVT testing. If the BVT testing passes as green, then it goes through our continuous test, and you can see we have some unit tests. We kick off some of the system test build-up environments, and we do performance tests on a weekly basis where our developers can see as they check in their code if it has impact on performance. And then we have, once we get through this automated test phase, then there's some testing that we're calling interactive, which is manual testing, which has to be done as a manual step.

So some things that we haven't quite figured out how to automate, some of our usability testing, and some of the free-form testing that we actually want our teams to be able to perform. And I'll talk just a moment, a little bit more about performance testing. And then we deploy that into our jazz.net environment where we use it to develop our own tools as well as we make it available to our clients for you guys to be able to see it and use it. And also we have sandboxes and things like that where you can try it out.

So this has brought us a tremendous amount of speed. We went to the point where it would take us weeks and weeks to be able to deploy into a production environment, and now we can do it every other day or on a daily basis if we choose to do so. One of the other fundamental things that we built into this that we feel is very essential is security. So when our developers kick off a build, not only does it run JUnit tests, but it also does scanning of their source code, and I think I heard Gary talking about that yesterday.

Because immediately the developers get feedback if they are injecting patterns of code that cause security issues, and it gives them that feedback and it helps them learn what they can do. Then we also have additional security testing that's done on the app as it runs, and then we also have security hackers that go in and try to break our tools, our on-premise products, as well as our live running products that are available out on the cloud. So this has been an exciting adventure for us and brought us a tremendous amount of value and speed that we never thought we would be able to achieve. This is a view of what our continuous delivery pipeline looks like.

I thought you guys might just like to see it because I think it's really cool this sort of view of the pipeline. We use these in our scrum meetings. An organization of my size, which I didn't share this with you, but it's around 1,000 people. We have to have some mechanism to keep people in sync, and we do scrums and then scrums of scrums.

And these dashboards are used to help us with that. One of our scrum masters said just this very simple pipeline view here helped to save at least 20 minutes out of the scrum because it was very easy to be able to see where the build was in the process, and you can drill down. You can click on the entry here and drill down. So if you'd like to see it, you can go out to jazz.net and look in our live dashboard, and you'll be able to see this one, our latest build.

And then you can see here just some of the other information that we're using to manage our teams on a regular basis. The other thing that I found was a big challenge for us is that because I did not have a good web UI framework for doing automation testing, a lot of different things had sort of built themselves up in the development organization. They were kind of doing their own thing, and it was very difficult for me to get a single view of quality because those test cases would not represent themselves within the delivery pipeline. And that's an effort that we have gone through in the last six months in a very significant way, is to get to that single view of quality.

Experimenting rapidly. This is an area where we're learning a lot about how to do this better because we've really just become more of a presence in our software-as-a-service business. But one of the cool things that we did that it was sort of innovative for my team anyway, is that instead of-- We were creating a new capability called Quick Planner, which is a simplified interface on top of our tracking and planning capability. And instead of immediately going to market, we experimented with that in our software-as-a-service, which is called Jazz Hub, which is a part of Bluemix.

And we were able to get some tremendous feedback from clients around our Quick Planner, helped us much better understand the scenarios that we needed to support before we actually put that into our on-premise product. And that's a pattern that we want to continue to follow. We also are playing around with and doing some things with AB testing, understanding which areas are driving more traffic to our site and getting customers to open accounts and things such as that. And that's an area where we're going to continue to focus.

But the power that the software-as-a-service and living online allows you to do is significant in terms of experimentation and getting client feedback and starting small and building on top of that, which is exciting new opportunity for folks like myself that have been delivering on-premise software.One of the things that I alluded to, which is absolutely essential, and I believe one of the most important parts of things like agile and DevOps, is a focus on continuous improvement. I had someone ask me when I first got here, "How does it feel to be completed with your DevOps journey?" Or something to that effect. And I was like, "Well, it's never completed, right?" It's a journey, it's ongoing, and you constantly are learning about what you can do better. And this is a very structured part of our process.

We focus on what our business metrics are. We look at what's impeding us from being able to achieve those objectives. We develop actions. Those actions are prioritized in our backlog, along with all of our other business requirements and client requirements.

And then we book those in sprints, and we work those just as we work other customer requirements or market requirements. And this has helped us to significantly improve. One of the areas that I will say that is a challenge for us, and I think in any large company where you have pretty well-prescribed measurement systems, is this blame culture. We are still working to eliminate that from our teams.

We've gotten better, but people are afraid to speak up. They're afraid to say, "I did something wrong." And that is something, as managers, we are really working on that with our teams and will continue to do so. So now, those are some of the practices that we've implemented in my organization. I wanted to talk a little bit about some of the things that we're doing to scale this up to 40,000, 45,000 people, and then up to 300,000 people, which is the size of IBM itself.

So we've created a few things to really help with that. So we have our software group, DevOps Center of Competency, which is driving a whole bunch of education across IBM. We have a really awesome community where we're sharing information about our DevOps practices, success stories, and all of that. We also have a lot of education.

You can see here, I'm pretty impressed with these numbers because we've really just started this education in June, but we've almost had 800 executives and managers educated, which is, as you guys know, a central part of making this successful. Over 9,000 attendees at our different education sessions. And we have much more of this planned. So this is very significant for us.

And also, at the same time as we are implementing our DevOps sort of scaling-up activities, we're also implementing and focusing quite a bit on a new design process. I don't know how many of you guys have done any research into sort of this design first thinking. So in addition to operations and development and test and architects and all that, now design is becoming an essential part of how we are making our decisions about what we're developing our products. Instead of just talking about the features, we are talking about the customer experience related to those features.

And it's a really profound, different way to think about it because you put yourselves in the shoes of the customer. You don't talk about technology. You talk about the who, the what, and the wow of any feature that you're going to develop. And you create a set of hills which guide how the feature will roll out.

And so we've had an opportunity now to combine some of this design thinking and these design approaches with DevOps, which has been quite enlightening and very interesting. And just in July, we had a huge summit out in Austin where our entire design organization, all of our development leaders, our business leaders, and our technical folks all got together to really get a deep understanding of how we're going to go about doing that. So it's really exciting to kind of see these things start to come together as a business. There's a lot of information available out, even on our IBM website, if you want to learn more about what we're doing on IBM design thinking, and it's implementing a lot of sort of industry best practices.

So now I want to talk a little bit about some of the other things that we're doing within IBM, because it's sort of neat because a lot of times people think everybody in IBM works the same way, but it's not true. We're made up of a lot of acquisitions, and we're also doing a lot of things to encourage people to work as startups. So here's a good example. Bill Higgins, who is one of our SCSMs in IBM, he has created a real robust startup environment where he's got a small team room, and he's doing really cool things and delivering new capabilities from our Tivoli portfolio, our CNSI portfolio, out into the cloud.

It's called IBM Service Engage. And interestingly enough, Bill just took on a new role of figuring out how to bring IBM design thinking and DevOps together, and he's going to be leading that and become our evangelist for doing that. So if you haven't yet heard from Bill, you'll be hearing from him. Another area, you heard Carl talk a little about IBM Watson.

I'm sure many of you have heard of Watson. It's sort of our marquee new brand of capabilities that we're introducing in the analytics space, and they are practicing and using DevOps capabilities and achieving a tremendous amount of value from those. And then finally, this is an interesting one, and this is actually one that I was involved in myself, is the transformation of the CICS transaction server organization, which is a z/OS-based team, to becoming a very agile and DevOps team. And you can see the quote here from Steve Mills.

He actually focused and presented this at one of our marquee conferences. So my point is, you can do it. You hear that there's a lot of pushback, but if you get the teams interested and excited and they have a compelling reason to act, there's a lot of value in getting these teams into being more responsive.So I've quickly gone through. I have spent as much as two days talking about this topic.

And hopefully, I just wanted to give you a little bit of the sense of the things that we have going on. This is sort of the process that we've been using. I talked a lot about the importance of business objectives, about assessing where you are, about identifying those pain points, and then executing and measuring against eliminating those pain points as you go through the process. And then in terms of areas where I need help, excuse me.

As I've said, we're a team that's moving from more of an on-premise sort of mindset to a cloud mindset. We've learned a ton, and we've got a lot more to learn. So anyone that's going through that, I'd love to hear from you. Enterprise scale, obviously I've done 1,000 people.

Now we're going to 40,000 people and even beyond that, so any hints or tips that you guys might have on that. And then, one of the other things is that I feel I wanted some tips from anyone that might have a perspective on how really to empower the leaders to take ownership of it and run. I feel people are still a little, I'll say, worried and timid about it, and afraid to take a risk. So those are the areas where I could use some input.

I have been around all week. I'll be around shortly after this, but I'd love for you guys to stay in touch. I've put my information there. The blog that I have listed there is information where I deep dive a lot more into each one of those practices and share the measures and specific cultural changes, process changes, as well as tool changes that we had to go through.

So I thank you very much for your time today. It's been a pleasure meeting many of you this week, and I thank you so much for the opportunity. Thank you. Thank you, Debbie Everett.

Thanks.