Ask the Speaker Panel

Moderator: All right. Hello, everyone. We're just about to get going. We're waiting for a couple of chairs and one more panelist, but we'll get going because as the speaker, it's very important we maximize use of your time. Hopefully, you're all familiar with this slot. We put this together to avoid the situation at conferences where a speaker comes off the stage, there's a queue of people waiting to speak to him, the first one in the queue takes two minutes, and it breaks open. So I'm sure you've all been at the end of that queue.

So I'm going to try and make sure that each of you gets a chance to ask any question you've got. We've had three great talks this morning and we have the speakers here.

We're recording it as well, so I'm going to try to repeat your name and where you're from, ask a quick question, the speaker will try and paraphrase the question, and then answer it. So you don't have to walk down and have your mic. Just speak as loudly and clearly, and that'd be great.

All right, so we've had three great talks this morning. Anyone, is there a question for anyone on the panel? Who wants to start us off?

Q&A

Q: Yeah. So I'm Brett from Health. When you say shared services, sort of the enterprise typical services — mail, email — how do you handle those?

Jason Cox: So we have a bit of a combination at Disney. Part of it is by design, and that is that we have multiple segments, and by literally design, each one has autonomy in terms of the technology direction. So they have multiple CTOs, and within each segment there are even multiple CTOs. And they are all responsible for their particular group and the products that are in their portfolio. Part of the design was meant to unleash them from any bureaucratic latency that could inhibit them from getting products out to the market.

However, there's still shared services. There's still a need for that, and we have an enterprise technology team that Brian and I happen to be part of, that supports all the segments. It's a very subscription-based model in a lot of ways, where the segments look at it and say, "Hey, that makes a lot of sense for us, we'll subscribe to that." And so by that we can gain some economies of scale when it's more commodity-type services.

Where it's more engineering, sort of deeper-dive services like teams, we literally embed within the groups. So I have team members that are part of, like Brian, who's part of our studio support team. He manages the engineers that are embedded in the studio. And they are literally — just as you heard with Movies Anywhere — in with the Movies Anywhere product teams, part of their lifecycle. The cool part about that from a shared services perspective is that he can reach out and talk to his peers in Imagineering. They're all part of our same team in the shared services organization.

And they can share best practices around Terraform modules, as an example. Share code, configuration management, like we talked about in our consumer products interactive stores team. So sharing those great ideas is a little more fluid because of the way that embedded, but still back to the shared services team, is organized.

Other services — API endpoints and things like that — vary. There is centralized mail service, Active Directory. You can think of those things that we at the enterprise technology layer offer up. The majority of the enterprise does use it, but there are cases where they don't. Even on the network layer, they may create their own because they have very unique requirements and specifications they have to meet, and so they'll build their own so they can operate faster.

Topo Pal: From a Capital One perspective, we do have shared services, and I think any big organization will have shared services. It's the question of the balance between how much is federated versus how much is centralized.

There are a few reasons why you would want to centralize certain things, where you want better control — such as your code repository. You want to make sure that the people who are actually committing code belong to Capital One, and the pre-receive hooks can catch certain sensitive information before they actually get committed. Also, you can have code scanning done on your GitHub repositories, looking for things that should not be there in your source code and all kinds of stuff.

And then the other side of it is, let's say, Sonar, for example. It doesn't need to be shared. You can have your own Sonar, we can all have our own Sonar, and it'll be fine. So it's a question of how much control you want to put and why, versus how much shared services you want to just force upon the teams.

I think we always keep this balance to the extent that we say, if we have to provide a service as a shared service, we better be as good or much better than an individual instance. So it's not like we are forcing it. We are standing up as a service, and we are behaving like we are a service provider and you are our customer. If you like our service, you come to us. If you don't like our service, try to build your own, and we'll learn from the way you build so that we can bring some of these best practices into the shared service so that other people can benefit.

---

Q: Justin from John Deere. You allow people to go do their own thing, which in the past — I know the term you were wanting to use is shadow IT. How does each of your companies manage those different portions that are going their own way with their provisioning or appointment systems? How do you manage it? Do you make sure they're doing the right thing, or do you basically say, "It's up to you to manage it"?

Jason Cox: Shadow IT — I would say in a lot of ways we've institutionalized it. We encourage it. Part of when you say shadow IT, I think what happens there is the thought that we need to control IT. And yet what we have found is that if you're imposing that control, you lose some flexibility. You lose agility at the edge. So you want to empower the edge. We have, like I said, shadow IT on top of shadow IT. We have some shadow IT organizations that have shadow IT organizations — no joke — underneath. And again, it's for the purposes of delivering the product, that experience, that content.

Now, the interesting thing, though, is that there are some things that are super important across the company that have an all-stop button — an EPO, if you will — and that's security. So we have global information security. Super important. That doesn't mean that the segments can't build their own. They do. In fact, every segment has a security organization that shadows our main security organization. It's because they understand their particular use case. It's considerably different if you think about content protection versus safety on a cruise ship. But at the end of the day, security is important. Our guest security and safety is number one. So getting that across all the enterprise is important.

And so that stays at the top layer, where a lot of the other things — like mandating a particular way of using the CI pipeline or mandating a certain source control repository — there is enterprise services, but you're welcome to build your own.

Topo Pal: Shadow IT — everybody has that. I think the point is you need to find out why those shadow ITs are there. And I think, in my mind, when I talk to developers, I find that the shadow ITs are created because that particular service or that particular quality or theme was not available as a shared service. That's why they are kind of forced into that shadow IT model.

So the point is that, yes, shadow IT will pop up if that particular development team finds that I don't have anybody helping me, so I'll build my own. So we have communities of practice, reach out, come to the table — kind of a culture where we actually know when shadow IT pops up. We don't call that shadow IT. That's what we call emerging technologies.

So you don't like this particular CI tool? Okay, maybe it's not for you. Do you have a better choice? Yes, go build. See how it feels. See if it meets all the requirements from compliance and security and all that. And if it is all okay and we are all onboarded, maybe some other teams will benefit, and we actually take that as a shared service. So yes, we know about the shadow ITs, but they're not in shadows. They are actually emerging.

---

Q: Chad from C.H. Robinson. Kind of a two-part question. Do you have any part of your pipeline where people can go in and define the process? And if you allow developers that power, do you basically have to audit them for the process to make sure that it's the same as it was?

Brian (Disney): That's actually a very good question. So yes, we do use a Jenkinsfile for defining a lot of our CI processes. We're actually moving over to GitLab now. And part of that process is you build that process into your CI pipeline. There are certain checks that actually have to happen. So at the end of our pipeline, there's actually a script per se that actually makes sure that certain tests are actually being run. That's how it allows us to kind of promote and safeguard from a developer being able to remove a test that actually has to happen. But a lot of it comes with trust. We actually give them full access to that file for them to actually change the CI process as they see fit, as they're adding on more features. You have to have a certain level of trust to understand that they're not going to remove a test that's actually required.

Jason Cox: Just real quick context on that, though. We have a lot of different models, back to the whole point I was making about the very distributed teams. In the particular case you're talking about — for Movies Anywhere.

Topo Pal: Yeah. So in terms of Jenkins — Jenkins 2.0 pipeline as code is new. And the people who have been doing Jenkins pipeline for a while have a mix of things, which is: number one, local configuration within Jenkins; then local configuration mixed with the workflow plugin and Groovy script; and then the Jenkinsfile, which is completely declarative. We have all three variations.

If we take the Jenkinsfile and it goes to GitHub, any change is peer-reviewed. You cannot merge to the master branch. So if you are taking any tests out, we catch it there. Number two, if it is a mix of local configuration or local Groovy files, then my point is that we need to detect if somebody's actually taking out — for example — a Sonar scan from that. And you cannot do those things from being inside Jenkins. You have to actually detect that by being outside of Jenkins, which means collecting your pipeline data and seeing if the unit testing actually was done. Do I have that data? The static code analysis — date timestamp, it came from Sonar, which said that it happened before your build — so that means you didn't do it for the current build, and you kind of stop the pipeline.

---

Q: Scott. I just have a question in regards to the human factor here with DevOps. How much pullback or resistance did you have with developers being responsible for their code when it got out there? Because in a traditional outfit, one of the engineers may have a problem.

Moderator: So Jason, Tisson — the question is, how do you drive DevOps transformation and articulate the value of it to the business, especially when it's already siloed?

Tisson Mathew: I can tell you a story around it. I was the Director of Engineering at Amazon. We had literally thousands of teams who were fully enabled to be autonomous. We had somewhat of a baseline engineering stack — one engineering system — which is mostly available now in AWS with a variety of different services.

And then I went to a healthcare company, fairly large size, $10 billion revenue, about 700 engineers or so, but very siloed. Architects did architect things. Developers did developer things. Infrastructure people did infrastructure things. And my first try was to create one team that is end-to-end — that can plan, build, deploy. It was such a nightmare to even do one single team.

I think organizational transformation starts with that first team. And I can tell you from my experience that first team was not a success initially. We tried it again, because they were not getting to a level of maturity compared to the siloed organizations. The security and infrastructure team was so solid in deployment — they were slow, but they were good at what they were doing. They were scanning all the code. They made sure all the checklists were there. They deployed it right. But it would take them a week and a half to do it. And now we integrated teams to do that within a very short period of time, and to reach the level of maturity that the enterprise is looking for — that is a challenge, because these silos are inherently good at that particular thing.

It took us almost six to nine months to even get to a level of maturity for this team to take on the whole thing. We first did planning and dev, and then deployments were still manual by the infrastructure team. And then we did a lot of education on AWS and all the tools, to get to a level of maturity. So it takes time, but I would say don't give up on it. Because it's sometimes hard to articulate the value of it to the business — why you've got to spend all this money. You can't just say that other people are doing it. It's like, look, Capital One is doing it, Disney is doing it. But they'll say, "Why you?"

But overall, that organization has now transformed. They were at AWS re:Invent — it's happening two or three weeks from now — they are actually doing large industry sessions. There are like 800 people in their DevOps, security and DevOps in healthcare session. And they are the ones who are doing it. They were the ones who had resisted this two years ago, and they are the ones now up on stage giving the presentation. So it can happen.

Jason Cox: I think for Disney, Jason and I would probably take your question and split it into two. For at least my team within Disney, my engineers are directly embedded with the devs. We're pair programming with them. We're in the sprints. We are working with them with the code, understanding how they're coding. And in return, they're also understanding how infrastructure works, understanding how we do our deployments. They're also on call. They're not just tossing it over the wall to us and having us deal with it. They're actually on call with us, and we do more of a separation of concerns when we actually reach out to the dev. I think Jason can probably talk about our journey on how we got there.

And I think that's kind of covering more of where I see that we're headed, which is the T-shaped teams. There can be teams that are formed that are full stack and operate that way. A lot of times we have more of this matrix effect. You form a T-shaped team that is operating that product. The T-shape — everyone gets that — from the horizontal, maybe you understand the whole area, and you have a depth of expertise. So you have individuals on the team that have an expertise in the language, or have an expertise in the infrastructure network, et cetera.

We still have teams that are very much the traditional way, and will probably be that way. I mentioned one kind of in the talk where when we launch a product to our parks and attractions, that's intended to run for 10 years. So from an iterative development process, it's a challenge for us. And we have operations teams that have to be there and are reacting to our park operations — they're separate from the development teams who are building the code. So things like that will continue to exist, I believe. We're just trying to blur the lines. Understand more of their space and bring some of that feedback back into the next project as those loops happen. But a lot of the digital transformation and digital products are going more the path that Brian just mentioned.

---

Q: I'm Jonathan from Honorati. One question I have is around test data management and testing. Do any of you do version control on that? And second, how do you create the metadata link between the user story, the app code, the test data, and the test?

Topo Pal: Yes, as I said, everything goes to source control. That includes your test cases, whatever they are — Gherkin script, Selenium, whatever the case is. They're source-controlled.

Number two, test data management. We use some popular test data management tools, and the test data shows up in the configuration — in some kind of XML or JSON. Test data itself is not source-controlled, but the scripts are source-controlled, meaning pulling data from production, wiping out the unnecessary stuff and putting it in — that is source-controlled.

Apart from that, the connection between user stories and test cases is a problem. It's a problem because you need to make sure every Gherkin feature file has the tag that relates back to the Jira or whatever story. And it's a question of discipline within the development. It's very hard because that's where the actual human touch comes in, and it's very hard to automate those kinds of things unless you go into some kind of natural language processing of your test cases. That's not going to happen soon. But yes, that's the human discipline that we need. It's very hard to implement that enterprise-wide. You can do it localized in smaller teams. But at least from my experience, yes, we got some success, and yes, we have some failures, too.

Tisson Mathew: I'll add a different dimension to it. How many of you know about behavior-driven development, BDD? A few. Okay. So I have experience with a few teams that actually went to BDD for exactly the same reasons you're talking about. The product managers were really writing Gherkin — which is the BDD way of describing how the behavior of the application would be. They're writing requirements almost like code. And product managers are writing it, and they know — like for example in healthcare, we have to use de-identified data for testing. We can't really use real data. So how do we de-identify the data? How do you manage privacy when it actually goes to production? So they have to go both ways.

BDD really helped them to understand the behavior of the applications, and then tests naturally came with it because if you write BDD scripts — we use Clojure and Cucumber. There is a book called Growing Agile — if you actually Google it, it's a very nice book about BDD. If you were writing highly secure, test-driven, behavior-driven applications that are very sensitive, dealing with a lot of sensitive data, I would strongly recommend taking a look at it. It's not a fit for everybody — there'll be some people who say, "This is overkill for us." But for some of the teams who want to do BDD, it's a great way to get started.

---

Q: Steph from QA. My question is, how did you approach and define the actual services? How do you know if you got too granular versus how do you know if it's too loose?

Brian (Disney): I feel that at least over at Disney and for my area, a single service should actually have one job. So for example, within Movies Anywhere, a microservice might be a profile service. Its whole job is to return back your profile data, and this is its only job. A second service might be providing authentication. So you want to target that — a service only really does one job well.

Topo Pal: I'll immediately say: next question, please.

I was hired at Capital One as an enterprise architect, especially focusing on SOA. And I don't know if anybody remembers that term, but during that time, everybody struggled hard in finding out SOA — the services, the granularity of the services, atomic services, compound services, complex services, orchestration, all of it. I think the same thing is again coming back in microservices. So I'll go back to my answer: next question, please.

Tisson Mathew: I can probably comment on a couple of things from experience. We tried two approaches to this. Coming from Amazon, it was very similar to a SOA world where we were coarse-grained in services. We had thousands of coarse-grained services. They were not fine-grained. And the teams were really fine-graining them as they went along. And then we connected these together to build large-scale business applications.

Then I went to a new company where they had monolithic stuff everywhere. It was not service-oriented at all. The advantage is that every functionality is in one application — you don't have to struggle around different services, where do you start? So one approach we took was starting as fine-grained as possible. We went by the book. A single business function, single team, allocate resources. But we ended up with too much time thinking and not writing any code, just going back and forth. Oh, this service name is wrong, or oh, this is too granular, or whatever.

The other way around is starting with something larger. What we ended up doing is something called alignment and autonomy. Alignment is aligning your capabilities to the business — so what you're trying to deliver. If you start there, probably as a low monolithic or coarse grain, you can always drive down, refactoring down to fine grain. That really made the business happy because they were getting something out of it. Otherwise they were like, "You've got 40 microservices — what is it doing really? What functionality is it delivering to the business?" That got traction with the team.

Now we're on a continuous journey of fine-graining, and to his point, the old SOA concepts are coming back. There is some new thing called mini service now, which is a composition. So it's coming back because you have to integrate these fine-grained services.

---

Topo Pal: I actually want to ask — going back to one of our previous questions — about shared services, shadow IT, and federated. I want to actually ask my co-presenter about her viewpoint from an audit, compliance, and governance perspective. Do you have any viewpoint on shared services versus localized shadow ITs, and what makes you more comfortable, or what gives you more discomfort?

Jennifer Brady: Sure. I can talk about that. I would say when I first joined Capital One a couple of years ago, we were federating everything, and that was a little concerning to me, being a governance risk management former auditor.

There are some things that, from a control perspective — especially if you are a publicly traded company and you are concerned about having SOX compliance — if you have SOX applications, you need to make sure they are following certain requirements and not doing anything that could potentially put you at risk.

So we've kind of swung the pendulum back some, and there are definitely things that we are recommending be centralized and be done at the enterprise level. Certain deployment tools, certain workflow that allows us to check all of the various things in the change management processes that are key to making sure we stay compliant with SOX and are aware of risk management.

So I would say there is definitely — you want to make sure that the things that you're federating are not putting you at risk. You really want to make sure that what needs to be centralized are things that your regulators care about, that could potentially put you at risk. Definitely from my perspective, it needs to be a balance. I understand that keeping everything centralized doesn't give your teams as much flexibility. You can't run with your builds as much. But you definitely need to make sure you are not putting your company at risk or in a potential non-compliance situation, because obviously that will be a lot more costly in the long run.

So definitely think about, if I federate this, what does that mean from a risk perspective? And definitely get your key governance and key risk management folks involved so that they can weigh in on that to make sure you're not federating too much.

---

Q: My name is Igor. I'm from 360 Degrees from Australia. My question is to Jason and Brian. Looking at what Amazon did — going from Amazon the bookstore to AWS — that was a huge leap. Do you plan to do something similar at Disney?

Jason Cox: You're asking that, looking at Amazon starting as a bookstore and of course transitioning to AWS — you see this big leap over into providing IT services — does Disney have a plan to essentially do the same type of thing?

I think Disney does do that. But from a core standpoint, we're an entertainment company. I think that's core. That's what we orbit. So whatever we do would tend to orbit that. There will be — and there are announcements that have been made by our CEO recently at the quarterly earnings call — we're going to come to market with some things related to entertainment, and hopefully look for some ways it can be disruptive, transformative even. That's how we stay relevant.

Moderator: IT definitely needs some entertainment. Is that what you're saying?

---

Q: Thomas from a Medical System. You have 300 products that are being continuously deployed on the average of four times a day. Am I right to assume that the core systems that hold account balances and transactions are not part of that? And if so, why not? Or if they are, then...

Topo Pal: The core systems that run on legacy platforms such as AS/400 — there's a good thing and a bad thing. The bad thing is that it's very hard to automate those things. The good thing is that they don't change that often. What changes is the top level.

So yes, there are many cores. Some of the cores have been rewritten into modern-day languages. They go through these kinds of changes that I just talked about. But the actual card — if you talk about the whole authorization systems and all that — they don't change that often. I don't know the last time somebody changed the interface for authorizing card swipes. Maybe it was before I was born.

To answer your question, the applications that we talked about mostly live on that higher layer, as opposed to deep down in the stack. But some of the core systems that we are now running on cloud actually go through the same lifecycle that I just talked about. They actually deliver four times a day on average.

Jennifer Brady: And there are some SaaS apps that are doing that as well. We follow Agile in all of our app delivery, and I've definitely seen — not maybe four times a day, but several times a month — we'll make changes to some of the SaaS applications. If you keep those right controls in place and make sure you're mindful of governance, definitely. It's like you start the car and drive at 10 miles an hour backing out, and then ultimately go 60 miles. We are somewhere in between.

---

Q: Hi, my name is Raj. You're pretty mature on the pipelines and making them deliver three to four times a day. But one of the things that we are struggling with right now is we have so many systems that we need to work with on the operations side. We just want to know how to integrate it and automate that process. Any suggestions?

Topo Pal: Can you explain a little more on the operational testing?

Raj: In the operational testing, what we do is we actually go and check how the field offices perform their daily work and make sure the parameters that we work against... So if you want to say the processing time for something is like 50%, it's not part of user acceptance testing. We go out in the field and work with a lot of different people to see how fast they are processing the invoices. We do go through periodic stress tests. The commercial side is working on the ground side. We're just trying to figure a way how to automate that process.

Moderator: Maybe we can take that offline.

---

Q: Hey, my name is Paula. I'm from Colombia. We are starting our journey — we are in the beginning. I want to know some examples of what you are planning to do to slay your monolith.

Topo Pal: So — slay the monolith, go into microservices architecture. The whole idea is we have seen that if we have a monolithic application, then we cannot achieve that speed that we want to achieve. There are a few things. One is start rewriting, or take pieces and parts out of it and create microservices and hook them up slowly until they die on the vine. Or even start from scratch. I know that at least one of our monolithic applications was actually rewritten in Go. So yes, that's our goal because we realized that with monolithic applications, you can do build pipelines and all the things that we just talked about, but it's not going to gain that speed that we are aiming for.

Jason Cox: It's basically the strangler pattern. And we've seen the same thing. It just takes time. You have to work through it.

---

Q: I'm with CA. How did you guys integrate your architecture team with your developer teams?

Brian (Disney): At least within my team, my engineers are the architects. We work with the dev team to architect the entire application. But there are certain areas that do have a dedicated person that is the architect. And we simply work closely with them to define what the architecture and design is, and we incorporate that into our best practices.

Topo Pal: This question is very near and dear to me because I was part of enterprise architecture when I started at Capital One.

So there are various levels, or types, of architecture. Every software needs architecture, but whether every software needs enterprise architecture is a question. And it also depends upon how you have defined your enterprise architecture. Is it just a bunch of white papers and PowerPoint diagrams, or is it a domain diagram that goes to all the class-level interactions?

The latter part — software architecture — is always going to be there, and that belongs to the product team itself. Now, the enterprise architecture, depending upon what kind of organization you are, may only say, "Thou shalt only use AWS," and that's it. "Thou shalt only use RDS," and that's it. Those are the things that are kind of followed by all the teams. And yes, they are not part and parcel of the everyday delivery process, but they are making long-term decisions and a roadmap for the whole company. So depending upon how you define your architecture, they may or may not be in that DevOps pipeline on a day-to-day basis.

---

Q: Hi, yeah. Kelly Berger with BW. We've actually started our DevOps journey about a year and a half ago. I hold operations, with engineering, specific DevOps support and allocation support. One of my challenges right now is organizational transformation. We are a very siloed organization that has architecture et cetera, and now trying to blow up that silo management. How did you handle the skills gap? Because the one thing I don't want to do is just bring in you guys, and that puts people in a position of feeling like they're not needed. So how did you handle that?

Moderator: How did you handle the skills gap when you had to introduce DevOps?

Jason Cox: First of all, we care about people. Back to your point, I think that's the right question — how do we take engineers where they are that have those skills gaps and bring them up to where they need to be? Because we know it's going to change. Part of it is, how are you creating a learning culture inside your company to encourage them to lean forward and take that?

We went on a journey where we did bring people in. So typical operations specialists — no coding skill set, maybe some Bash, maybe a Python, a little bit here and there. And we said, we're defining all infrastructure through code. We're going to a new world. And that was a first step for us. It was about giving the training to those individuals to understand best coding practices. To me, it's basics — software engineering basics. It's how you check code in. You've got to start there.

And I think that's really important — that we invest in our existing talent because they probably have a passion. There's probably a reason why they're there, and they resonate with the mission of the company. Why not invest in them?

That doesn't mean that you can't also bring in outside talent, and I think it's really important that as you add to the team, you bring in those that will be inspiring to the others. Those who were added to the company began to influence because they're sitting right next to the team. They're all working in the same direction, but they look over and see what they're doing and say, "I want to do that too. Teach me that." That was probably the best training of all — bringing in that new, sharp talent, placing them into those teams, and having them become the ignition to light that fire in those different groups. And to educate others within that team as well.

Topo Pal: For us, there are three different streams. One is hire new people, of course; retrain the existing people; and then mix people. And by that, what I mean is I have seen — at least personally, by doing code reviews — that normal application developers write better infrastructure code when given a chance. When I see a Jenkins pipeline with an API call and a try-catch block, I'm like, "That is written by a developer. That is not written by an ops guy." And which is good.

There is a huge investment that is needed from an organizational perspective to actually train and certify people. For example, at Capital One, we have the highest number of AWS-certified engineers after AWS itself. That was a big investment, and everything is paid for by the company.

Jennifer Brady: Yeah, and you need to make sure your governance folks understand that enough too. I have people on my team who are getting AWS-certified because I want to make sure we can talk the talk and speak the language.

---

Q: Poonam from American Airlines. For Capital One — in an era where we're looking at the cloud for everything, what are the biggest points where developers need to be thoughtful of governance, security, and network when using the public cloud?

Topo Pal: So there are a few touch points, and they're always going to be there. Number one — I need my repository in GitHub. You don't need to talk to anybody. You just go in, create a repository, you are done. I need to get my Jenkins build. All you need to do is go in, log into Jenkins, and create your builds. If you need help, you can talk to somebody who's there as a shared service provider.

Then there are security group creations. That is centralized, and we have automated that too. So all you do is basically do a pull request and it gets reviewed by a security architect. They all say, via GitHub reviews, "It looks good, LGTM," whatever the case may be, and it goes and changes your security group.

So there are a few touch points that will remain centralized, because it's not that they cannot be automated — it's because you need more eyes on those than just one delivery team who may or may not know all the details. It's very complex when you actually get there. So yes, there are a few touch points, but they're very minimal.

Poonam: And network? Let's say you're writing a new application and some of it will be related to AWS, some of it is going to be in the cloud — what's the interface with the network team?

Topo Pal: Yeah, that takes long. You're talking about opening up port 22 and all that crazy stuff. That takes a while. That takes a lot of discussion. And those are — when you discover them, the people who actually handle the whole process of opening up ports back to the data center note them, and they track them. And since our CIO said that all applications will be in AWS by next year or so, we know — no more data centers.

---

Q: A lot of companies like Toyota have done these all-speed things with their manufacturing and found that ultimately they had to tackle the rest of the organization. I'm starting to realize that what we do — the bottlenecks for me are going to start to be my business partners that run manufacturing. Because they think of working with me every month as a huge inconvenience. How do you get them to start to love new technology and change their business practices? They only change their business