Host Intro (Gene Kim)

To introduce the next speaker, I want to share a story about, for me, one of the most interesting passages in Wiring the Winning Organization, which I co-authored with Dr. Steven Spear last year. I wrote about how in 2017 my father had a stroke. He was transferred from the neuro intensive care unit to the neurology ward, and after that transition I got to see the daily rounds. He was seen by a group of clinicians and nurses, case managers, and so forth. I remember on that first day the doctors couldn't decide whether to put him on blood thinners because the brain MRI images wouldn't be available until later in the day.

When I heard this, I showed them a picture I had taken the previous evening while I was shoulder-surfing the neurologist looking at the MRI images — I'd taken it so I could send it to my neurology friends. When I showed the doctors the image, I asked, "Would this be helpful?" And they said yes. They studied the pictures and decided to put him on blood thinners later that day.

For years after, I was mystified by how and why this happened. There was all this technology and computers in the room, technology deployed throughout the healthcare system — why couldn't the clinicians view the MRI images in that moment, during the ten minutes that they had? I carried that mystery with me for years. During the book launch I got to meet David Wright, the next speaker, who specializes in solving complex technology issues in healthcare. When I shared the story with him, he said, "That's so weird — I'm helping a hospital system with those issues right now."

To use the language in Wiring the Winning Organization, this was a layer-three organizational problem made so much worse because of the things going wrong in layer two — the technologies and the mechanisms that connect various parts of the organization. I'm delighted the next speaker is David Wright, CEO and Founder of Disruptive Innovations, to tell the story of what it takes to fix the organizational wiring in complex healthcare settings and the deep technical and organizational expertise needed to do so. This story has so many lessons that are relevant to any large complex organization, especially those that have grown by acquisitions and those working in safety-critical areas that need real-time communications. Here's David.

David M. Wright

Wow. Good afternoon, friends. My name is David Wright. I'm the Founder and CEO of Disruptive Innovations. They say don't meet your idols. I remember Andy Luedtke gave me The Phoenix Project a number of years back, and we were just texting about how crazy it is to be here with you all today. Gene is one of the most humble and brilliant guys I've ever had the privilege to know in our community, and I'm honored to be speaking with you today.

Today we're going to be diving into the chaos of a disjointed health system and the resulting siloed knowledge that plagued this group for years — and, for that matter, most of the health systems we encounter throughout the country, and a lot of the enterprise organizations you'd think of that are the classic late adopters. This is not necessarily unique to healthcare — the effects of compartmentalization, islands of information and people, et cetera. If you don't work in healthcare, I just invite you to try to identify, and I'm happy to relate after the fact with other industries.

Quickly, to qualify myself: I'm from Brooklyn, New York. I am a nerd fully — grew up building computers, studying MCSE by the time I was in high school, played Warhammer, somehow ended up getting married, and having two daughters. I'm a serial entrepreneur. I've invested in a couple of software startups. I'm an active member in HIMSS and CHIME, two healthcare-based organizations, and I'm a CHIME Certified Digital Health Executive.

Disruptive Innovations — I founded in 2018 after helping grow another consulting firm to one of the preeminent IT and telco consulting firms in the private equity arena. We're a digital business and IT consulting shop with a heavy healthcare focus. Within healthcare, we focus a lot on customer experience, patient/human experience, and advocacy. We started with the patient experience optimization module, which we productized during COVID. We saw the writing on the wall. We realized we had to look into people, process, technology and methodology surrounding the patient experience life-cycle — everything a patient leverages to interact with a health system throughout the continuum of care. What does it look like today? Where do they want to go? Not only how do we design the technology initiatives that roll up to that, but then how do we attack those layer-three issues — the clinical and operational workflow optimization, the organizational change management that needs to take place — and how can we tie that back to ROI? How can we make stakeholders feel like this was their idea? How do we create those quick wins that create the trust we need in order to actually affect change? It's not about us.

So, story time. Gene mentioned that story about his father, which really resonated with me based on the work that we were doing — the fact that they couldn't access that MRI image. There was a computer in the room. It's connected to the wide area network. We talked about this after our podcast interview, and so I'm going to tell you this story — you can close your eyes if you want, or not. Imagine slow system performance, incomplete team expertise, and frustrated doctors creating this perfect storm that's truly jeopardizing patient safety, and almost led to mutiny in the organization. We're going to explore this real-world case study now.

For the sake of anonymity, I've anonymized the health system. It's the Lagwood Medical Institute, a sprawling academic health system with a storied history of creating really great outcomes for patients. They have great doctors, great facilities, but they were plagued by these latency, interoperability challenges, and image transfer issues within their radiology department.

I mentioned during the module that we like to go really wide. Classically when I worked with clients, we worked in IT and digital, maybe some of the executive stakeholders. Nowadays, especially with health systems, we're talking to the clinicians, we're talking to the operational folks. These are just some of the folks we ended up speaking with. Chip — the CIO — actually reached out to me because Penny called and said Ima and the rest of the doctors were going crazier, and the emails were getting angrier and angrier. The CEO was involved now, and he said he needed results immediately.

So what was the big problem? Ultimately, slow system performance of their PACS and CPACS systems. What are PACS? Essentially Picture Archiving and Communication Systems — a digital image management system used in healthcare to store, retrieve, and transmit medical images and reports. Think kind of half file server, half database. CPACS are just the cardiovascular arm of that, used to share cardiovascular images. They also integrate with other modalities — the echocardiograms, nuclear cardiology — so used to diagnose and treat heart issues. DICOM — Digital Imaging and Communications in Medicine — is the international standard for the acquisition, storage, printing, and transmission of medical imaging pretty much across the world. As you can see, that significant latency in their DICOM imaging was impacting radiology and cardiology workflows for internal and remote workers. The impact, which might seem obvious, was delayed diagnoses, heightened patient anxiety, and physician frustration. Like I mentioned, those emails were getting bad — ultimately impacting the quality of care.

This is just a high-level diagram of their environment. I'm going to reference this a little bit later, but you can see over to the left the McKesson PACS client could be an example of an on-site radiologist; the remote radiologist toward the top, communicating back over VPN. Most of the servers were housed in that data center location in the middle, and an alternate stack in the UTC data center. But what I'm going to get to is there were a lot of opportunities for bottlenecks there.

Getting into our initial assessment and engagement overview: we were engaged over a twelve-week period. I had people on my team a lot smarter than I am — two ex-CIOs. One was actually a doctor — or actually is a doctor rather — and one is probably on the genius spectrum, as well as a pit bull of a project manager. She was just a beast. We communicated across the organization, all the executive team, SMEs, the vendors, et cetera. This is just a high-level overview of the project diagram. We'll share this with you — I know it's like an eye test — but basically agile methodology broken down into stage gates. The CIO wanted the improvement as we went, so we were fixing the plane as we were flying it, so to speak.

What did we find? Initially, from a layer-three perspective, much of what we were told prior to the engagement and walking in the door was wrong. We expected three PACS systems; we found five. People were not as transparent about certain things as you would hope. Ultimately the organization grew acquisition after acquisition with very little assimilation. We see that a lot in healthcare. Sometimes the EHR gets integrated, sometimes it doesn't. Sometimes there's some tape and bubble gum put on. But in order to remain competitive in healthcare and to negotiate within a region with different insurance providers, they just need to do something — so it's not their fault. That's a larger conversation. But we also saw fragmented IT groups, ego, politics — all of that led to this culture of distrust and fear in the organization.

Two of the core issues that were rooted in what we estimated the problem to be: one great example was in the CPACS systems. The OAD work list — basically the clinician trying to bring thumbnail images up — they just weren't populating. Or if they were populating, it was like loading on a dial-up modem. And then finally, once they did load, he would try to click in and see this cine loop, which is essentially an ultrasound — a sequence of images that can be compared to ultimately treat cardiac issues. It wasn't loading, or it was super slow. Problem two — this is exactly what Gene's father experienced: the images from the PACS were not available to clinicians when they were doing the rounds, either images that had just been taken or historical images that they needed to access in order to track the progression of disease. In some instances they were taking up to ten minutes to load.

I mentioned that diagram before — one of the things we saw was this file scanning that was going on between the Microsoft Server Message Block. The images were going through firewalls that had been set up and were querying the images, additional provisioned web servers, load balancers, network switches, different policies of QoS that were slowing things down. It was death by a thousand cuts. There were so many things that had been set up by these different siloed teams that weren't really communicating with one another. There was a membrane, and maybe they had tried to break through in the past and it didn't really work out. And then they got complacent and they just kind of decided to keep the lights on.

In regard to that, we found outdated documentation, insufficient vendor support and accountability — in fact, some of the vendors would get new account executives, people that didn't really know the full story, they only knew what was right in front of them, and they'd make recommendations that were actually just wrong. It was the wrong thing for the organization, but the organization would take it as gospel. Lack of application performance monitoring tools. Inadequate capacity. You can see that these issues collectively compromise the efficiency and effectiveness of the health system, really perpetuating those layer-three issues as well.

In regard to the methodology we took: a central-to-peripheral approach. This was actually contested initially by some of the folks at the health system. But we wanted to focus on the internal systems first, starting with the data center that warehouses the application servers, and then extending out to the external sites and the remote users. We felt that would be the way we could get the biggest bang for our buck, get the quick wins, affect as many users as possible. The daily huddles and continuous improvement and feedback loops were obviously just completely necessary in order to try to do this body of work in twelve weeks — which we did successfully.

The steps we took: the root cause analysis was paramount. We conducted those interviews with server folks, storage folks, network folks, applications, cyber, clinicians, compliance, all the executive team. The performance monitoring tools — when we came in we just didn't know what we didn't know about what was going on. So deploying Wireshark and PerfMon and Cisco ThousandEyes — all of those tools gave us the underlying performance data that we needed in order to assess the situation and that they could use to upkeep the systems on an ongoing basis. We reviewed the servers, the storage solutions, the network devices, the communication links, and then we dove deep with all their vendors — Dell, McKesson, Philips, Windstream, Comcast, et cetera — to get the service histories, the findings, and their proposed solutions so that we could back into the rest of what we knew and help guide the client to the right solutions.

Issues found and solutions offered: you could see the issues and the solutions, but I think what is interesting is that the technology staff was so splintered that no one group could solve the problem by themselves. And there was no coordination mechanisms across those groups. One of my colleagues gives the conductor example — if you have an orchestra with no conductor, would the music still sound the same? I think that orchestration layer that could run across the stack — we had to create that with the client, and we needed the executive sponsorship necessary in order to help facilitate that.

In regard to some of those underlying layer-two issues: the hyperconverged hardware underestimation, the database servers were inadequately provisioned with RAM causing excessive memory swapping versus the database files being immediately available in RAM. Not to get into the weeds, but basically with this system review and the reconfig, we were able to solve for a lot of these issues — but we never would've been able to if we didn't address the layer-three issues that were going on.

CPACS, very similar story. The suboptimal configuration, the changes to architecture that were recommended that shouldn't have been, and ultimately this redesigned deployment architecture ended up solving for that — ended up sorting out the SMB storage settings. And then the Philips Radiology remote access — suboptimal network performance over the VPN. We enhanced network collaboration, optimized routing configurations. We addressed the load balancer. So you see it was multiple different things, not just the one — that's the theme here. Same thing with the VPN and remote access. We needed to review the network. We took some immediate remediation actions. We did some traffic prioritization, routing cleanup, and capacity planning, and that solved for that. The bandwidth was adequate there.

Immediate outcomes wise: we addressed the memory deficiencies and the inaccurate configurations, which led to improved performance for the radiologists. We reduced the latency and improved the remote access experience through the network reconfig. And we addressed the configuration issues with the multiple vendors, leading to the immediate improvement in the retrieval of those legacy images.

Long-term, we talked about this performance monitoring framework — so you've got to see what you're doing and actually know what's going on. Establishing a robust monitoring framework, an accountability structure for the team, fixing the storage and the network — optimizing the storage solutions and the network topology. And then finally creating these layer-three mechanisms so that all the different teams could actually work together on the things that they support, without bringing people like us in, and fostering this culture of continuous collaboration and innovation — which is going to take some time, but we're on our way.

The complexity of these issues required, again, people a lot smarter than me in order to address them. The hospital's existing team lacked that necessary expertise, so they needed to bring in some folks that understood and were able to walk across and be that conductor in order to get to the root of the issue — that systematic approach. So that methodology that we used and the diagnostic tools that we used enabled us to uncover the actual root causes. The data is what told us what the layer-two issues were. And the collaborative effort: it took talking to stakeholders across the organization, and really drilling down into the SMEs in order to get to the bottom of things.

So now we have the radiologist at LMI, Dr. Blinkmore, who's trying to assess an active stroke patient. Before, she's dealing with these excruciating delays waiting for images and load times, and this is a situation where every second counts. Now, after our engagement, she's enjoying seamless and rapid access to images, enabling her to focus on delivering care to the patient. These are just some testimonials we received. CIO very by-the-book — but together we achieved those immediate and sustained improvements. It's funny to get things out of the doctors; they always want a little bit more. So it's kind of hard to get the testimonials, but "lag time decreased, and so is my stress" — that was enough for me. And Penny knew that we were doing something right because she stopped getting calls from the CEO.

I'll end with this: my colleague that I mentioned before is full of analogies. So another analogy — he got from his father — you always have to keep your eye on the ball. He's his father telling him this, and in this situation: what is the ball in healthcare? The ball is excellent patient care. And I think sometimes as technologists, I lose sight of that ball. So how am I bringing my team together to remind them what our mission is, what we're trying to do here, who we're trying to serve? Bringing those folks to do rounds, or just having a leader that's able to really touch, move, and inspire their teams and remind them that we're not just button clickers here — that we're actually, you know, it's lives at stake — is really important.

So again, I'm incredibly grateful for this opportunity and to be with you all today. And, yeah, if you want to talk further after the fact, I'm here. I'm available. Thank you very much.