Host Intro (Gene Kim)

Thank you, Steve.

So I've known Kurt Andersen for almost a decade, back when he helped lead the SRE function at LinkedIn, and he's been a member of the DevOps Enterprise community ever since. And I was so delighted, and to be honest, astonished, when he submitted a talk proposal with one of his colleagues at Blameless, Christina Tan.

They wanted to share the learnings from Dr. Al'ai Alvarez, who is Clinical Assistant Professor of Emergency Medicine at Stanford Medicine, and he will be talking about how it is sometimes surprisingly difficult to apply principles of blamelessness and compassion in the context of medical professionals.

There is so much about his experiences and expertise that defy easy explanation, and I am so excited about what he will be teaching us today. He'll be co-presenting with Kurt Andersen, who is currently SRE Architect and Head of Strategy at Blameless, along with Christina Tan, who facilitates strategy and executive alignment.

Here's Dr. Al'ai, Kurt, and Christina.

Al'ai Alvarez

Imagine you're in a resuscitation in the emergency department. The ambulance brings in somebody gasping for air. It's the pandemic. You and your team are getting ready to resuscitate. As a team leader, you're orchestrating medical interventions while also making sure about your staff's safety. This is a very common scenario for us in the emergency department. We make high-stakes decisions in high-risk environments. Medicine is a VUCA environment. It's volatile, filled with uncertainty. We're not sure who's coming to us and with what diagnoses. It's complex, including their disease processes. Is this COVID, or is this just a collapsed lung? Maybe both. And so we work with a lot of ambiguity.

There's a saying in medicine that patients do not read our textbooks. They present with vague symptoms, and it's on us to discern which symptom or physical findings are relevant. We don't always know whether interventions will work until implemented. In my line of work, mistakes can mean life or death. We do this several times over our shift, one after another.

Christina Tan

That sounds incredibly challenging, and there's a relatable but not equivalent parallel in IT. Sometimes an incident could cost a company millions of dollars, and the engineer who unintentionally triggers that VUCA process would have challenging feelings that are very difficult to resolve.

As John Allspaw, who happens to also be on the program agenda for this conference, has pointed out, incidents are unplanned investments in our systems, and it would be a waste of that investment if we did not learn from it.

So what can we learn about working through and analyzing IT incidents from the high-tempo, very high-consequence world of the medical emergency departments? And what does vulnerability have to do with team collaboration and creating a learning organization? These are the topics that we'll be exploring today with Dr. Alvarez and Kurt.

First, a round of intros.

Hi, everyone. I'm Christina. I'm on the strategy team at Blameless, strategizing for market positioning, executive team alignment, and building a culture of high performance and connection. I'm passionate about showing how blameless culture can work for not only engineers but also business leaders.

Kurt Andersen

And hi, I'm Kurt Andersen. I've been in SRE since before the term was coined by Google, working at a variety of organizations. Currently, my title is SRE Architect, and I'm the Head of Strategy at Blameless.

Al'ai Alvarez

My name is Al'ai Alvarez. I'm the Director of Wellbeing at Stanford Emergency Medicine. Clinically, I'm an emergency physician at Stanford. Also relevant to this talk, I serve as the Associate Residency Program Director focused on quality, patient safety, and process improvement. I also serve as the co-chair of the Physician Wellness Forum at Stanford WellMD. By the way, we're among friends here, and so please feel free to call me by my first name.

Kurt Andersen

So Al'ai, tell us how retrospectives are typically done in your emergency department.

Al'ai Alvarez

Thanks, Kurt. There are two main types of retrospectives for me. The first one is on-shift immediately after the incident, which I'll refer to as debriefs. Then there's the case reviews for critical events. There's a team that meets twice monthly to review cases.

Kurt Andersen

Well, let's come back to case reviews in a little bit. Tell me first a bit more about the on-shift, in-the-moment debriefs.

Al'ai Alvarez

For debriefs, I try to gather the team so we can reflect on how we did. This is very different from my training in the Bronx in New York City, where we rarely did debriefing after any incident. As a resident, I remember we would talk about cases in subways, on the way home, or at gatherings. Now at Stanford, our nurses even ask for this. It's simply just part of our culture.

Kurt Andersen

So who all do you pull in when you say the team?

Al'ai Alvarez

Yeah, so for on-shift debriefs, the whole team is invited, the nurses, physicians, the techs, the respiratory therapists, pharmacists, really anyone who was in the room, including the social worker.

Kurt Andersen

That seems like it would take quite a bit of time. How long do you dedicate to these events?

Al'ai Alvarez

Yeah, this is the challenge. We often do not have the time to do this, as there are constant influx of patients in the emergency department. However, we're very intentional about this, and we would spend about 5 to 10 minutes, maybe 15 minutes max, if it's a challenging case.

Christina Tan

Thank you for sharing an overview of debriefs, Al'ai. Let's dig in some more. Confronting a negative outcome can be very stressful, especially when a person feels some degree of personal responsibility. So Al'ai, how do you alleviate the emotional trauma of post-incident debriefs?

Al'ai Alvarez

I think it's important to preface debriefs. During this gathering, I say something like, "Hey, I know we've done multiple debriefs. Here's the structure that I want us to use today. Let's comment on the individual, then the team, then the environment. Because we all showed up today with the goal of saving people's lives. When we talk about the individual, please focus on yourself only so that we can avoid pointing fingers."

The next key part is vulnerability. This creates psychological safety and provides a clean framework for the discussion. Before I did this, I would either get a lot of kumbayas or kudos that were very superficial, "Great job. What a team," or the opposite, where people blamed each other, and so when we leave the debrief, you're feeling frustrated and angry.

And so, for example, I start with a mistake that I made. There's always a mistake or opportunities for improvement, so I pick one. Everyone makes mistakes. Because again, VUCA environment. We're doing the best we can with the information that we have at the moment. I pick something that I can normalize to everybody, like, "I'm not sure if you noticed, but when the patient arrived, I was actually nervous. I was worried she was going to go into cardiac arrest, and I started taking over a procedure. Now, while I was able to successfully do the procedure, it distracted me from seeing the big picture and created a void in leadership in the room."

By the way, this example is a very common mistake, so it's very easy to normalize this. This moment of vulnerability creates a psychologically safe space for my team. "Okay, the team leader can own up to their mistake, maybe I can as well." And it's actually very cathartic to admitting mistakes in a psychologically safe space. It takes the uncertainty and ambiguity away, and also you get feedback from others whether the decision may actually be the right choice.

Also, I've learned to ask permission, and for them to focus on the 1%. What is that 1% change that I can do to better the outcome or the experience for everyone? I think this takes away the pressure from somebody, especially when you're talking about devastating events. Again, at the get-go, there's no naming, blaming, or shaming. We're simply focusing on that 1% that we each can do better, which is more achievable.

I think the next step is then leveling the team. Here's an example. "Hey, I thought Dr. Christina did a great job in that resuscitation, and I'd like her to lead the rest of the discussion." What people may not know is that I've already primed the senior resident about the structure so she knows what she'll focus on in the debrief.

Kurt Andersen

So it sounds like you've learned quite a lot about how to make these effective. Are there key takeaways that people should bear in mind about elements that make them particularly effective or might undermine them if they weren't done?

Al'ai Alvarez

Yeah, that's right, Kurt. For on-shift debrief, it really helps to do it immediately. That way everything is fresh and everyone is contained. Whether there's good or bad outcome, the goal of debriefing is to just unpack and come up with a shared mental model. Because my version is not always the version that the nursing perspective may have or from the trainee. Debriefing allows us to reflect and connect with each other instead of ruminating on the incident on our own as we drive home, or for me, right before we go to sleep.

So there's also the need for multiple touchpoints. This is very important. It's unrealistic to try to cover everything in one debrief, especially since we're talking about five to 10 minutes only. So for individual feedback, I would just pull them aside later on. For something that I anticipate people will be perseverating with, an example would be decision that many people will not agree with, I will quickly find time to pull aside the person who made the decision to give them a heads-up. "Hey, we're going to talk about this at the debrief. This is what I saw. I wonder what your thoughts are." That way, instead of hearing about it for the first time in front of everyone and processing the information to save face, they really have a few minutes to just collect their thoughts and not feel defensive.

We never want someone to feel defensive during a debrief. This also allows them to talk through their thought process with me, and then I can coach them on some learning points. This is something that can be very powerful, especially for someone who's beating themselves up for a mistake.

Kurt Andersen

So can you give us an example of how all these come together with maybe a case example?

Al'ai Alvarez

All right. So I recently had a pretty tough case. The patient died, which happens, but there was a lot of mixed emotions in the room. Immediately after the resuscitation, I can tell the head surgeon was very upset with how things went. So I pulled him aside. I said, "I know you just came on to this and I can see that you're not happy. Let's talk this out because I want us to be united in front of everybody when we debrief. I'll share with you my perspective of why we did what we did, and I want to hear from your perspective." And so we did just that.

I actually didn't get a chance to pull my senior ahead of time because of that meeting, and so I didn't really give them a heads-up that they're going to be leading the debrief. And so I stood next to him during the entire debrief to show that I support him because again, the case did not go so well.

I'm sharing this because I want to highlight something that I did not expect. The senior surgeon, as we were going through this, immediately shared, "Hey, I screwed up. I should have done this instead." And I never would have expected trauma surgeons to say this in front of our nurses, the emergency medicine team, the techs, and acknowledge this, and we have a great team at Stanford. And so this is just not part of the norm. So for me, this was a signal that somehow we've established a safe space enough for a surgeon, the surgical chief, to acknowledge a mistake.

And that was pretty cool because then when the leader openly owns their mistake, everybody else starts sharing things like, "Oh, and I could have done this better, and next time I will do this." And again, it's not focused on the screw-up, it's instead, "Here's what I've learned from this, so next time, here's what I'm going to do instead."

Christina Tan

Yeah, and I think this is the forward-looking accountability that improves the team's ability to learn and adapt as a whole. Al'ai, as a practitioner of mindfulness, how do you coach your students to deal with self-blame?

Al'ai Alvarez

This is not as easy, Christina. I think it's important to feel the negative experiences, but it's also important to put it in context that they're not alone in this experience, that everyone feels this way, and that there are ways to improve if they're willing to, if they're actually open to exploring that. This is where mindfulness is important. It allows us to slow things down and process the experience, and also where self-compassion comes into play. Instead of focusing on the shame, we get to move past that and think about how we can improve better. This is very hard to train, like I said, especially for physicians who have spent all our lives motivated by this fear of failure.

I also don't like the notion of stripping away those negative feelings and just trying to be positive, because that creates an internal conflict. Just like you, I am allowed to feel crappy when I fail, and I also do not need to identify with my own mistakes. So my goal in training is to help prepare them for when they are practicing independently. Hey, I'm letting you know that you'll have many mistakes, and just like me, you'll also feel crappy when you go through them.

And this is where compassion comes in. Just like me, when I make mistakes, I need to be able to be kind to myself in order to actually process what's happening, as opposed to start blaming myself and focusing on the should have, could have, would have, or the difficulty of that situation. Using the just like me perspective, we're able to see the possibility of being kind to yourself in order to not just do the job, but actually own our role and the privilege of being able to do the job, and be accountable for the actions and decisions, and also being accountable to how we're going to iterate ourselves to change for the better. This is how we promote wellbeing and professional fulfillment.

Kurt Andersen

So it sounds like this model you have of not only dealing with the team but dealing with yourself is to be compassionate to both, taking care of both yourself and the team, so that instead of being a victim and feeling dumped upon, you're empowering everyone to understand that you have the capability to make an improvement. And then that fans out from there. As you throw the rock into the pool, the ripples stretch out beyond. So how do you put mistakes into perspective when they are of such high consequence, potentially?

Al'ai Alvarez

Yeah, that's right, Kurt. Our goal is to empower rather than being fixated as victims. Ethan Kross, in his book, "Chatter," talks about using time as a variable. In five days, in five months, in five years, will it really matter? And at least it allows you to then think, "Okay, what can I do next?" As opposed to, "Oh my gosh, I'm going to lose my job."

And so, so far, we've focused on things that are going bad. Debriefs can also be about celebrations and appreciation. Here's an example after an incredible resuscitation. I pull a resident aside and share, "Hey, I just want to pause and take this moment to let you know that this is what it feels like to save a life." Right? I'm getting goosebumps just sharing this because oftentimes they're so focused on the inner critic and blaming things and focusing on the screw-ups that we don't recognize when we're doing amazing things. We need to also bookmark positive feelings.

Christina Tan

I love that, bookmarking positive feelings. Now, going from the individual to the leader confronting the incident, Al'ai and Kurt, should we, as leaders, expect physicians and engineers to not make high-impact mistakes, and are mistakes a reflection of lack of competence or skill?

Al'ai Alvarez

Yeah. Thanks, Christina. This is very unrealistic. I will acknowledge that in emergency medicine, when bad outcomes happen, people can end up with long-term complications, or they die. And so how can you tell somebody to say that the mistake was okay? And I think that's very hard. But I think part of this is to also realize that there's only so much that we can do. We have to understand that what's really within our circle of influence and what's not. What is actually part of the disease process, and what are systems issues that led to the outcome?

So for example, let's talk about the hypothetical scenario that I shared at the very beginning. What if that turns out to be a massive heart attack? The question then should not be, how can a smart doctor working in a top medical institution, who went through four years of college, four years of medical school, and four years of residency training, miss a simple heart attack? It's inconceivable, right? And yet, this actually happens, and it happens not so infrequently across the country. And it's because of interruptions. It's because maybe you're thinking about something else, and then it was wrong. It's the wrong context. In the scenario that I shared at the beginning, it's because it's the pandemic. And imagine back in March of last year when we didn't know a whole lot about COVID. Everybody was scared, and everybody had COVID until proven otherwise.

Kurt Andersen

We're all equally susceptible to the pressures of the systems that we operate within, whether they are in the moment or whether they are entrenched in the procedures that are around us. I think that's one of the things that is highlighted in the "Checklist Manifesto," which is a book for people to look up if they haven't read it, because it highlights that even for well-trained, extremely expert people, having a basic pattern of steps to follow, not necessarily to check off, but to check that it is done, helps to avoid these mistakes. And so that can be a countervailing pressure against the pressures of the system.

Christina Tan

Yes. In situations of stress, we can all benefit from structure. Moving on from retrospectives, let's talk about designing for wellbeing and performance through culture. This is a question for both of you. How can the leadership of an organization use language to build an adaptable, psychologically safe, and learning organization?

Al'ai Alvarez

At Stanford, my medical director changed the name from peer review to case review. This is the second version of the retrospective we mentioned earlier, and it highlights how it's important to not focus on the peer. It's not about the peer anymore. I guarantee you that peer has learned a lesson, and worse, they're in their shame spiral. By focusing on the case, we move past from the naming and blaming and shaming of this experience to really dissecting the case, understanding the systems-driven factors that led to the outcome. This is how we advance patient safety discussions, and this is how we can also protect physicians and their frontline workers.

Monica Worline and Jane Dutton wrote the book, "Awakening Compassion at Work." If we look at events from the perspective of just like me, we can be kinder and have better empathy on how we would want to affect change.

Kurt Andersen

In highlighting the impact that language can have, just the little word why, if you change that in your questioning to a what or a help me understand, it makes a huge difference in the power or the force that's coming across in that question. Courtney Eckhardt had an awesome talk, which is linked to in the resources at the end of the talk, that highlights that the word why asks for a judgmental or an agentive answer. It's asking, who can we hang for this? Or blame, frankly. And so avoiding why questions helps to open people up because it defangs that power.

Al'ai Alvarez

Yes. I mentioned I trained in the East Coast in the Bronx, so I've had to learn it the hard way. In the Bronx, my attending would ask, "Why did you do this?" And in California, I've learned reflective listening and appreciative inquiry. So, hey, I noticed this. Can you walk me through your thought process?

Kurt Andersen

The other thing to do is to try to help keep people from making counterfactual statements, "I should have done X," because that's not what happened. Regardless of what you think after the fact, you're doing it with the benefit or the hindrance of hindsight bias. You're not doing it in the moment. And the decision-making in the moment is the key. And so you're trying to look back and understand retrospectively what happened. Don't try to bring the you from after the fact into that event. Be factual, not counterfactual.

With the hindrance of the pandemic now having been with us for almost two years, the old term in the IT field of postmortem has got a lot of weight to it now. And so generally, we're seeing the industry shift to call it retrospectives because it doesn't have that negative connotation.

Christina Tan

Yeah, and language is a powerful tool for shifting culture. Let's talk about the next tool, rituals. Besides blameless and timely debriefs, what other rituals or programs have you created in service of building a learning organization?

Al'ai Alvarez

Yes, I'm really excited about this. We brought back Save of the Month awards last year at the peak of the pandemic. It was important that we also hear we're doing great instead of all the constant barrage of information about COVID. Also in medicine, there's this morbidity and mortality or M&M rounds. In the East Coast, this means me standing up in front of everyone and discussing the case, and answering the questions from the audience of faculty and trainees. Talk about name, blame, and shame. And it was always humiliating.

Because Save of the Month also acknowledges one person only, this year we've implemented the offshoot of M&Ms, which is the Amazing and Awesome or A&A. The recipient of the Save of the Month presents their case in front of faculty and trainees, talking about the team and their efforts that led to that save. We're highlighting positive variance in front of everyone, which is the opposite of the humiliating experience of M&M.

Christina Tan

A&A is a very creative complement to M&M, Al'ai. Can you tell us how you conduct your biweekly case reviews and what objectives you're trying to accomplish there?

Al'ai Alvarez

So for critical events, we look at contributing factors from the human level and the systems level. We then develop action plans for each and track incidents and interventions for patterns. For example, in the hypothetical case that I shared at the beginning, when you start seeing a couple of misses of heart attacks during the pandemic, you create a system that automatically checks for it, so no one forgets.

Kurt Andersen

And in the SRE world, we're seeing some similar evolutionary practices in retrospectives. First, one of the developing practices is to interview the individuals beforehand. That's similar, Al'ai, to what you do in terms of giving a heads-up to your seniors before you go into the debrief.

And then secondly, the other thing that's showing itself as being very valuable is to separate the identification of action items or fixes from the group meeting or the group retrospective itself. Because this allows the group conversation to focus on the things that can be learned. And the things to be done, everybody can come up with those. But doing it after you focus on the learnings and in a separate context really helps to clear the mental space and keep people in the learning mode.

Christina Tan

Yeah, and these rituals repeated consistently create a new culture. Al'ai, you mentioned Monica and Jane's book earlier on "Awakening Compassion at Work." They talk about role creation as a way of integrating compassion and fostering psychological safety at work. So how was your role as director of wellbeing created?

Al'ai Alvarez

Now, I'm fortunate to work in a place where the chief wellness officer was first created as a role. This opened up the role that I have now, which is the director of wellbeing. We have one for every department focusing on faculty and staff wellbeing. Before this role, I was one of the associate residency program directors in emergency medicine. My role was to help train future emergency physicians, and because of my background in quality and patient safety and clinical operations, and because of my experience with peer review, I created a role for myself focusing on providing trainee support that are involved in case reviews.

I normalize with the residents how just like them, I still get palpitations whenever I get those secure emails about a case. This offers me the opportunity to share resources available to debrief anonymously using our physician resource network.

Christina Tan

So Al'ai, what kind of business case is there to justify the investment in wellbeing? What metrics of success can you point administrators or managers to?

Al'ai Alvarez

So according to Stanford WellMD, on a survey of over 1,000 physicians, physicians who practice self-compassion have low levels of burnout and vice versa. Doctors who are burnt out have low levels of self-compassion. Wellbeing and professional fulfillment leads to retention. In fact, the opposite is very costly. According to data, it costs the institution about $500,000 to $2 million per doctor who leaves the institution due to burnout. This includes recruitment efforts, loss of productivity, onboarding.

It does not even include costs attributed to medical errors and malpractice. There's also the cost of human lives. We have yet to see the cost of the pandemic on the mental health of frontline workers. Before the pandemic, it's estimated that about 400 physicians die of suicide per year. Beyond patient safety, we can do better to support mental health. This is why I do my work.

There's a study showing that at time of matriculation, medical students are less likely to be burnt out, have better quality of life and less depression than the general population from their similar age groups. Then shortly after, at every level of medical training from medical school to residency to being a practicing physician, doctors are twice more likely to be burnt out than the general population from similar age groups. This is why psychological safety goes beyond team debriefing. It leads to better accountability and engagement and creates a more resilient workforce.

Christina Tan

Yeah, absolutely. Language, rituals, and programs, and role creation can all contribute to building a culture of psychological safety, which also means better wellbeing for the team.

Now, as a concluding question to all of us while we leave this talk summary up here for the audience, what measures of success matter most to you when you assess your progress towards building a learning organization? And what principles of leadership do you carry with you?

Al'ai Alvarez

Yeah. For me, whenever a resident comes up to me to talk about a challenging case, this tells me that we're creating a more psychologically safe environment. My hope is that we can expand this to quality and patient safety discussions without fear or risk of losing our job. The pilots have figured this out.

Also, the humanizing aspect of our work requires us to see ourselves as fallible. I believe that each one of us can play our part in awakening compassionate work, and it can start with how we interact with each other when we discuss mistakes and bad outcomes.

Kurt Andersen

When we're looking to create a learning organization, and when we're looking for how our retrospective practices feed into that, we need to look beyond the simple numeric quantitative data that has traditionally been looked at, like time to respond or time to detect. We need to start looking at how many people are learning from this incident, how many people participated in the group meeting and learned there, how many people have improved their mental models, and after the fact, when there's a write-up from your retrospective, how many people look at that afterwards, and over what duration of time?

Christina Tan

A measure of success that I care about is how responsive and adaptive an organization is, particularly to bad news. So let's say parts of your critical system needs a very costly and time-consuming rearchitecture, and most of your engineers already know this. How long would it take for the company and management to prioritize this rearchitecture and support it with resources accordingly?

It would be really unfortunate if management doesn't find out until months later and can't react as quickly as competitors because the company lacks psychological safety. But imagine if the rearchitecture is planned and resourced because the engineers brought it up without fear in a timely manner. That's the type of organization I want to build. And it is possible if we, as leaders, make time to listen to the team and always consider how they might be right, especially when they raise things that contradict with what we believe in.

Kurt Andersen

So as a tradition at the DevOps Enterprise Summit, Gene has asked us all to leave you with something that we would like your help on. You can do it in the Slack channel, or you can do it through Twitter to us. My question is, have you found specific and effective practices that help in socializing learnings from a retrospective amongst a wider audience than those who were just directly involved? If so, please share.

Al'ai Alvarez

Please support initiatives that protect the mental health of physicians. The Dr. Lorna Breen Health Care Provider Protection Act was unanimously passed by the US Senate on August 6th, 2021. The goal is to reduce and prevent suicide, burnout, and mental and behavioral health conditions among healthcare professionals.

Also, thank you for those of you who are getting vaccinated. 18 months ago, doctors and healthcare workers were touted as heroes. Now there's a sentiment of betrayal from those who make decisions that put themselves into harm's way and also put us, the frontline workers, in harm's way. By being vaccinated, you're protecting yourself and everyone else around you. So thank you.

Christina Tan

Thank you, Al'ai and Kurt. The three of us would like to now invite you, the audience, to conclude our talk with a two-minute mindfulness reflection.

Christina Tan

Al'ai and I actually met through a mindfulness community at Stanford, and I'll be guiding you through this reflection as you close your eyes. But before you do, I want to share the goal of this exercise.

Really, the intention is to give you some mental space to experience vulnerability and compassion yourself, because oftentimes we're so busy going from one task to another without any time to think and reflect. Secondly, we want to give you the chance to consolidate some highlights from today's talk.

We now invite you to close your eyes. Deep breath in and out.

Feel the loosening in your body with each out breath.

Now recall a mistake that you have made in the past that you're still holding against yourself. Bring that to your mind.

Who do you see? What do you hear? What feelings are you experiencing? And notice, how does your body feel? Is your throat tightened? Your eyebrows furled? Let that tension go with an out breath.

Recognize that being imperfect is a human condition. Just like you, I've made many mistakes, and just like me, you will make many mistakes in the future too.

Now, put your hand on your heart and ask yourself for permission by repeating silently, "May I be happy. May I be peaceful. May I be free from suffering."

Feel free to continue resting your hand on your heart. Reflect on the talk today. What was a mental image you had during the talk that you want to commit to action? Make that image as clear and vivid as possible. Plant a seed of intention for the mental image to become a reality. Give thanks, let go, and open your eyes when you're ready.

Thank you very much for tuning into our talk. Until next time.