Eleanor Taylor

[00:08] Hello everyone. My name is Ellie Taylor, and I'm an auditor. And I've been an auditor for over 25 years in my career. So why am I here today?

[00:24] Well, my most recent role at Nationwide Building Society was as an audit senior manager taking care of the transformation and IT portfolio. As part of that work, I interacted with lots of IT and change teams, and I saw an increasing use of words and phrases that were relatively new to me at that time. This was about four years ago.

[00:49] People started talking about Scrum and Kanban boards, and sticky notes, and ceremonies, and all sorts of things that sounded really intriguing. So I decided to do some research and find out what it was all about, and whether or not this was something that we should be looking at in internal audit, and whether we should adopt those new ways of working. What I found really intrigued and excited me, because I was absolutely clear that we could start to adopt and adapt some of those ways of working within internal audit.

[01:23] So today I'm here to share our lessons along the way, and how we went about adopting new ways of working within internal audit, what new ways of working mean to us and how they've benefited us, and those key lessons that we keep close to us as we continue on our journey.

[01:43] So before I go into any details about that, I think it's really important that we think about the internal audit context. And actually, I think internal audit is a good area for this sort of work. It's really, really fertile ground. So we have around 90 auditors, and they are broadly aligned to business areas. The majority of them are located within and with the business in our Swindon head office, or they were until very recently. We also have a few auditors that are scattered around in some of our sub-offices, like Bournemouth and Northampton, and we now have some in London. Our teams are typically small.

[02:25] They're around two to five people, probably no more than that. And the average lead time of start and end of an audit takes around 16 weeks on average. We run a traditional hierarchy, or we have a traditional hierarchy within internal audit. So we have a chief internal auditor who sits at the top, who's supported by a senior leadership team, who then oversee lots of audit teams that deliver those audits.

[02:51] And we run a standard linear methodology. And by linear I mean that we have a planning phase that's followed by another phase, which we call risk and control evaluation, which is then followed by some testing, and is then followed by some reporting. And between each of those phases, there's a gateway, so that you can't proceed from one phase to the next phase without going through a gateway. That's how it was.

[03:20] What we do have in internal audit at Nationwide is a bunch of hugely enthusiastic and highly motivated people who are willing to have a go at all sorts of things and try stuff out. Try new stuff. And that really can't be underestimated, and it's been critical to the success that we've achieved within our function.

[03:42] So it's taken two years, probably two and a half years, but most of our audits are now using agility and different ways of working in some way. Some of them to a larger extent than others, but that depends on their individual context. Before I go into any details, I just wanted to share with you some testimonials from our auditees, some feedback that we've received along the way. These are really important because they show that we haven't tried to do this in isolation, and we've really thought about how this impacts on our auditees. Because at the end of the day, what we want to do is deliver valuable audits to our auditees.

[04:27] If we can't manage that, then we are not going to achieve what we need to achieve. So if I start with the first one, and that's from our chief executive officer, Joe Garner. And he came along and visited us and saw all of our whiteboards in the office. And he also spent some time at a daily standup, which was a bit unnerving for the teams at the time, but was great because what he saw was really good collaboration between the audit teams and the business, and how effective that interaction can be, even on a short amount of time.

[05:05] The next one really shows for me how we've achieved something that many audit functions have tried for years to achieve, and sometimes manage it, but not consistently manage it. And that's no surprises. So we all have charters in our internal audit functions that say, "No surprises." In most of the places that we've worked, that's really, really hard. The adoption of new ways of working has allowed us to make that a consistent reality, and that's seen in this feedback.

[05:38] And then the last two bits of feedback really were the start of what we now have as a huge library of feedback that demonstrates the value that we bring to our teams, the positive engagement, how engaging it is, how the business now understand what it is that we do because we allow them to see what we do, and we work with them. And the final sentence on this slide is probably one of my favorite sentences of feedback that I've ever received, or the teams have ever received.

[06:12] And that is, "I am a great fan of the agile audit." Imagine that as an auditor, somebody saying they're a great fan of your work. It's absolutely transformational. So if I move on to, so how did we do it?

[06:28] Well, context is the thing. So throughout all of this, it's all about thinking about what is the internal audit context, how is that different, what are the things that are right for internal audit that may not be right for IT and that world, but actually in audit it works.

[06:52] The first thing we looked at when we were looking at how we were going to do this was, what are our reasons for change? So why are we doing this? What is our why? And on reflection, when I step back from this, I think those whys are probably very similar to organizational-wide whys. So we're looking to be more forward-looking and responsive in an ever-changing world.

[07:19] And when we first embarked upon this, little did we know how ever changing and volatile and unusual the world would now be. And by allowing us to move to be able to adapt to that ever-changing world, that would allow us to continuously provide audits of the highest value and continue to do that, rather than setting a plan at the beginning of the year and then just plowing our way through some audits and hoping that some of them will be valuable.

[07:49] We now flex our audits throughout the year to make sure that they're the right ones. And that helps us to grow internal audit's relevance as a function. So if we don't adapt and we don't provide audits of the highest value, that means that the work that we do, and that the organization pays for, becomes less relevant. It's not important.

[08:11] And that's not a nice place to be. And I saw this as a wider opportunity to reinvigorate the wider profession. So the internal audit profession provides a great deal of value to organizations, and I saw this as a way to really showcase what value we can bring, the quality of the work that we do, the expertise that we bring to the organization, and that second pair of eyes to just challenge but also support the wider organization as they start to build and grow.

[08:46] I was also conscious that it would be really nice to be able to be sat around a table with some people outside of work, and when they ask what it is I do as a job, that I would feel quite proud to say, "I'm an internal auditor."

[09:02] Whereas at the start of this, I felt that people saw us as secretive and would write down anything that you said, and weren't really that trustworthy. And I wanted to bring that out into the open and bring a level of transparency that would enable people to feel really proud of their profession, and also for the wider business to see the value that we bring.

[09:26] And also to improve efficiency, and that was an interesting word at the beginning of all of this. I now tend to look at it as improving flow and pace rather than using the word efficiency, which seems quite blunt really. It has delivered better efficiency, but it's because we've moved to better flow, and we've improved the pace of our work.

[09:51] So that was the why.

[09:56] So then we looked to what values were important to us.

[09:58] So what are the values and principles that we would want to stick to while we were doing this work to make sure that we were making the most out of moving to new ways of working? We started with looking at the Agile Manifesto. When we looked at the wording in the Agile Manifesto, which is for software development, and sometimes that's missed out of the title, it is very IT based. It was written for the IT world, so that's not a great surprise. When you look at the way that some of those values are articulated, and the principles are articulated, they are quite strongly IT.

[10:36] That's not that they don't work for us, it's just the way that they're articulated. We did go through a period where we thought we might just try and adapt them, and then we thought, "Well, actually, why can't we just create some values and principles that are our own and that we really want to stick to and are the things that we want to stick to?" And this is what we came up with. So we still want to, it's really important to us, fundamental to us, that we deliver valued audits.

[11:03] And in terms of those new ways of working, those would be underpinned by the principles of continuous collaboration, continuous prioritization, and longer-lived teams. And for those of you who've probably seen long-lived teams before, we deliberately changed that to longer-lived teams because when we shared this with our auditors, there were some that felt that they didn't really want to be in a team for 18 months.

[11:32] The thing that they enjoyed about their job of audit was that they moved around the organization, and that helped them to build a wider picture of the organization. So now we try to keep our teams together for a longer period, so around six to eight months.

[11:49] So that's why we use the term longer-lived teams.

[11:52] So those are the values and the principles that we try to adhere to.

[11:56] So when we make changes, those are the things that we come back to, to make sure that if we are making changes, can we still hold ourselves to this. In terms of our vision, again, looking at delivering valued audits. So that's our vision. We want to deliver valued audits, and that's underpinned by some things that really explains that vision, and that's efficient delivery, valued audits, and engaged teams.

[12:23] And all of those three things together focus on the work that we do from an efficient delivery perspective, the valued audits from our customers, our auditees. So how do they feel about the audits that we're delivering? And then our people, our teams themselves, are they happy? Are they engaged in the work?

[12:44] To make sure that we are moving closer to our vision, we've set ourselves some metrics for each of those to make sure that we continue to be quite balanced. Because if we focus on efficiency, that means that our valuable audits might drop, the quality of our audits might drop. If we focus on value, it may mean that actually our efficient delivery drops.

[13:08] And if we just focus on efficient delivery and valuable audits, that might not be so great for our auditors because we may be killing them by working them too hard. So we still want to make sure that our teams are engaged and motivated and they're working at a sustainable pace.

[13:24] So the metrics that we've chosen for those are lead time for efficient delivery, and that's the beginning to the end of an audit. Post-audit feedback for valued audits. So at the end of each audit, we ask our auditees, "Did you find that audit valuable?" Really simple question, yes or no. And then for engaged teams, we run a net promoter score on a quarterly basis. So we ask a question of, "Would you recommend internal audit as a place to work for your colleagues and friends?"

[14:01] And we use each of those things, and we monitor them to see whether or not we're moving towards our vision. In terms of those benefits and the metrics, so some of the results of that I just wanted to share with you just to show how positive this has been for us. So from an efficient delivery perspective, we've reduced our lead time from 16 weeks to 12 weeks, and we do over 100 audits over the course of a year.

[14:27] So that translates to quite a lot of extra audits that we can do, which I'm sure you will imagine the business are absolutely delighted about. But that just means that we can provide more assurance and more value to the organization. In terms of those valuable audits, 100% of our auditees now consider our audits valuable, and that's a brilliant place to be. And it makes me feel really good about my profession when I see that, and the rest of the team.

[14:58] And then our engaged teams, we run a net promoter score, which is currently running at about 12, which is positive. So there's still some room for improvement, but it's a good place to be. Interestingly, from that score, we have seen peaks and troughs in that score.

[15:15] And initially, when we started running it, it was running quite high, which was exciting and disappointing at the same time because I thought, "Oh, there's nowhere for us to go because everyone's really happy." But actually, over the course of the next few quarters, the NPS score dipped. And initially, I thought that was because we were moving to new ways of working, but it quickly became apparent that that was largely because people felt safer to share how they really felt about things. And we included some open narrative at the bottom that allowed people to feed back why they had scored that, and that gave us great insight into where we needed to focus so that we could address that, and then we could start to lift that NPS score.

[16:01] And as you can see, that NPS score is now starting to move up again. One of the added bonuses, and we didn't put a metric on this, I wish we had now, I'm not sure what it would be, but was that we now have a better understanding of the way that the rest of the business are moving to work.

[16:18] And that means that we can both support and challenge the business as they start to adopt new ways of working on a bigger scale. So in terms of changing our ways of working, we spent quite a bit of time trying to work through how we were going to change our ways of working. So there's lots of options.

[16:41] So when we looked at our context, we really looked at the flow of work.

[16:45] So most of our work is complex.

[16:49] So when you look at it, we don't know the answers.

[16:53] So we do some work, and we might need to do some testing. And as a result of that testing, that then determines what we do next. So that pops it into the complex area. And there's some that is complicated. So we might not need to do quite so much testing, and the results of that testing might not change where we go. But it's mostly in the complex space.

[17:18] And when we look through the Cynefin model, we look through the lens of that, that complex space is where agility and Scrum really sits. And we deliver our work using Teams, and that really made Scrum an obvious choice. So it's a framework to build high-performing teams.

[17:35] So who wouldn't want to build a high-performing team?

[17:38] It has an enforced inspect and adapt loop, which means that when we do our testing, we have a pause, we reflect on our findings, and that determines what we do next. We use Kanban boards to visualize our workflow, and that helps us to then collaborate and communicate both within the team and out with the team when we talk with our stakeholders and auditees.

[18:00] So we started with Scrum.

[18:02] And it was an experiment at the time. We didn't know how it was going to work. But actually, it's a really good fit for our team.So while Scrum is a really good fit for our teams, some things are helpful and some are not so helpful.

[18:15] So I just thought I'd run through some of those.

[18:18] In terms of what's helpful, the three roles and the four events are seen as very helpful. They provide us with a framework, and an understanding of what the expectations of individuals, and the expectations and outputs that we expect from, and outcomes that we expect from those events.

[18:34] Not all of those events were viewed as useful at the beginning. Some were seen as additional things that we might not want to do, like the retrospective. Why would you want to do that every two weeks? But the more that we've coached our teams, the more that they can see the value that those bring.

[18:50] And I think it's fair to say now that all four events are seen as really helpful for our teams. In terms of sizing, sizing has really helped us.

[19:00] That's something that we didn't use to do. We now break our work down to a day or a day and a half on each ticket. That's been really helpful in enabling us to identify where there are blockers and where work isn't flowing anymore. And that's something that we didn't use to do, and that's been really helpful for us.

[19:19] Planning has been really good because what it's helped us to do is to bring work forward, rather than happily going along and then having a hockey stick of activity at the end of an audit when the deadline is approaching. We now break that work down and bring work forward.

[19:40] And that just means that we are able to predict the outcomes a bit better. But happy path planning is very tricky with auditors. So that's something that we're mindful of.

[19:51] Auditors like to build in a bit of contingency. So when we're working with those teams and when we're coaching those teams, we try to make sure that there's no contingency built in because then we can't see where the blockers are. Visualization has been great from the perspective of the teams and also with the business. So that transparency helps both of them. So it helps to see the flow of work.

[20:17] It helps the business to understand what the work is that we're doing, where we're currently focused, so that they have a better understanding of what it is we're trying to achieve. And as a result, sometimes it can help prevent us from exploring areas that are really of no relevance, which is something that we used to do quite a lot of before.

[20:39] Before the last few months, electronic whiteboards were seen as a last resort, but actually we did start to use them. And for some of our teams that are not based in Swindon, their preference is an electronic whiteboard. I'm really glad that we did that now in retrospect, because actually that's been really helpful to help us to continue to work in this way.

[20:58] If we didn't have electronic whiteboards, it would make our lives a bit harder at the moment, and we'd have to be learning lots of new skills. And those whiteboards have helped us when we were physically in the building to co-locate with the business. And that's something we've been trying to achieve with the business for years.

[21:14] But now they can see the work that we're doing. There's the transparency of the work on the whiteboard. They're happy for us to sit with them. Whereas before they may well have felt that we were listening to every single word, and we weren't really clear what it was we were after. So now they allow us to sit with them, which is lovely.

[21:35] In terms of the things that are not helpful in audit, I'm not going to labor any of these, but some of the jargon doesn't really sit well. So I'm really conscious of some of the language that we use within, that's just common parlance in the IT world when they're using Scrum.

[21:51] But some of those words don't really work well within audit. So things like ceremonies, user stories, T-shirt sizing, planning poker, some of those things we don't even need in audit because it's a different context. And some of those words are just not appropriate for the world of audit.

[22:11] We don't need full-time Scrum masters in our audits. And the teams can't sustain it. We can't afford it. And so we have trained our auditors who have volunteered to be Scrum masters. We've trained them, and we coach them to be the best Scrum masters that they can be.

[22:31] And they're really good at it, and they love it. It can be a bit awkward sometimes when you have to then take your Scrum master hat off and move to being a member of the team again. But they are getting more and more used to doing that and maturing the way that they work in that way.

[22:52] Our audit teams are cross-functional from an audit perspective, so they are made up of different specialists, with different functional specialisms from within the audit team. But we don't invite the business into those teams, so they're not truly cross-functional. And that's actually just to maintain our independence so that we can maintain adherence to the principles of internal audit. We'd get into a lot of trouble if we couldn't show how we are being independent from the business.

[23:20] So that's something that we have a bit of a red line on.

[23:24] And that's the same for our internal and external events. So for the same reason, we don't invite them to our sprint planning meeting. So we work out what it is we want to do, and then we talk with the business about how we can do that.

[23:39] And similarly, we don't invite them to the sprint retrospective. Our audit teams feel that during the audit, it's not appropriate to be airing the dirty linen in front of the business, and I really get that. But what we do is we invite the business to a retrospective at the end of the audit, to get their thoughts on what went well, what didn't go so well, what could we do differently. And we also give them that opportunity every two weeks in the sprint review. So we still get that information, we just get it in a slightly different way. Okay, so I'm just going to finish up with the key lessons.

[24:16] So I'll run through some of the key lessons that we've learnt through our journey. So if I start with number one, number one is all about context.

[24:25] So context is key.

[24:27] We started with a framework that was built for IT, and we needed to adapt that so that it worked properly in our context, in an audit context. One size doesn't fit all. So even within the audit function itself, what works for one team might need to be tweaked for another team.

[24:45] Some of our teams are co-located in Swindon. Some of our teams are distributed across different sites. So again, even within the function itself, actually the context is different between teams. Leadership, making it safe to try. So we had really good engagement and support from our chief internal auditor, and she was so helpful. She was really visibly and vocally supportive on a regular basis of what we were doing.

[25:20] And where we were seeing successes, she was really keen to share those, both within the function and outside of the function. But also when things didn't go so wrong, she made it really clear that that was okay because we'd learnt something. So it wasn't a failure, it was something to be learnt from. And what that did then was to grow psychological safety. So then the teams were able to feel safe to try new stuff, and that has been critical to the success that we've achieved.

[25:54] If we didn't have psychological safety, people wouldn't have tried new things. They wouldn't have felt safe enough to try stuff. And that meant that we wouldn't have learnt the lessons that we learnt, particularly as quickly as we did. It's all about evolution, not revolution.

[26:10] And when I say that, so we never once called this a transformation program. We never referred to it in that way. But it has been truly transformational. The results we've achieved are transformational. And when I talk about evolution, what I mean by that is we started small.

[26:27] So we started with one team. We saw how it would work with them. At the end of that first quarter where we worked with that team, we took some time to reflect. We undertook a retrospective to say, "Okay, what worked? What didn't work? What do we need to adapt when we start to take this to other teams?" And then when we took that to other teams, we used individuals from that original team to seed into the new teams so that those new teams weren't starting from scratch.

[26:57] They already had the confidence of having somebody in their team that had tried this before. And with that, that meant that we could grow it. It's all about volunteers. So we've never imposed this on anyone. And that first team self-volunteered. We asked people to put their hands up and self-select who wanted to have a go at this first, and who would be working in an area where the business were volunteers as well. What's really interesting for me, on a personal level, is that I probably wouldn't have selected the team that first put their hands up for it. And that was a big learning for me really early doors, was that if you've got enthusiasm, enthusiasm, take it, because those volunteers will want to make it work.

[27:42] And that was a really big lesson for me. Treat everything as an experiment.

[27:49] So when we first started using the experiment, it probably wasn't in as common usage across the rest of Nationwide as it now is. But we found that referring to things as experiments meant that our teams would have a go at stuff, and that things weren't set in stone, and that because it was experimental, if it didn't work, it wasn't a failure, it was a learning, and we could roll things back. It wasn't a pilot.

[28:14] It wasn't to test something out and then roll it out. It really was an experiment, and just that shift of language really helped us. So we treated everything that we did new as an experiment. Measures and metrics really matter, particularly when we're looking at this. This is behavioral change, and inevitably, at times, when you're looking at shifting behaviors, that gets quite difficult.

[28:42] If you've got some data that shows that you're going in the right direction, that can help people to continue to face into some of the problems that you're seeing. So identifying what measures and metrics you want really early means that you can build up that body of data.

[29:01] So for us, for example, when people were finding it difficult, we pointed to the fact that our PAF scores, our post-audit feedback scores, were increasing and were higher than they had ever been in the last few years. So our stakeholders and our auditees were really enjoying it.

[29:19] So why would we stop doing something that was obviously hitting the mark with our auditees? Maybe there was something else that we could do differently. So those metrics really helped us to continue on our journey.

[29:35] In terms of changing those behaviors, we found that changing the system is probably the quickest way to change those behaviors. If you ask people to change their mindset and change their way of thinking, they might want to do it, but they're not really sure how to do it.

[29:50] If you change the system, you can help them to shift those behaviors. So for example, if I take that linear methodology, we had those gateways. We wanted people to move more freely between the phases of the audit, and sometimes do some testing before they'd done some planning for some less prioritized work. But while those gateways were in position, the auditors found it hard to move through different phases of the audit.

[30:22] Once we removed those gateways and we changed the system, that meant that they felt more comfortable being able to be in a different phase of the audit because it was a much more fluid approach. Develop within. So it might need small and targeted investment in order to make a difference to this.

[30:45] So in terms of that small and targeted investment, I was taken out of my day job as an auditor, and this was the focus of my work for the whole time. And sitting alongside me, we had an agile coach. So we needed to spend a little bit of money to bring in that expertise.

[31:04] But that means that you retain focus on it, and I still believe to this day that if we hadn't have done that, we wouldn't have achieved as much as we did, because it would've been side of the desk. So other things would've come in as a priority, and when things got difficult, we probably would've just not stuck at it. So taking people out of their role and having and using that targeted investment has been really helpful for us.

[31:32] And then above all, it's a means to an end, right?

[31:34] So what we are trying to achieve is better flow and increased value for your auditors and your auditees, happier teams for your auditors. So if something doesn't quite work for you, one of my key lessons is, well, let's see if we can do it in a slightly different way and adapt it, not stick to something if it doesn't work.

[31:57] Understand what you lose as a result of doing something differently, but that doesn't mean that you have to stick to something slavishly. So I'm going to finish there. I believe we have a Q&A session next, so I'm really interested to hear your thoughts and get your feedback, and hopefully answer some questions that you may have. Thank you very much for your time today.