Security + Devops + Ambidexteriety = Fun
Visma embedded security into their DevOps transition from early on. This has led to advances in both fields that makes words like DevSecOps, SecDevOps redundant. DevOps seasoned with security is natural, fun and really empowering for the DevOps teams.
Espen shares the Visma story in a fun and engaging way for those that want to avoid security becoming a delay in DevOps and rather control it from the Developers perspective and accelerate development.
Chapters
Full transcript
The complete talk, organized by section.
Espen Agnalt Johansen
Hi everybody. I'm Espen. I'm the father of three daughters. I'm married to Trina. I have a dog and a cat, but most of all, I love security. I love security, and I spend most of my professional life doing security in some sense. From both work in the armed forces when I was a younger man, working for the United Nations, to working as a manager and operator in many companies. I've also worked as an undercover cyber operator for clients for many years. The main job was to infiltrate criminal environments and learn how they worked and figure out how my client's IP was abused by them. But I also am in my fifth midlife crisis, and that I take great pride in. And this one is about sailing, and I'm completely immersed in that one. I even have pictures on my wall depicting sailboats. I dream about long cruises to faraway shores. But mostly, I actually spend time on YouTube learning about sailing. And also buying the cool stuff that goes with this particular midlife crisis. And I recommend all of you to have a couple of them. They're really fun. Declare them out loudly, and since this creates the opportunity to do something really fun, and people accept this ridiculousness. So back to security again. So I've learned over the years to respect many different trades, and I've also learned to be curious about people and how each and every one can contribute to delivering top-notch security in their services. So in this talk, I will try to make our research more available, and I will try to make it easygoing. For those who really want depth, you should read the research. It's well-published. I, of course, would love if any of you take the time to read and comment on some of the publications that we have made. In this talk, my focus will be on security, DevOps, and ambidexterity combined. I will also share how we have experimented with using ambidexterity as a method to merge these into a more potent tool for the self-managed teams.
Some developers and operations people view security people like this. And we also see that people like this, classical security guards, sometimes want to subsume others in their own paradigm. These people are often seen yelling things like ISO 27001 or similar words. While some security and operations people view developers like this, like kids with a keyboard and little impulse control. And they also want to subsume others in their own paradigm. They're often seen yelling words like DevOps, Agile, and Jira, and even Confluence sometimes. And some security and development people can view operations people like this. Kind of stressed, rather skilled, and have a need to subsume others in their own paradigm. These are often seen yelling words like ITIL, network, backup, and service desk. But what I've learned over the years is that everybody just wants to be a ninja. So this is what all three characters in this narrative agree on. But can we really become ninjas? I've also done lots of martial arts in my life, and I know that to become really good at that stuff, you have to do that and not anything else for a very long time to become really, really good. But all of us have to agree that we are involved in the same circle of life as depicted here, and therefore might argue that we're all the same. So why do we need this to put emphasis on the differences, where I should look at the similarities and find ways of breaking down the walls and acting together? Our practices and experiments suggest that if we speak the same language, if we work together, we can accomplish a hell of a lot more than if we fight and focus on the differences between us.
So in the DevOps infinity model, all three competencies are tightly interwoven. And also, we have to agree that we're all just humans. Even though if you take on the form of a developer or a security person or an operations person, we're all just people. We have slightly different views, slightly different ways of working, but all of us have different competencies, and these are complementary. That's at least what our research suggests. So one thing that's in common is we're all humans, and humans, as a species, are territorial. And since we're also an apex predator, we only have to fear ourselves, don't we? So with that in mind, so we are territorial. We do have different opinions. We are quite good at what we do, all of us. So why couldn't we be more like quokkas? So quokkas are the world's happiest animals. They live in Australia, where basically everything wants to kill you. But trust me, yeah, a quokka would probably hurt you, but it can't because it's quite small and it's nice, isn't it? It's a lovely looking creature. They're not territorial, and they're happy to share space, even food and shelter. So despite being hunted by predators, like we are hunted by hackers, aren't we? They have a chill view of life and are happy to share both information and everything else. So let's be more like quokkas.
So with that in mind, I'll bring you through the outcome of the research that we have done. So for people investment, this is a mantra for management to reflect if you have empowered your people, are they enabled, and have you embedded and ensured the two above? The small cutout in the bottom of the screen is a part of our security strategy for the company and where we try to exemplify this.
But ambidexterity can be a bit confusing in the beginning. Let's just take you back to the roots of what ambidexterity is, because DevOps is, of course, a word here that everybody understands, but ambidexterity might be new for some. So ambidexterity in itself is the ability to use both the right and left hand equally well. So left and right equally well. So let's do a small exercise together. So now have both your eyes open. Extend your right arm and point to the word "equally." Okay, now close your left eye. Open it again. Close your right eye, and open it again. So most of you will experience that one eye is better at aiming than the other. The vast majority of people have either the right or the left eye as their dominant eye. So in the shooting circles where people kind of shoot guns for fun, this is known as the dominant eye. So even inside your brain, you have dominance issues. Let's do the second exercise. This is quite fun. Now put both hands in front of your face. Let your index fingers point at each other. Now rotate the right hand forward while the left remains still. Stop the right hand. Rotate your left hand backwards while the right hand remains still. Okay. You all got this right, and now stop the left hand. Now you take the right hand forward and the left hand backwards at the same time. Come on. We start. And we stop. Now the left hand forward and the right hand backward at the same time. Stop. Most of you will realize that this was difficult. Some of you are really naturals at this, but most find this to be quite difficult. And even your body does not intuitively work with you. So even inside your own body, you have dominance issues still too. But there is hope. You can train yourselves to become better if you want to. Our way is to use a diverse group of people and make sure they are empowered and accepting that one competency cannot always rule the others. There is room for all if you make some adjustments. I myself, I chose to share leadership, or I choose to share leadership with people that are my complete opposites. And I respect their opinions, and I learn from them. And I've accepted that I should not subsume others in my own paradigm.
So when it comes to security, organizations usually are focusing on applying the top-down approach. I'm also inclined to do that. As I mentioned earlier, I'm ex-military. An ambidextrous governance model can be adapted from the healthcare industry, which combines a top-down approach by establishing protocols and penalties, like standard setting, monitoring, and accountability, and a bottom-up approach by helping the teams to become better in doing security activities, which basically is training and stimulating interventions and stuff like this. This approach basically assumes that the teams are self-managed, agile teams that need leadership on security, but must retain the continuity and self-management of their software development, ensuring the adaptation and implementation of software security practices, enabling and empowering software development teams to adapt and add to overall mandates, and embedding cultures of improvement.
So when I break it a bit down, let's go into ensuring, which is the first of them. This is about adaptation of evidence-based best practices. So for us, if you're going to look at the theory behind, there's a top-down role in the program. It's just focusing on creating an overall strategy, embracing the performance and process standards, and defining the roles and the responsibilities. So this is where we make sure that security requirements are based on a combination of customer needs, best practices, and in compliance with privacy laws and regulations, for instance. It also mandates for me, as a leader, to create a financial platform for later activities to thrive. So given the main three characters of this narrative, remember the police guy, the kind of kid with the keyboard, and the flaky operations guy, this is also an activity where you form, made from a top-down perspective, decide to break up silos and put these two competencies together in the same reporting chain. Well, it can work for some. Doesn't have to work for everybody. Our experience, at least, show clear indications that this works best when they share goals and it's an easy approach to move them into the same team. This force a common goal. But it might not suit all of you.
So the quadrant two identifies basically a persuasion-oriented capacity building role for an organization is to enable employees to engage in bottom-up improvement activities, including adapting activities. So the focus is on education and training, enables action on the basis of performance information. Although potentially counterintuitive for a bottom-up approach, the need to spread improvement capacity across the system means that the centralized provision of change resources and training, as well as networks to spread learning is appropriate. Importantly, like the provision of information from other researchers suggest that education and training complement other regulatory instruments. So in very short, give someone the means to do something. So when I translate this into a very simplified context, it's using a bottom-up approach that focuses on persuasion, education, and training. Okay. This is the key of this one. So when I take this into the technical context, so we started quite early with static analysis. Static analysis is a very technical exercise. So for me, as a traditional military officer, I thought, "Okay, let's implement SAST, and let's turn on all the compliance elements in it." And let's break build if there exists any kind of poor version of encryption present in the code. Well, that doesn't work. It really doesn't work. So we had to reimagine this, and we thought more of SAST as a training tool. And when we started seeing it as training, then it became really good. So we learned from our experience to show that DevOps teams, they stop making security bugs because it's the DevOps teams that make the bugs, and it's not some kind of magic that injects the bugs. It's the developers who make them. So they stop making them. Some very fast, just two weeks. Others needed two months of exposure to SAST in line. Other kind of methods of implementing SAST gave more time for people to learn. But essentially just see SAST and other testing rigs as training instead of seeing them as compliance tools. This really helps when you get the adoption rate up. So from the intelligence perspective, we can see that ransomware is quite popular amongst criminals these days. So maybe a small tabletop exercise for management, easy-to-use stuff so they can familiarize themselves with what to do if things should happen. Basically prepare them and avoid what we call the flapping of arms when something hits them. Makes people prepared. So train hard, fight easy. Part of it.
The third quadrant notes the important persuasive organizational role in empowering employees to utilize their bottom-up improvement capacity in their own organizations. So empowering is basically creating a supportive climate for service improvement. This is the core of the empowering exercise. So for me, when we did this, it is trying to understand, like in the example before, when we had these three characters. It is that they need to be empowered as a team. They need to be able to make the decisions themselves. And this came from a profound learning when we really dug into lots of the various incidents that had happened over the years, and we figured out that the only one who could actually fix a problem in any of the products, well, it's basically the DevOps team, isn't it? So if there's a breach, who has to fix it? It's the combination of development, operations, and security. It's not the security guy. It's not the operations guy. Probably it is the developer, but it might be a joint effort. So they need to be empowered. And this is also very important for the innovation part in our company. So when they're not empowered, they are more compliance-oriented. So, and for this activity, it's important to give people a voice and to utilize their bottom-up improvement capacity in their own organizations and let them know that in their context, they have the power to decide. One of the things that we have done to experiment with this is to establish what we have called the Trust Center, which basically is a public-facing page, where we have listed information about the various services that we deliver to customers. And we have also made it abundantly clear that the ones who will participate in meetings with customers are not sales. It is the actual developers. They will have to meet the customers from time to time. And this is a really, truly empowering act because the customers are the ones who have the power in any company that does in private industry. So empowering is for me key, and it also comes at a cost. Some parts of the empowering is to accept that me as a director of security might not always be the one who should make the decisions on security. And I've learned that that is a very good thing when you reach that realization. You release that self-insight. Decisions have to be made by the ones who can do the tasks to fix them.
The last quadrant talks about embedding, and the buzzword is to incorporate as an essential part or characteristic. Well, this is for us, one of the absolutely most important components. So if you reflect on this, so the traditional way of doing security from my own perspective is to write some kind of policy and then publish the policy and maybe even have some kind of system in place to determine if people have actually read the policy. So by reading things, you assume that people will do as you tell them. Well, that's not how the world works, to be quite honest. So in the embedding part, it is the importance of organizations engaging in top-down efforts to embed the culture of improvement, innovation, and learning. This comes via board policies and priorities, clinical governance, local improvement support, and celebrating success. So this can arise via volunteerism. It doesn't always happen like that, and also in response to kind of accountability mechanisms like performance and process standards and things like this. So from my perspective, it is about making the act natural. For me, it just made it important to incorporate the security work in day-to-day activities, rewarding good behavior, and at the same time not rewarding negative behavior. So in this context, I can give you an example. This is our dragon. We just call it the dragon. If you can see, this one is basically a quality management system or an information security management system, but it's made in the context. It's made in the same way that it can be recognized by both developers, from operations people, and also from security people. It kind of resonates well because it details the entire life cycle. In this, you can view one of them, for instance, on top. You have a small crow. This is an important part of the embedding and also the empowering. The crow signifies our intelligence program. So if I expect the teams to make good decisions or make good security decisions every day, I have to give them at least the same information that I have available. So I'm lucky enough to be in a company that is quite well-funded and works well. So we have devised a quite extensive intelligence program. That means that the developers and the operations people and the security people have the same information available for them. So when they make decisions, they make them together, and they make quite good decisions every day. The other one is the security self-assessment, which basically is a template with lots of good questions, where we have tried to learn from the rest of the world, both from OpenSAMM and many of the other very good schemas or frameworks out there, and tried to translate this into our context. So all of these services that basically is depicted in this image in the dragon, represent individual steps that people take.
So we have chosen to organize this in a small but very skilled team of experts in the middle. These are the security people. They deliver security services. So the way we started, we just started with what we had and moved from there. We had already manual vulnerability assessments. I did have a colleague that was very good at penetration testing. We also did have security self-assessment. We had some building blocks when we started this process. But we chose very early on to invest in static analysis. Since most research and our analysis that was in Visma at the time indicated we could get lots of good effects with this investment. And I can testify, and you can also read it in the research, that it's worked quite well. So SAST, if you don't do SAST, please do SAST. Very simple. Then we just added more services. All of them are based on the same fundamental principles. And in this, you can see that we have added, for instance, CTI up in the right-hand corner, cyber threat intelligence. We have included some more training platforms. But all of them are, again, under the same embedded extras model. We're not here to decide, we're just here to help.
Today's status is roughly around this. We deliver these services. And one main focus for leadership in this, both for myself and others who are part of the leadership, is to balance exploration versus exploitation of these services. The exploratory part is important. While the use of lean methodology and exploitation means that we have to keep a focus on cost efficiency while delivering the services and cultivate the mindset of closing down services that do not make sense anymore because they all evolve over time. For that purpose, we have used something called retrofitting as a methodology to analyze and understand. This will, however, be much more detailed, described in a new publication that will be released shortly. That's going to be a cliffhanger for you. It's going to be coming out, new research, and I hope you'll be reading it. So parts of what we have learned is that this is a really sustainable method of delivering security services on. And it really works. It actually works, this one. So the next big plan is to embed more services in this model and maybe merge some more that are complementary. And all in the spirit of exploration and exploitation. If you're, for instance, coming from a small to medium enterprise, like Visma is quite big.
So just remember that security in your development doesn't come for free, and it's not a freebie where you can just buy a product and then you're done. You can use available tools from your cloud provider. So like Azure DevOps, well, they have lots of good things going there. And if you have really good people in your dev teams, some dev teams are just one person effectively. Can be unicorns out there that can do all of this together. But the kind of fundamental learning from us is that security is not free. It comes at a cost. And for us, as a big company, it was quite needed to focus our attention on AppSec as a separate domain at first, and now spreading the same thinking to other parts of security.
So this is how we manage the managing directors, and also how to give advice to the DevOps teams. This is what we call the security maturity index. And all the services are visualized in indexes. They're carefully crafted for different contexts to make sure they're relevant and give proper guidance. So when they perform well, there is no need for actions, like indicated here in the smiley. So that this is two services, basically, the one on top does well, the one on the bottom has some issues. And it's important to try to contextualize this, depending on the audience you have available. So when they perform well, there is no need for actions. You don't have to do anything as the MD or the team. You are doing well. On the bottom, they are having some issues. And the kind of tools that we have provided them with gives them some clue of where to look. So some of this might be a management issue. Maybe they haven't spent enough money, maybe they need some skills, maybe they need just some love and affection. So it's remarkable in our research how much just paying attention matters. Just see people and be there for them. So when you in the audience who are managers, some of you get lots of questions from your sales and marketing that you need to have certification A or B or C. Well, this entire model is data-driven, and the processes basically produce the data in a systemic way dynamically, basically every day. So this data is then used as evidence for auditors that the processes are actually functional and adhered to. So remember this, if you really are onto certifications and all that stuff, you have a choice. You don't have to have the stressful preparation sessions that most people are used to where you have to flap your arms again and then ask everybody to conform to the processes at least when the auditor is here. You can just make the processes produce data that validates that they work. And this is how we have chosen to do it, especially in one of our models that I'm very proud of is the Visma Cloud Delivery Model. And if you search for that word on YouTube, you can find a brilliant video of one of my colleagues where he explains that in more detail.
So if you remember from the start, are your people empowered? Are your people enabled? And have you embedded and ensured the two above? This is part of our learnings, and we really hope that you are still curious. There's much more data and research for you to enjoy if you would care to do that. And we're also publishing more research in this field as we will progress. If any of you would like to make contact with either me or some of my colleagues, we're always willing to share.
And as a final thing, remember the quokka. The quokkas are not territorial. They are happy to share space, food, and shelter. We need to be more like quokkas. So we're all in this together, both developers, security personnel, and operations personnel. But since we are all territorial, it doesn't make sense that only one of us decides to, does it? So we have to get over our very internal language, and we have to start communicating in the same language and work better together. Thank you for your time, and I hope you have enjoyed this talk.