DevSecOps Security Champions and How to Implement Them Successfully
This session is presented by Sonatype.
Chapters
Full transcript
The complete talk, organized by section.
Robbie Tyrie
Today I'm going to talk about the security champion role and how to do it right. This is something I've done in a few organizations lately. It gets massive benefit between the development team and information security.
It fits into the whole development lifecycle. This is one of Nexus's screens that I've put in here because predominantly it's going to touch on quite a bit of open source coding as part of this role as well.
Who am I? I'm not an expert by any stretch of the imagination. I've learned a few things along the way. Just a word of warning here: other Trump images may appear in this presentation at some point.
I've got a varied background in information security and IT in general. I've got over 20 years' experience. I started at the Scottish Government on the Year 2000 project, which should give you an idea of things. That was when Windows was still on 3.11 and DOS was still a major thing back then. I moved on to Police Scotland, where I worked on some decent projects, such as G8. I did solutions architecture at the NHS and then moved on to the private sector for Tesco Bank, where I helped build the bank from scratch. That was more of an infrastructure role there. Then I moved back into security, because I did quite a bit of security at the police.
I moved back into security at JP Morgan to help set up their DevOps management team there. That was 21 developers across the globe managing their entire estate, which was quite a big project. I then put in a secure software development lifecycle at Clydesdale Bank, moved to Virgin Money to do a bit of cloud migration, and at Aegon, where I am just now, I'm an information security manager responsible for a multitude of projects. I have put in a secure software development lifecycle there as well, because I'm an application security specialist, but I've also worked on projects for database security, Splunk, and cloud security. So that's a bit about me.
I'm going to discuss why we need a security champion. Firstly, we need to know what the problem is. There are multiple projects on the go, many development teams across different languages and technologies. Some of the staff might be outsourced. You might use a third-party supplier. There are thousands of software components, especially in the open source landscape. And there's no security culture in the development community. One thing I would note here that I've seen everywhere is delivery is king. We need to solve that, and security champions are going to help with that one.
How do we create a security culture? When you're doing the testing of the app, do you take security into consideration? That's one question you should be asking yourself. Are the tools appropriate and configured to the business requirements? Do you know where your critical applications and functions lie? These are all questions that you need to ask yourself when you're looking at a security champion and your full security culture as well.
What is a security culture anyway? It's having an open mind about security in your project. You really need to take security into consideration and have a responsibility for your code as well. You want to be sharing knowledge across your teams. Don't keep everything to one area, and have an awareness for security. That can be things like attending seminars and meetups. Have a personal engagement. You want to take an interest in InfoSec. And the biggest thing is getting management buy-in as well. It's really taking responsibility for your code quality, and being aware of security issues and trends.
What are the blockers for implementing security champions? Obviously, everybody's too busy. Everybody's got demands to meet and project deadlines to adhere to. Risks are not relevant. That's one thing I've seen time and time again, where management will say, "Oh, you've got firewalls that are controlling your security estate. Why do we need to take your software security into consideration?" Obviously, saying that if you're going to do this, do it only as a pilot project as well. Well, yes. If we do it as a pilot project, then you'll see the results and you'll want to roll it out everywhere.
The other thing, especially in an agile environment where everybody's moving to now, is that change is too fast to consider information security. I've seen teams that are releasing three times a day. If you find a vulnerability in any of your code, then that's your whole pipeline blown apart. Again, I'm going to mention this time and time again: delivery above security. Delivery is king in many, many organizations, so security seems to be an afterthought. People are scared to find issues which will take time to resolve, and that will impact your delivery schedule. These are the reasons why it's difficult.
But if you've got that attitude, you're guaranteed to hit some problems. You'll end up with a massive backlog of technical debt when you do decide to do it. My advice would be: if you're thinking of doing secure software development and you're putting security champions in place, do it sooner rather than later, because the later you leave it, the bigger the issue you're going to have on the other side. It's only a matter of time before a security incident occurs, especially in the open source landscape when vulnerabilities can change on a daily basis.
That's one thing that we've seen a benefit from the Sonatype products. We can see a history of vulnerabilities in each of the library files and what versions to move to if you do have a vulnerability. I'd say that's been a massive help for us. But if you don't look at it, then it's going to blow up in your face sooner or later. So what's the solution here? The solution is to implement the security champions.
What's a security champion? I've just got to read off the OWASP sheet here. Security champions are active members of a team that help make decisions about when to engage the security team. They will act as a voice of security for a given product or team and assist in the triage of security bugs for their team or area. There's a link there if you want a more detailed description from OWASP, which is quite helpful.
Who are the security champions? You've got your developers. It can be developers, architects, or a project manager, the guy sat on the desk here, or the boss. In my experience, dev leads make ideal security champions. They've got a vested interest in their code, they know the code inside out, and they're the ones that can talk on a technical level with your information security team.
What I would say makes good security champions is people who've got good communication skills and are also happy to talk to their management team as well and raise what the problem is. If you find people with these skills, they're pretty good. But the purpose is to self-manage wherever possible. If an issue's found during your scan, ideally, the security champion should be able to fix it. It's only when they can't fix an issue themselves that they should be working with the information security team.
What are the benefits? The benefits of having a security champion are that you're embedding security within your team, engaging non-security people, and creating a security culture. Security is no longer at the back of your minds. You'll find the defects earlier, and you'll be able to make informed decisions.
Some of the good attributes for security champions are to get them to learn about the OWASP Top 10 for web apps, secure coding methodology, and basic threat awareness. If they're skilled, they can learn more about the development lifecycle, cloud, or new technologies, for example, and then they become a champion. Champions, in my experience, know how to use the scanning tools, verify the results, perform code reviews, be involved in the release project, work with information security and release management, and help their other development colleagues fix security problems early.
You can have different levels of security champions here, so there's career progression as well. That's another way to sell it. Ultimately, there has got to be a single point of contact for that particular area. It doesn't have to be a dev team itself. You can do it on a project level, or on a product level as well. I know places that have done it based on products. But I've found it worked best based on a technology stack such as Java or .NET. You have a single point of contact for each. It does help the information security team greatly if you do it that way.
The expectations of a security champion are to share knowledge between their colleagues, helping the decision-making from an information security, management, and release management point of view. They carry out security reviews, so doing code reviews for more junior staff. They monitor vulnerabilities within the code scan results. There may be licensing issues where you can only allow so many licenses for your scanning tools. I've seen that at some places, and the security champion should always have access so they can see what the results of the scans are.
They should be able to help prioritize the security issue in the backlog. Information security can help them there by working on the most critical business apps first or doing risk assessments. Have an input into the release process. Buy-in is another good one if you're a security champion, because you've got to be quite popular.
This is a bit about the timescales and how to actually do it. Ultimately, I've seen it done typically between four and twelve weeks to implement a security champion role from scratch and management buy-in. It depends on the size of the organization as well. Ultimately, you identify the team, define the role, nominate the champions that you're looking for, and set up communication channels.
We do a lot of work through Jira. That seems to help when it's all audited as well. If they're passing queries through Jira to information security, then that does help build a knowledge base. You can also do lunch and learn sessions, which is quite helpful, and maintain an interest. That is key. Once you've got the security champion in place, you want to make sure that they remain on board and they're still keen to do it after a few weeks when nothing's happening because you've got your secure lifecycle in place and it's all working smoothly.
One thing that we've seen with the security champions, another benefit of having them, is not just on the application side of things. Because they're a single point of contact for a particular team, if there are other information security projects on the go, they're the first point of contact that InfoSec will reach out to for any queries or to get their opinion on things. So it's wider than the whole application and threat landscape as well, which is good.
When you're nominating champions, it's always a good idea to get some help from your information security people, because they're the ones that have got to be working side by side with them as well. They should have an input into the nomination.
The first step is to identify the team. I've spoken about this already. You can do it by technology stack, which I think works best. You can do it by release as well, or by product. But again, InfoSec should have an input onto this one. You can have multiple security champions per team. Deputies are a good idea, to be fair. When you're on leave or in busy periods, you can speak to a deputy, but ultimately, it should really be a single point of contact for that particular area.
Defining the role is the next step. Measure the current security state in the team. You want to know how mature your security landscape is. Knowing what the skills are for particular developers, architects, or whoever is going to be a security champion is a good one. Define the short- to mid-term goals. For example, start off light: just scan a couple of apps and get the security champion to have a look at it. Find out what they want to achieve as well. Identify areas where your security champions could help. If you've got a lot of bugs in your code, that's probably a good area to start in. Provide clear, defined roles for the champion.
Setting up a RACI matrix is good as well, so they know their roles and responsibilities. What I've seen typically is verifying the security output of the scans, actually conducting the scans themselves, being involved in the release process, and doing code reviews as well. Looking at technical debt is another thing. If you're new to implementing a secure development lifecycle, then when you put it in, in the first instance, you're going to have a massive backlog of security vulnerabilities, your security technical debt in effect, that you're going to have to work through. Involving the security champions in that process is going to give you quick wins because they'll be able to tell you quite quickly what they're able to fix fast. Ultimately, that's what management's looking for. If you see a problem, you want it fixed quickly.
Nominating a champion is the next thing. They should have a very good understanding of the team's goals and how it operates. You'll need to get approvals from all levels. I cannot stress that enough. Otherwise, you're going to hear, "I've got no time for security." You're probably looking for experienced team members here. You must have commitment from senior management. One way that I've done this in the past is to add security-champion-specific objectives onto their personal development plan, so management can see the benefit quite quickly as well during official reviews.
Ultimately, if you don't get approval for this, there's no point in doing it. You can do it off the side of your desk if you want, but if you've not got management buy-in, you're completely wasting your time with this one. You really need to get them on board.
Once you've got your security champions lined up, make them feel like a champion. Introduce them to their peers. Get them into mailing lists. Make sure security knows they're a single point of contact. Give them a pay rise, or promotion in some cases. Get them known throughout the organization as well, that they're a single point of contact to go to in the first instance for application security queries. That's quite useful from information security's point of view, as you can probably imagine. Again, buy them a beer if you're in the pub as well. Make them feel wanted.
Setting up communication channels is the next step. Put them on mailing lists and Teams groups. Have regular meetings with your information security team as well. Every couple of weeks, get together and review what's been happening. Just have a general catch-up chat. Lunch and learn sessions are another good thing as well. Also get management to understand the role in detail, so invite management into some of these meetings as well, just so they're aware of what's happening in the application security landscape also.
Build a knowledge base. Have a security meta team or list of security champions. Clearly define the roles and responsibilities. I've mentioned RACI matrix beforehand. I'd definitely do that. List your risks and vulnerabilities that you're currently aware of, and then that can help you focus on training for the developers as well. I could name specific decent training courses for developers, but I won't. You can reach out to me if you want a heads-up on that one. Training's quite a big thing. That's something everybody should be doing.
What's next? How do you maintain and grow your interest? Develop a community of practice. Define your secure coding standards. If you're doing web apps, you can base it on the OWASP Top 10, for example. Have a set of rules of the road for developers. What I mean by that is that you'll mandate scanning at each point of the development lifecycle. You'll have toll gates built in with checkpoints on what you actually do if it fails a toll gate.
What I've done in the past is set blockers up in the pipeline so the developers cannot merge their code back into the repository. If a vulnerability is found, they have to fix it at that point. That pretty much guarantees secure code in production as well, which is a good thing to do. Branching strategy is another thing you can align across your organization, and get them involved in new information security projects as well.
Have the security champions involved in decision-making and information security. If you're looking at strategy, for example, it's good that the champions can be involved in setting the InfoSec strategy because they've got their feet on the ground at the end of the day. Setting up monthly or quarterly review meetings is the next thing. Maintain the integrity of the role. Get the champions empowered to make decisions on InfoSec, give them feedback for their end-year appraisal, and put them on any appropriate training as well.
This is just an example of some scanning tools that are out there that you can use. I've got Sonatype, which is where I've seen the biggest benefit, especially with open source vulnerabilities as well. We're seeing the most issues there, obviously, because they can be written by anyone. Nine times out of ten, when somebody brings them into your organization, it's good to have a check there. Although we're seeing the most vulnerabilities, that's where we're also getting the biggest benefit. If you're going to focus on one part of the software development lifecycle and you're using open source, 100% of the time it would be that I'd be focusing on.
Here are some tips for you. Use a framework, especially if you're working with third-party suppliers. For example, we're using the CVSS scoring system. Anything that's rated seven or above, we'll want builds to fail. That's not only useful for internal developers, but if you're working with third-party suppliers, it gives them something to focus on as well that's understandable and doesn't match against any particular security scanning tool's rule sets. You're working on something quite generic there, which we've seen a massive benefit from as well.
Dashboards: create a dashboard. Dashboards are great. Management loves dashboards and reporting. To be honest, in all the projects I've worked on, when we're setting up secure software development lifecycle, it's the first thing I've been asked for: when can I get dashboards? When can I get reporting out of there? Ultimately, at the end of the day, this is what management wants to see. Doing trend analysis is one thing that I found to be pretty beneficial as well. We can do that for a particular department or particular product as well if we want to focus on a micro level. That's one thing you want to be considering from day one: looking at your dashboards.
Where does this leave it? What's the actual benefit? I'm going to put a slide that I absolutely hate seeing: shift left. Everybody mentions shift left these days, but it is true. The further left that you're doing your scanning, the fewer issues you're going to get in production. That's a good process.
There's another slide I'm sure everybody's seen as well: plan, code, build, test, release, deploy, operate, monitor, with continuous monitoring there. I'm going to focus on release here next. Right at key in the middle, that's absolutely key. Because delivery is king, release management must be kept informed, and they must be bought into the overall process. If they're not, you're going up the wrong tree here, I'm afraid. You need to be able to say to release management that if you've got a vulnerability that's critical for your organization, you have to stop releasing. That is the most challenging part of your whole secure software development lifecycle, and this is where the security champion can work with release management and be key to the whole process.
In some cases, do you actually know what code is released in production? Do you know what your code quality actually looks like? You want to make sure that your voice is heard during releases. Make sure information security has got a seat at the table. Define your toll gates and work with release management on what works with them also. Look at your blockers or your risks from an InfoSec perspective on the landscape.
Have an exception process. This is a good point. Especially in the open source landscape, some library files may have a vulnerability, but there's not a fix available for it. You can't even roll back to a previous version, and there's no newer version that doesn't have a vulnerability. What I've done is time-based waivers. If there's no business impact and you've got the right security controls and InfoSec is satisfied, I would recommend giving a 90-day waiver process as a maximum. That way, after the 90-day grace period is up, potentially there could be another open source library file available which can be added into the code, or at that point you can discuss with information security about re-engineering the whole code base to remove the usage of that library file. That's a worst-case scenario, but it's one that I've seen elsewhere in the past. Ultimately, you want to be taking a risk-based approach to things here.
How do you get management buy-in? This is always the big thing. Ultimately, I use scare tactics. That's worked nine times out of ten. Be brutal with them. For example, at Tesco Bank, they got fined GBP16.4 million for having a security breach. When you start banding figures like that about, they'll sit up and pay attention. But ultimately, sell the benefits, make allies, get people in the business who you're allies with on it. Do a proof of concept; that's always a good way to do it. Start off small and grow from there. Do trend analysis of reports, charts, and statistics. That's all management cares about, these statistics. Again, buy them around if you have to. Start small and start delivering results that way.
Just to have a recap here: know how to define a problem statement, know what it is, identify the champions, define the role, set up communication channels, and the key one there is management buy-in. If you've got all of them in place, you'll have a security culture in your organization.
That's it. So it's now over to you guys to do likewise. If there are any questions or things, you can post them in the chat. And that's me.