Tracy Bannon

Hi there. My name is Trace Bannon. Today I'd like to talk with you about something that I call #NoHobbyists. It's really an approach to helping to shift cybersecurity. Let me open up a screen share and tell you a little bit about myself, and we'll hop into this content.

My name is Tracy. I go by Trace. I'm a senior principal with the MITRE Corporation. We're an FFRDC, a federally funded research and development corporation chartered by Congress to provide cutting-edge concepts and technical leadership to the U.S. government. I spend my days as a software architect and engineer, and I love to focus on really difficult problems and helping other people solve difficult challenges with software. On the right-hand side you'll see a couple of hashtags that are often associated with me. I'm also an ambassador for the DevOps Institute, as well as being part of the Value Stream Management Consortium.

Let's get into the real topic today. Look at the screen. This should almost frazzle you a bit. I looked at this in the early spring and I decided not to update it, because it would take a lot of time on a daily basis to provide all of those new security breaches and incidents, concepts, and attack vectors that are cropping up: everything from prison data breaches, healthcare data breaches, SQL injections, GitLab having challenges, and the Red Cross being hacked. No wonder there is such an imperative to figure out how to improve cybersecurity.

The first thing that we often do is say, hey, we need to shift this left. Let's shift cybersecurity left. But what does it really mean? It means that your security activities start at design. It starts at vision. It extends the entire way throughout the SDLC, from vision to fielding operations to production, and there's continuous feedback at every step.

I often like to add on to this and say, after you shift left, then you need to really shift everywhere. Make cybersecurity ubiquitous. Take a look at the diagram on the right-hand side. What's important is that security boundary is around everything related to DevOps, everything related to how you are creating, delivering, and operating software. Cybersecurity and cyber resiliency are everybody's responsibility.

Anytime you go through shifts, you're going to fail if you don't address all four facets. You must be holistic about it. You've probably heard these things before: people, process, tech, and culture. But let's dive into it just a little bit.

When you're talking about the people aspect, you do need to net out what different roles are going to be, and you need to have the tough discussions of what the autonomy situation is. Autonomy for a small team is not the same as autonomy for the overall enterprise. There are different scopes of autonomy. Have that discussion. Also understand where people are and what their upskilling and training needs are.

From a process perspective, you want to understand what your workflows are and define your ceremonies and such, but you also need to consider whether there is anything special about your domain, about the context for the type of software and the systems or solutions that you're putting together.

From a technology perspective, obviously you need to address whether it's new technology, whether it's greenfield or brownfield. But what about digital platforms? Do you need an underlying digital platform? Gartner has said that in 2023 that is going to be one of the biggest trends on the horizon. You need to have that conversation and address that together. And what does production really look like? Are you deploying to the cloud? Are you deploying it to an appliance on the edge?

Culture is the toughest one. This is actually where projects are made or broken, where programs fall apart. While you may address the technology, that's the sprinkles on the ice cream. The tech is the fun part. The process becomes more natural. People start to work together. But if you're not very specific about engaging in culture building, you may have a failure on your hands. Think about trustworthiness. That's how you present whether or not you are deserving of somebody to afford you trust. Trust is not automatic. Talk about psychological safety. Talk about where the safe conversations are to be held. Can you have those during your team time? Is it a retrospective? Are there open doors where you can go and tell people what your opinions are, where you believe there are risks, and where you think that there are other benefits that can be brought to the table? All of that nets forward to help you have the type of collaboration you want in order to shift your security.

Sounds logical, right? But how do you do it? Let's start with the culture-building part. Let's start with the toughest piece. The word together matters. Together, define your unifying principles. Together, agree on what the roles and the responsibilities are going to be of teams and of individuals. Define your secure software development framework, your SSDF. Do it together, and work as closely as you can without crowding. Elbow to elbow is a term that doesn't mean you have to be situated in the same geolocation. Elbow to elbow means having that time to engage with one another, to brainstorm with one another, and together demonstrate the model behaviors that you want to see in your organization.

Let's talk a little bit about the secure software development framework. What's important for this is: don't start from scratch. There's no need to if you use industry standards like the NIST Special Publication 800-218. It will tell you what to do, and you as a team will define how you're going to do that. Again, roles and responsibilities. What are the types of security trainings that are necessary? Agree on your software development life cycle. Agree on your secure coding standards. Build and leverage reusable objects to the extent that you can, and verify your security controls.

How about team threat modeling? Yes, this is really something that you can do. The Threat Modeling Manifesto tells you why to threat model. You want, as an organization, to understand what could go wrong in the system. Together, you pinpoint the design and implementation issues, and you want to mitigate them. But who should be threat modeling? Everybody. Anybody concerned about privacy, anybody concerned about safety, anybody concerned about the security of the system. This comes from the Threat Modeling Manifesto.

How would you do this? To make it a team sport, you're going to need to invoke a framework. There are many different threat modeling frameworks, and we're going to keep this at a relatively high level so that you can bring people together to build culture and to build the security mindset. There are many different types of deeper security modeling and different threat modeling that you can do. In specific, we could ask these four basic questions: What are we working on? Which module, which service, which domain area? What could go wrong? What are we going to do about it? And then a retrospective question, an introspective question: did we do a good job?

One of the things that I like to do with my team, and I've had real fun doing this recently, is capturing threat information in plain English by adding it to our user stories. You're probably familiar with the user story madlib: as a blank, I want to blank, so that I can X, Y, Z. How about extending it, adding in a critical asset and a threat? Something that you want protected and who you want it to be protected from. This has worked really well when we've come in and there's already been a certain amount of user stories worked through, so that we could go back to the user stories and extend them with this mindset.

What was interesting as we went through this recently, and I'm doing this with a government organization in the Department of Defense, is that this is a business application as opposed to a weapon system. But it has been bringing people together who didn't sit down and talk through all of these facets at the same time, and now they're starting to understand one another's mindset.

Let me show you an example. Adding in a critical asset and a possible threat. This is just a made-up example, and it came from a conversation that I had with Alyssa Miller. As a car driver, I want to enter in a destination name so that I can navigate without putting in an address. What kind of things do I consider to be critical that I need protected? I want my search history to be protected so that it's not accessed by others.

What's interesting is to see the light bulb go off on different folks, because they're used to thinking about the user story and the decomposition of the user story. They're not necessarily used to thinking about critical assets and the threats that would be there. Oftentimes our delivery teams are builders, as opposed to our security teams, who I often call breakers because they're thinking about how to break in and how to upset the apple cart, so to speak. This is a really great example of something that you can do to build culture, to build your team, and to raise and heighten your awareness of security.

So is it time to code yet? No, not quite yet. Let's talk about security hobbyists. There's more and more responsibility being placed on developers and on delivery teams. Oftentimes shift left is being interpreted as developers are going to do something. There's a constant addition of all sorts of new tools, and the training that we're providing to our delivery teams lacks depth. We continue to pressure our delivery teams to go faster and faster. Oftentimes I think DevOps is becoming hyper-focused on fast delivery, not necessarily being as holistic as we want it to be, and we're asking people to read and to experiment on their own time. I love the quote at the bottom of this: developers are not experts who are waving magic wands to cast spells that defend against the evil hackers in the black hoodies. This is a little bit of a hoot, but it's also a little bit true.

Our security professionals and our developers have different skill sets, and we need to start to cross-train them. We need to cross-pollinate so that we can have the right sort of mindset to help us bring security to the table earlier. #NoHobbyists. We don't want anybody doing this on their own time because they're forced to do it on their own time. Let me talk to you about some techniques for dealing with this.

How do you protect and enable the developers? Training and education; making sure that you are secure by design from the very beginning and it's not bolted on; having security standards and secure coding standards that are agreed to; leveraging design patterns; encouraging experimentation; and doubling down on tools that help to reduce noise and cognitive overload. These are actually kind of difficult at first to get jump-started on because oftentimes we will not have provided enough time and enough budget to address these things, and we assume that the development team will know how to do these things. Let's not make those assumptions.

Upskilling and training the developers oftentimes means that we give them some classroom training, but really your classroom training should be about 10% of the training. The 70/20/10 model tells us that 10% is formal, 20% is social, like what we're doing today. It's also informal events that you may have. It could be brown bag lunches. And 70% needs to be experiential. That means on the job. You're doing this on the job. You're taking the formal training, the informal and social training, and you're bringing it to the job.

Here are a couple of ideas for non-classroom training. Have a hackathon. Use an OWASP project like Juice Shop or WebGoat, but have a team hackathon where you try and figure out or try to beat one another using those projects. Or join a gamified competition. MITRE has something called the Cyber Academy. Join that, but do it as a team so that you're experiencing it together. We can all go home and sign up for Pluralsight, Udemy, or one of the many wonderful individual trainings that are there, but working on something together competitively, doing this and having fun together in a non-classroom setting, really increases our experiences and how much we learn.

How about pair programming in particular? This is one of my most favorite things to do. When I was working with the Commonwealth of Pennsylvania, with one of their largest agencies, we actually pair programmed and used those SAST tools so that we were scanning together, looking at the findings that were there, and talking through what we found. Fantastic results; frustrating at first. You've got to get over that initial embarrassment of having somebody else see that. But as we move towards DevOps, as we move towards test-driven development, as we move towards pair programming and other sorts of transparency, we're going to have to start to get over that we're not perfect, that we will make mistakes, and that we can learn quickly from those mistakes. You can also do something like having a storm-the-castle lunch event, where you take a tool and point it at a module that you have in a non-production environment and go ahead and do some dynamic analysis together.

So is it time to start coding yet? I know every developer, every delivery team here is probably asking that, and I would say almost, but not quite. We need to have good design, secure design. How a system is built is applicable to talk about at all levels, from code to architecture. That means you have to have active decision-making. Architectural decision-making is very important. Oftentimes when we take an emergent architectural perspective, that's relevant for highly innovative initial tranches, but we sometimes do not spend the time up front to get into the details of design that are necessary.

Before you code, make sure you're secure by design, because security is a concern. Security is not a requirement in the backlog. It's not a feature. Explicitly think about security. Understand your domain context, because context matters. What you're doing for weapon systems in defense is very different than when I'm working with medical systems, very different than the work I've done with fintech and with audit organizations that are non-governmental in nature, very different than M&As that I've worked through in the past. Each one of them has unique architectures and unique context that impacts the different security features, the different security aspects. Remember, to not contradict myself: security is the concern, not just an individual bolt-on feature. Focusing on your domain means that some security bugs may be brought forward implicitly because you may be following codified patterns that have already been shared in the public space.

Secure coding practices are absolutely imperative. I think this is a terrifying number: 82% of software vulnerabilities are from coding errors. We're going to see a lot of change on the horizon as we start to apply AI into the actual codification, the development up front, as we start to use low-code and no-code platforms. But it doesn't mean that secure coding standards go away. Identify your secure coding standards as a team. Focusing on that togetherness, make sure that everybody knows the standard and run those SAST tools together. Make sure that they're catching errors. Intentionally seed in the error so that you can catch it and validate that, and have the developers running these tools before code complete.

This brings me to a frustration point. Oftentimes security tools are licensed in a way that they may not be provided to the developers. I ran into this firsthand with an integrated eligibility system for one of the states on the East Coast. They had secured licenses only for their cybersecurity professionals. It was actually a different funding stream and a different organization. I said, I would like to have the developers have access to it. It actually took six months of negotiation to be able to borrow a license so that we could start to do those experiments. We were not just simply handing code or modules over to another group, having them do the analysis, and handing it back.

This is an eye chart. It's meant to show you that there are lots of secure coding standards, and that in general, on the right-hand side, these are the types of things that a secure coding standard is going to put forward: your input validation, your output encoding, authentication and password management, session management. There are tons of things here. The important takeaway is that you need to decide on this as a team, and you need to make sure that everybody is educated about it.

Is it time to code yet? Almost. Make sure that you understand your dependencies. As we saw from one of the earliest slides, the software supply chain is very delicate and it needs to be focused on. Document your packages, understand your program dependencies, and align streams to your secure software development framework. Also don't be afraid to run those composition analysis tools. Run the SCA tools together to understand and eventually get to the point where you're automating your SBOM. Folks may say that using something like JFrog or one of the tools they have right now gives them all of the SBOM and SBOM lineage that they need. I can assure you, in looking at the different standards that are out there and understanding how difficult it can be to truly track your open source and make risk-based decisions on which of your open source libraries or which of the modules that you've consumed are going to have a higher risk ratio so that you address it before you address something that has a lower likelihood of having concerns, that it's really a difficult thing. You can do this if you bring your minds together and put that thought diversity at the table. Right now I'm working with the Army, and they are getting into the meat of SBOM, and they're coming up with some amazing solutions on how they're going to enable, and how they are enabled, and how they have enabled their DevSecOps pipelines.

Learning to break things: you might not have thought that I was going to put this here, but I alluded to it earlier. You have to be able to be both a builder and a breaker. If you are a developer, you need to learn to think about how somebody might break something. The delivery team needs to have that cyber pro mindset. The cyber pro needs to think about how the builders are building things. Together, everyone needs to learn how to break things.

Learn to love application security testing. This may sound funny, but I got to tell you, the more that you're exposed to it and the more you're able to improve what you're churning out, the more motivating and energizing it is. Your earliest detection is from your SAST tools. Go ahead and do that static testing, then move to the dynamic testing that's going to happen before production. Don't wait until production to do your best. There's agent-based gray-box testing that you can do in the runtime, but also that software composition analysis that I mentioned earlier. It never goes away. Learn to love all of these things. Learn to embrace them. Learn to automate them where you can. Learn about the auditability.

What about cognitive load? There is simply too much code being produced for humans to handle alone. Too much code. How can we help delivery teams? We can do it by identifying tools. This is a teeny-tiny sample of the many different tools that are being created, and more are being rolled out on a daily basis. Of course, you can get tool overload and have a certain tool sensitivity, but you should be looking at different types of tools that can help to reduce the noise. SonarQube helps you with your code quality. There is something from WhiteSource, I believe they're now called Mend, which is automated security remediation of your code. It doesn't mean you have to allow every change that's recommended to go directly to production. But imagine a jump-start that gets you 60, 70, 80% of the way there. With AI being applied, with machine learning algorithms being applied into your environment, it just keeps getting better.

Security is everybody's responsibility. Everybody needs to be equipped. Our cyber pros need to be trained. Our developers and delivery team need to be trained. Actually, everybody needs to participate in security. So let's code.

I hope you appreciated today's conversation, and I hope you reach out. Here's my email address both at MITRE and my personal, and you can see my website where my blogs are. Please reach out. I'd be glad to go into detail and give more examples of how to keep from having hobbyists providing for your security. Thank you.