Brittany O'Shea

Hi, everyone. I'm Brittany O'Shea. I am the Director of Security PMM here at GitHub, and I'm very excited to meet you each virtually today.

A little bit about me before we get started, and I'm excited to learn a little bit about you in the chat. I have been in security for about a decade now. The last five years, I have focused on application security. Outside of my day job, I love to travel, and I love to cook. So if you have any good recipes, I'd love to hear that in the Slack channels as well.

I am excited to talk to you today about a topic really near and dear to my heart, because I think it's very important that we all emphasize this in our own organizations or our own software development: supply chain security.

For our journey today, we're going to start with the state of supply chain security: what are the trends driving supply chain attacks, why is the supply chain important to us? Then we're going to get into the types of supply chain attacks, and we're going to use two example attacks to understand how complex these attacks can be and how organizations are susceptible to them. Then we're going to dig into the exciting, fun part: what are the best practices you can do today to start improving your overall security posture in your supply chain, starting with how to secure your accounts, securing your code, and securing your build systems.

So why is the state of supply chain security rapidly evolving? Companies now have to think through one of the largest digital landscapes and secure a wider attack surface, but they also have to think through more complex and advanced attack vectors. In supply chain security, we've seen that in perfect motion. We used to only have to worry about malicious components or insecure components, and now we have to think about how package managers or build systems can also be compromised.

We see in the data that it's really important because we here at GitHub love open source. I'm a huge proponent of open source myself, and we find for good reason. When we look at modern applications - and I'm talking applications developed in the last five-ish years, more or less - 80% to 90% of these applications are made up completely of open source or inner source code.

When we look at how rapidly the open source community continues to grow, in the last report we did a year ago, we found that there were 15,000 new open source packages released daily on GitHub. This is really exciting because these open source packages, this type of code reuse, are allowing us to continue to propel the digital ecosystem that we all benefit from today. It allows us to innovate faster and, really excitingly from a productivity perspective, allows us to increase our productivity, meaning more value to the world, more value to our customers, and more value to ourselves in a more productive and more efficient way.

The studies show, when we look at companies that have enabled an open source culture, shared best practices, and allowed frictionless reuse of code inside companies, we find that those teams increase their productivity, have higher developer retention, have higher learning rates across their organization, and overall have a higher ability to use those benefits and get value to their clients, get value to all of us across the community in a more meaningful way.

That being said, the attack surface has adapted to this. When we look at the types of supply chain attacks we're seeing, next-gen supply chain attacks are up 650%. What is a next-gen supply chain attack? These are things that you're commonly probably hearing in the news: dependency confusion, typosquatting, malicious injection of code, compromising a build system.

It's not these companies in the news that are solely being impacted. About 64% of orgs - more than half of organizations - have been impacted by some form of supply chain attack in the last 12 months, meaning we are all in this together. We are all looking for ways in which we can be more responsive and also better secure our environments to prevent these types of attacks as a community.

When we look at where we are in that journey, there's still a long way for us to go as a community. Forty-five percent of orgs, when surveyed, said they're about halfway finished with securing their supply chain, which is tremendous progress from where we were a few years ago, but still leaves us a lot of interesting initiatives and best practices we can apply to improve our overall security posture.

Before we get into that, let's think about how a supply chain security attack happens. The supply chain can be compromised at multiple points. As you can see by this diagram here, there are two main things you can think about as you're rationalizing or wrapping your head around supply chain security. First is the integrity of your source or your source code, and second is your build integrity.

You could have accidental or malicious insecure code added into your source code repository. You could have that source code repository compromised and have malicious code injected in there. You also could have a compromise of a build platform or through your package repo, and that could allow attackers to either change the code that you thought you were producing or inject malicious code and put that into market or put that into the ecosystem to be used.

Two attacks that you've probably heard a lot about in the news help explain just the different flavors of what happens here. We'll start with SolarWinds, a very complex attack that we're still finding new information about. You'll see in many attacks, when we first hear the news about this compromise, it's really only the tip of the iceberg about what's happening and what the attack pattern really was.

What we know about Orion was that Orion was a network management system for SolarWinds. At some point in time, attackers gained access either through a zero-day vulnerability, social engineering, or brute force. Then they watched and collected information until they injected malicious code into the build process. That code was downloaded later by customers, and therefore they were able to compromise thousands of organizations through this breach vector. We still learn more about it today.

Code42 is a little bit of a different story, but still similar in the supply chain and interconnectedness. It started with a container running in prod that had a bug in it. The attackers there were able to compromise that container and get credentials. With those credentials, they were able to modify the bash script. Through that bash script, they were able to leverage that to gain the customer credentials as well and lead to a further breach of compromise.

You can see, as we get more interconnected in our software supply chains, as I talked about before - the adoption of reused and open source components - and as attackers use different kinds of attack patterns, these one attacks can actually affect all of us. So it's important for us all to prioritize not only securing our own code, but also securing the supply chain overall.

So how do we do that? It really comes down to using all the great lessons that we know about DevOps. Because DevSecOps - I hate that buzzword, but we'll use it - is what DevOps should be from the beginning. It's including security in every micro-practice that you do and building a culture around security in everything you do.

End-to-end supply chain security sounds really big and bulky, but what it really means is looking from your first line of code to your prod code and thinking about how you make sure that the code you distribute hasn't been tampered with, and that you are confident in the integrity of that code and the integrity of the systems being used.

We're going to go over a lot of different things that you can do to improve your security posture, but what I'd recommend is this is your journey. You might have some of these things in place. You might be just starting out. So look at the things that you can apply today that are easiest for you to get started with, and then work your way into some of the more complex items or more complex practices.

Let's start. We're going to go over three core areas: how to secure your personal account, how to secure your code, and how to secure your build processes. I will also point you to a more extensive reading where you can dig in more, as this is just kind of the tip of the iceberg on supply chain security. I'll use a lot of examples that are GitHub-specific, but know that you can replace GitHub with your tooling of choice, and this can be agnostically applied.

Securing your personal account: why does this matter? If someone gets into your personal account, they can obviously do things that you might not like them to do, or that can compromise downstream users of your software or your code. It starts with really good hygiene factors around authentication.

If you're not using two-factor authentication, add it. Two-factor authentication allows you to add an extra layer of security to protect you when you're logging into your apps or your websites, and it performs a different type of authentication that typically is harder to compromise outside of just your username and password.

Also, it's important that many folks are developing or they're using private SSH keys, and we want to think that we're using those securely. What does that mean to use an SSH key securely? It's a good idea, if you're storing this in a disk drive, to make sure it's protected with a passcode. Another great option is also to generate SSH keys on a hardware security key. This completely separates out potential points of compromise on your device and gives that extra layer, extra protection of security.

If you're doing this on GitHub, you can add two-factor authentication to your mobile device using your device itself, and it's something that you can turn on not only at a personal level, but also, if you're an administrator, at an organizational level.

When we get into securing your code, this starts to think about how can you prevent or how can you take an inventory of vulnerabilities that you might have today. I really think about this in three main sections.

First, start with a vulnerability management program for your dependencies. This doesn't matter if you have one repo or you have 1,000 repos. You want to start by understanding what are all the repositories or areas that code is living in. Then you want to establish an inventory of all the dependencies you're using. Are they up to date? Do they have security issues in them? You want to establish a consistent way to get alerted on new vulnerabilities as they're found in your ecosystems.

Once you have those capabilities in place - we at GitHub use Dependabot, but you can also use any kind of third-party providers to do that, or integrate into your tooling of choice - you want to make sure that you have the ability to assess the impact of that vulnerability.

What I often see as a pitfall for folks is they buy or turn on an open source solution, and they get overwhelmed by the number of alerts or overwhelmed by the amount of patches they have to do because maybe that version history or that version you're on has changed substantially from the latest and greatest. That's where prioritization comes through. You can look at things like vulnerability. You can look at things like severity. You can look at things like: is the vulnerable component actually being called by your syscall? These can all help you decide what you should fix first.

In a best-case scenario, you'd love to automate that fix process. Being able to automate those patches and getting into a good behavioral practice of always upgrading not only insecure vulnerabilities, but out-of-date repos.

Next, you want to make sure that you are securing your tokens. Every system needs secrets to run. They need to communicate to another system, and this often happens in a way that isn't vulnerable. But sometimes secrets can be included in your source code accidentally or maliciously. Your secrets can get leaked, and it's important for you to be looking out for that or detecting that.

You can do this through a few ways. At GitHub, if you're doing this on a public repository, we scan for leaked secrets on 80-plus partners to look for those secret patterns that can be released, and we work with those partners to automatically revoke them.

If you're using a different solution or you want to start using this in your private repositories as well, it's great to use a secret scanning tool. Those secret scanning tools, depending on their nature, can help you not only detect leaked secrets, but they can also help you prevent secrets from being leaked in your environment. This becomes really important because it starts making you build the security practices that stop security tech debt from being in your code downstream, or stop security incidents from ever happening from a secret leak.

The most important thing, though, with your secret scanning tool of choice is that you need to have a process and best practices in your environment for how to revoke a secret when it is in fact leaked. Write down that action plan and make sure that action plan is ready to be implemented in your environment, as this can happen quite commonly.

The third thing is you want to start getting a best practice of keeping vulnerable coding patterns out of your repository. This can be really hard to spot using a manual code review. For example, if a function's not memory-safe or if there's an escaping user input that could be leading to a vulnerability, you can check for these patterns automatically using SAST tools, or static application security testing, to look for those types of patterns. This becomes really important in the open source ecosystem because you can look for malicious code that may or may not be injected through a committer.

When you tie all these things together, this becomes really important to put these practices in place because if someone's able to compromise your code, they can also compromise environments downstream.

The last thing we want to focus on is securing your build system. Protecting your build system is really important because you want to prevent folks from tampering with what you're actually building and the end result of your build process.

This starts by making sure that your build system has security best practices in place. There are several capabilities that every build system should have. The first is build steps should be clear and repeatable. It should be consistent across each build. You should know exactly what's running during the build process, and there should be transparency there. Lastly, you should make sure that each build starts in a fresh environment, so if a compromised build happens, it doesn't persist or affect future builds.

One thing that's become more important over time and can be helpful as well is to make sure you're signing your builds. What this really allows you to do is verify that your code hasn't been tampered with and that the code that is being delivered as an end result is ultimately the code that you submitted.

This is often done through either public or private cryptographic key pairing. If you use a private key to sign the build and you publish your public key so users can use your software, it verifies the signatures on the build before they use it. If these builds are tampered with or the code's tampered with, that signing will be removed. This can be really great to also help the ecosystem and help the community verify that what you're putting out there is actually signed off or authorized by you.

Lastly, we come full circle for build security. You want to make sure that your credentials are minimally scoped. You want to limit who can make changes into your workflow, and you want to frequently inspect which users should and can have write access to your repositories. This helps ensure that the right folks can actually access, but that you don't have a leakage or an overage of folks who can make changes on your build system, which makes it difficult to maintain the security across the board.

What are some actions that you can take today if you want to apply these into the environment? One of the things I'm really excited about is that if you want to do any of these things in GitHub on your public repos, you can enable those for free today. If you want to learn more about this or deep dive into some of the technical complexities of how to get started, we have a great end-to-end guide that you can read offline, as that talks about the different ways you can automate some of these practices and talks about how you can also make these unique in your environment based on the type of code you're building.

I'll link to all of these within the Slack channel, but if you have any questions, feel free to drop those in the chat. Thank you so much, and I hope you have a wonderful rest of your day.