Host Intro (Gene Kim)

All right. The next speaker is Dr. Stephen Magill, who is now VP of Product Innovation at Sonatype. So he and I got to work together a couple years ago when we worked together on the State of the Software Supply Chain report, where we got access to all the amazing data inside of Sonatype Maven Central to understand the update behaviors for the heart of the Java open source ecosystem.

So he'll be exploring the connection between classic compiler attacks, the astounding XZ attack, and the emerging AI threat landscape, including training data poisoning, and how the InfoSec community needs to step up to ensure adequate knowledge — not just of their software, but now their data, model identity, and provenance.

Here's Stephen.

Stephen Magill

Hi. Thank you, Gene.

Yeah, so I am here today talking about something that I am calling Attacks from Wonderland. And these are attacks that are really interesting and unique in their operation, their execution, and their impact.

The first one of those that we'll talk about, as Gene mentioned, is XZ. So what is XZ? It's an open source project that implements a compression algorithm called LZMA. It's the algorithm used in XZ files. So if you've ever downloaded like a .tar.xz archive, that's the algorithm you used — you used this open source project. It's also the compression algorithm used in Ubuntu, Debian, and Fedora packages. So it's really pervasively used across the Linux ecosystem.

And it's also an indirect dependency of OpenSSH. And so what do I mean by indirect? It is not a declared dependency. If you go look at the OpenSSH source code, it doesn't depend on liblzma. However, OpenSSH as packaged by Ubuntu, Debian, and Fedora depends on liblzma in a sort of indirect route. They've added systemd support to OpenSSH, and that pulls in this dependency. So if anyone ever tells you it's really simple to figure out what the dependencies of an open source project are — like, 'just go look at the repo and you can tell' — this is an example of: it's not that easy, right?

Not gonna talk about that, though. What I'm gonna talk about is this attack that was launched against the XZ project and could have, if it were fully successful, compromised most IT infrastructure on the planet. And what do I mean by that? Well, if it had been fully successful, what it would have meant is that Ubuntu, Debian, and Fedora systems running OpenSSH would have been vulnerable to this attack that would let this attacker bypass authentication and get access. There's also indications that included remote code execution support. So they would have had arbitrary code execution on a huge section of the servers in the world.

It didn't get that far. It got as far as Debian testing, which is pretty incredible — it's like one step away from stable. If it had made it into stable and had been there for any length of time, the consequences would have been much worse. Even as it was, making it into testing meant a lot of servers had to be rebuilt, infrastructure had to be created from scratch, because it was assumed compromised. If you were running this version of testing, you were sort of assumed to be compromised by this attack.

So how was it discovered? It was discovered about five months ago — March 29th, 2024. Andres Freund, who's an engineer at Microsoft and also a developer and maintainer for Postgres, discovered something was weird when he was doing some micro-benchmarking. He was trying to set up this system to do fine-grained timing analysis, trying to get the system to behave in an expected way, be quiescent, have low CPU utilization so he could measure things. And he noticed some anomalies in the CPU usage. In particular, SSH was taking up more time than he expected.

So he did some digging, and there's a number of coincidences that sort of all aligned to help him discover it and notify the community — and then us to all learn that this could have gone very differently. So what are the coincidences? Well, he happened to be running Debian testing. If he'd been doing this experiment on stable, he wouldn't have seen these same effects. He was doing performance analysis, so he noticed these small changes in CPU load. He remembers this random valgrind complaint from Postgres that he encountered on the open source project he contributes to. That all sort of tipped him off, made him skeptical, and made him dig into this. And then he had the knowledge and skills to dig into it and get to the bottom of it. Someone else discovering this might have just submitted a bug report saying, 'Hey, the new version of SSH is a little slower than it used to be.' And maybe someone would have looked into that right away — probably not. Maybe they would have looked into it eventually, but we wouldn't have gotten the exploit report that we did, basically right away, thanks to Andres's work.

And it's a complicated exploit. So how does it work? The liblzma repo includes some binary test files. This is normal. This is expected. It's a compression algorithm — you need test cases. Compressed files are binary files, and so of course there's gonna be binary files in the repo: both valid compressed files and invalid ones, because you've gotta test the failure cases too. In particular, there's two test files in there — `bad-3-corrupt_lzma2.xz` and `good-large_compressed.lzma` — that are involved in this exploit.

In the vulnerable versions of the code, these have been modified, so that when you package the library for distribution, a build script — which is not part of the repo but is rather part of what's distributed to maintainers (so Ubuntu and Fedora and so forth) for packaging up the packaged version of liblzma — this build script extracts some bytes from the binary file and writes them into a bash script. That bash script gets executed. It pulls in more information from the other binary test file, adds that to an object file: `liblzma_la-crc64_fast.o`. This is expected — this is a normal object file that exists, gets generated by the build process in previous versions, vulnerable versions, non-vulnerable versions. It's always there. But in the vulnerable versions, it's been modified so that when it's loaded, it gets triggered by this `crc64_resolve` function.

That triggers the vulnerable code. `crc64_resolve` — basically everything that's happening at load time is involved in selecting the most efficient CRC algorithm. This is a compression algorithm, it's on the critical path for a lot of things — performance is super important. And so certain critical functions, there are optimized versions for various architectures. And of course at load time you want to see what architecture am I on, let me pick the most efficient version of this function and let me use that. So there's this dynamic runtime selection of which function to run — hooks into that logic to load the exploit code. And it uses the same logic actually to trigger an exploit of OpenSSH. So if it notices that it's in an environment where SSH is running in its address space, it hooks into this `RSA_public_decrypt` function in OpenSSH, and that's what lets it bypass authorization. So now it's at the point where, if it's in this situation, it's exploited OpenSSH, and now the attacker could — given their access — get into that system.

What I want to point out is: what's in these green boxes is totally normal. It's normal to have binary files in this repo. It's normal to have this runtime loading going on, this loading that's dependent on your environment. Those are all normal. The only thing that's strange is what's happening during this build and package process. And there's just a couple of lines that have been added to some build scripts. If you've ever built an RPM from scratch, you know it's a whole process. The automake, autoconf, make commands, then the scripts run, and it's a whole mess of stuff that usually happens. It's not surprising that this sort of flew under the radar.

So that's this crazy attack. And, you know, who are the geniuses that think of these ways to subtly hide things in binaries and extract them in a way that people won't notice? Well, I want to go back to another genius who had similar ideas quite a while ago.

So 20 years ago, there was a paper — a seminal paper in computer science — called 'Reflections on Trusting Trust,' published by Ken Thompson. I actually, when I learned about the XZ attack, I remembered this paper and went back and looked. I had actually forgotten — it's not a paper he published as part of his normal work or whatever. It was his Turing Award acceptance speech. So how amazing are you when the speech you give as you're accepting your Turing Award is also a seminal work in computer science? It gives me hope that we have people just as smart on the defensive side as offensive.

Ken won the Turing Award for creating Unix with his collaborator Dennis Ritchie. And then in his acceptance speech he describes a very similar sort of attack. He's a compilers guy. He wrote B, the predecessor to the C programming language. He's also one of the developers of Go. And so he describes an attack on a compiler that's very similar in its subtlety to how XZ works.

First, let me describe how compilers work. If you've played with compilers at all, you might know it's very common for the compiler for a language to be written in that language itself. So the C compiler is written in C, the Java compiler's written in Java, the Rust compiler's written in Rust. But how does that work, right? When Rust was first defined, there was no Rust compiler. So what are you gonna do with Rust source code? There's nothing to do with it. And the answer is: well, the first version is not actually written in Rust. The first version of the Rust compiler was actually written in OCaml — which is one of my favorite programming languages, alongside Haskell and Clojure, and Gene, we can discuss later which is better, over drinks or whatever. But that first version used an existing language, and then it was only two years later that the Rust compiler was rewritten in Rust. And that's called becoming self-hosting. It's a milestone for a programming language to reach that point where your compiler is now self-hosting — it's written in its own language.

But to get there, you have to accept this binary into your trusted base. There has to be some initial binary, some stage-zero of the compilation process that takes that Rust code and generates a binary. And then you can use that binary as the next version. And that's how compilers progress. You're always generating new binaries based on trusting the old binary.

And so what Thompson realized is: you can hide a backdoor in this binary. Nobody can look into it and notice that there's something there. So if you can hide some code in there that inserts a backdoor and propagates itself, you can really hook deeply into this process in a very subtle and non-discoverable way.

What does that look like? You insert a compiler backdoor into the compiler that looks at the input, looks at the program it's compiling and says: does this program look like the compiler? Am I compiling a version of the compiler? If so, I'm gonna insert the backdoor code into the output; otherwise I'll do the normal thing. And in this way, it continues to propagate, it continues to be part of that trusted binary base.

And then of course you need some action — if it's gonna be an exploit, you need some effect. And so he picked the login program — when you sit down in terminal, type your username and password, the program that grants you access to a command prompt — and just added an auth bypass to that. Very similarly to how XZ bypasses SSH auth.

So there's a number of parallels here. In Thompson's 'Trusting Trust' case, the backdoor gets inserted during compilation; for XZ, it's during build and package. They're both exploiting the fact that binaries have to be trusted and can't really be examined the same way as other files. The effect is different, but that's sort of immaterial. In Ken's case, it operates by modifying another program during compilation; XZ actually gets its exploit loaded or triggered by the loader — by this loading process I mentioned. It hooks into the dynamic function selection process.

Thompson actually thought about that. He comments that in demonstrating this attack, 'I picked on the C compiler, but I could have picked on any program-handling program such as an assembler, a loader, or even microcode.' I think that's interesting — targeting program-handling programs — because we've heard about some other kinds of program-handling programs in this session, right? AI code gen.

So you could imagine a similar sort of attack where the backdoor is now inserted during training rather than compilation. It hides in the binary model or the model weights, which are just as inscrutable as binary executables. It can introduce incorrect code the same way the compiler backdoor can mis-compile things — except it's one step earlier in the process. Instead of mis-compiling source code, it's mis-generating the source code that will eventually get compiled. Modifying those code suggestions. And it doesn't even have to be code gen. A general AI system — anything that's making decisions — is subject to this sort of attack.

And this is not new. These are called model poisoning attacks or data poisoning attacks, and people have described them and speculated about them. I think what's interesting to me is that we see the possibility of the same sort of malicious supply chain attack patterns being discussed in the AI space now that have been demonstrated in the traditional software space with the XZ attack — and that were actually theorized 20 years ago by Ken Thompson in that work. So just the provenance there is really interesting to me.

Another thing I haven't talked about yet that I want to be sure to mention is the really interesting timeline of this attack. So how did that vulnerable code get into the XZ library? How did those binaries get put there? Well, they were put there by a maintainer. Not the original maintainer, not the person who started the project and had supported it for years and years and years, but a relatively new maintainer that basically started working in the project in October 2021. A year later, he becomes a maintainer. A year later, he inserts the backdoor code. So a really slow burn — like, two years from first commit to finally committing the backdoor.

In retrospect it's realized that this is a persona. There's probably some larger attacker behind him — maybe nation-state, maybe large hacking group. There's speculation on who it was behind this. But you can see the planning and execution that's just really impressive that went into this.

And it wasn't just this Jia Tan persona. There were a number of other personas — sort of sock puppet accounts — that were piling on, basically saying, 'You need to speed up your process here. It takes years to accept patches from this list. It's been over one month, and no closer to being merged. Not a surprise.' Guilt, guilt, guilt — they're layering on. The poor maintainer of XZ is really getting beat up on by these fake accounts. And then one of them even starts suggesting, 'Hey, Jia Tan has made contributions recently. Why can't he just commit these patches directly?' And so eventually he does become a maintainer, and that enables this.

What's interesting is there's a parallel event that happens around the same time that got a lot less press, because it wasn't successful the way that the XZ attack was — or wasn't as close to being a full success. And that happened with the OpenJS Foundation project. So OpenJS is an open source foundation, sort of like Linux Foundation or Apache Foundation. They support the development of Node.js, jQuery, lodash — a number of really critical JavaScript libraries.

Around the time that the community was learning about what was happening with XZ, they noticed that some of their projects had been subject to similar pressure campaigns — emails were coming in saying, 'Hey, you guys aren't being responsive enough to issues. You need more maintainers. Why don't you promote this person to be a maintainer?' Those sorts of suggestions and pressure.

The difference is, these are larger projects in terms of their contribution base. These are projects with more maintainers, more resources. They're being supported by this foundation. XZ was a single-maintainer project. This guy was supporting it on his own as a side project, in addition to probably a million other things going on in his life. So of course he's being maybe less responsive — it's harder to address community needs, and he is more susceptible to this sort of pressure.

In the OpenJS case, they sort of are suspicious rather than just accepting more maintainers. They're suspicious about this, they look into it, realize what's going on — and the attack is not successful in this case. So I think that's a nice counterpoint.

And it gets us to: what can we do about these sorts of attacks? How do we prevent this? And maybe the answer is: you can't. Given how close this came to success and how surprising and coincidental it was that the community even noticed, it's probably not hard to imagine that there might have been similar attacks that succeeded. I don't know. But certainly thinking of your infrastructure as potentially compromised and adopting things like zero trust, and doing what you can to lock down your core services, is certainly advisable. I think even more so given what we learned about this.

But there's also things we can do to increase our chances of detecting attacks like this in the future. And I think one is increasing transparency. There's a number of things under this category. I want to know which repos involve binaries — that's, we've learned, a great place to hide these things. As in binary data in repositories. Which of those involve binaries, and which should I attribute more potential risk to? Which projects are well-funded? Because we saw with the better-funded projects in OpenJS, they noticed this — so maybe that's a factor. Which version am I using, and what are the specific hashes of the software I'm using? Because one aspect of the XZ attack is: whether it hooked into OpenSSH, whether it could execute the exploit, depended on the environment. Some distributions, OpenSSH was vulnerable; other distributions it wouldn't have been. So you can have the same version, but it's compiled differently, it's packaged differently, and so the vulnerability is different.

On the AI side: what models am I using? How are they derived? There's technologies that can help here. There's metadata that can be pulled in on whether binaries are included. There's things like SBOMs — software bills of materials — that let you write down what's in a piece of software. You can write hashes as part of your SBOMs; that's supported by the format. So you can get that sort of transparency and record-keeping in place. And then there's discussion around how to use AIBOMs — what would an AIBOM look like, and how can we use that to capture similar sorts of transparency for AI models, which could help with these sorts of attacks as well.

At Sonatype, we're looking at how to bring this information in, make it more accessible, make it more actionable, for people developing software. I'm involved in a lot of these community efforts in this space. And so if anyone's interested in talking about anything I've talked about in this talk, or about what we might do in the future to help with this, I'm happy to have a discussion. I'll be at the Sonatype booth after this. Thank you.