Neil Stickels

Hi everyone, I'm Neil, a solutions engineer with Sonar, and I'm here today to talk about how you can ship with zero issues by having clean code.

Working. Oh, there we go. Sorry.

So code is the most important asset that every software company has. Just like a tree, there's two main parts of that software: the visible part that users can see, and that invisible part.

And so the visible part is going to be how it functions, how it looks, how it performs. But the more important part is underneath, just like the root structure of a tree. Is it extensible? Is it reliable? Is it safe? Is it secure? All of these things that users can't see.

And most software companies, they're going to focus on building the right software, making sure that what their users see works exactly how they expect it to. But they can often overlook building the software right.

There's two main issues that plague companies in trying to build the software right: developers who make mistakes with their code, and companies that have policies that sometimes are contradictory or inconsistent. Both of these lead to bad code.

Bad code is plaguing the software industry. Trillions of dollars have been lost due to poor software quality. Even more has been lost to software security leading to data breaches. But in today's session, I'm going to focus on the $85 billion that are spent annually for developer time to fix bad code.

So developers are really the hero of the software organizations today. They're the ones that are actually writing that code. They're innovating new features, new functionality, new products. They're also the ones in charge of making sure that that codebase has quality.

But going back to the $85 billion per year spent on maintenance, the average developer spends 17 hours every week just fixing bad code. What if they could spend a fraction of that time fixing bad code and could actually be using that time to develop new features, new functionality?

And going back to the $85 billion per year, if you have a software organization with a million lines of code, it's estimated that over five years you're spending millions on developers. Not products, not new features.

So how do you do that? Well, with a methodology that we call Clean as You Code.

So I'm going to talk about Clean as You Code, talk about the benefits of it, tell you how you can get started and how you can implement this within your existing workflows, and then finally provide a real-world example of a company that has achieved that clean code status and now releases confidently with zero issues every time.

So before getting into Clean as You Code, though, let me first talk about what is clean code. Clean code is code that's consistent, intentional, adaptable, and responsible. And obviously this applies to your source code, but that also applies to your test code, your infrastructure as code, scripting code. All the code needs to be clean code. It's reliable and it's maintainable.

Now, there's lots of static analysis tools out there that can give you this report and let you know all the places where you have issues within your software. The issue, though, I'm sure that you guys have seen if you've used any of these, is it's usually thousands, if not tens of thousands, of issues that they're giving you. It's years of technical debt to fix all of that.

You don't have years available to just spend fixing existing issues. You want to be writing new software, putting out new features. And if you don't have a plan for how to make sure those are free of issues, well then your quality just gets worse over time.

That's where Clean as You Code comes in.

In any given year, 20% of your codebase is going to be newly written code or code that you've changed. Clean as You Code is built off of this methodology. It focuses on that new code and that changed code and makes sure that that code is free of issues. By doing that, it empowers your developers to be able to make sure that they're delivering quality releases every time.

And the best part: this is done without changing any of the workflows that your developers are using. You can do this using the same IDEs they're using right now. You can run your build using the CI and DevOps platform you're using right now.

So how do you get started? With a simple commitment: no new dirty code shipped to production. Sonar is going to help you with how to do that.

So I'm going to come back to this and go a little bit more in depth, but I just wanted to focus right now on the fact that, as I was saying, this plugs right into your existing workflow. Developers can use the same major IDEs that they're already using. You can use the same CI tools and DevOps platform that you're already using. You don't have to change those to leverage Sonar.

And the benefit of this is, as I was saying, it's non-disruptive. It lets your developers take advantage of this without having to change what they're doing, giving them more time that they can now spend to write new software rather than having to fix old problems. And it's going to help your organization achieve that clean code state.

So now let's go back more into detail on this slide.

There's two main offerings within Sonar. SonarLint, which is an IDE extension that's going to work with all of the popular IDEs out there: Visual Studio, VS Code, the Eclipse family, and the JetBrains family of IDEs.

Developers can use SonarLint to see existing issues within their software, get education when there are issues that are there to know why is this an issue, how do I fix it, and read any kind of extra information they want on it, and be able to instantly identify new issues as they're coding before it ever gets checked in.

For code that is checked in, we have SonarQube, which is a self-managed solution, and SonarCloud, which means it's a SaaS platform, both of which can plug into your DevOps platform and allow you to see issues as your code is being built.

It'll identify any of these issues and run that through what we call a quality gate. This quality gate will look at all of those issues, the types of issues, the severity of those issues, and ultimately decides, should I pass or fail?

And if it does with all of those, then you can leverage that decision to say, should I fail this build? Should I fail this PR merge? Should I fail this push to production?

And one of the advantages if you're doing this is then you're saving all of that processing time down that DevOps pipeline if you do fail early. If you do detect issues in this build, then okay, fail the build. We don't have to worry about the next five steps in our pipeline having to even run.

And obviously then any issues that are identified, well, now they're going to be available to your developers again within their IDE.

So if you wanted to implement this, there's really four steps. First, you need to commit to clean code as an organization. Returning to that mantra I mentioned earlier, you need to commit to no new dirty code ships to production.

Leverage Sonar. You can choose either the self-managed SonarQube or you can use the SaaS version, SonarCloud.

Make sure you set up your projects to take advantage of clean code. You need to define a new code period that's relevant to your projects, and you need to make sure that you're establishing these Clean as You Code guidelines that I've been talking about.

And the fourth step is probably the hardest, and it really goes back to number one, that commitment that you make as an organization. You need to respect that quality gate. If the quality gate says that it fails, then you don't push to production. You need to make sure that it's passing before push. And you need to try to make sure that all of your developers, as many as possible, are using SonarLint to really take advantage of this.

So I mentioned a lot about new code and focusing on code that's new and changed. But more than just knowing that it's new code, you need to define how long it's considered new code. And that's where this new code period comes in.

And it's really up to you as an organization, on a per-project basis, to decide what that means. Does it mean anything that's different than the previous released version? Does it mean anything that's different than some release branch within your SCM? Or maybe you're using Agile and you have rolling time windows, and so if you want to leverage that, you can use any of those within Sonar.

But the key here is making sure that that new code period is long enough that you do have time to fix any of the issues, but short enough to put priority on fixing those issues as they're identified, so they don't just sit there stacking up. And most importantly, it needs to be relevant to your release cycle.

It doesn't make sense to set a three-week new code period if you're releasing every two months. It doesn't make sense to say, "Oh, well, I want my new code period to be three months," but you're releasing monthly. So defining that new code period is critical.

So I've gone through and kind of talked a bit about how you could do this. Now let's actually dig in and talk about different types of companies and how they could leverage this.

So first we'll say just a small software company. They have 10 developers. They're making a web app that runs, and it's built using Java, Kotlin, and JavaScript. And so those developers then are using Eclipse and JetBrains. They build all of this on AWS, and they're using SonarCloud.

So let's actually jump in and see what this would look like for them.

So this is AWS CodePipeline. That's what they're using to be their CI pipeline. Everything is in CodeCommit. They're checking that out, running that through CodeBuild, and oh, CodeBuild is failing.

We go into CodeBuild to go in, and again, it's telling me that it failed. I could go and look at the logs, but instead I'm just going to jump in and show you what the build is doing here, and I'll see why it failed.

So it's pretty simple. In this case, they're just installing Java, they're installing Maven, and then they're using Maven to do their build and to then call Sonar, passing in parameters so that they can connect to SonarCloud and getting that quality gate result. And ultimately, if that quality gate fails, I'm putting in a step here to say, okay, fail the build if that quality gate fails. And that's exactly what happens here. That's why our build fails.

Because if I go and look, sorry, this that I was just showing you, and we can see that it failed because there were bugs that were found, there were vulnerabilities that were found, and because of those, this quality gate failed.

So now I've talked a bit about quality gate. What is that? How do those work? Well, let's actually go in and look at the quality gate here.

So there's three main types of issues that Sonar is going to be identifying: security issues, reliability issues, and maintainability issues. And so we're going to look at, as I said, the number of issues and the severity of those issues and grade each of those. And by default, we want to make sure all of those are graded as an A.

There's one other type of issue that we identify called hotspots. Hotspots are security issues, but they're a little bit more nuanced and contextual. And you really need to go in and see, is this an issue for us in this circumstance? And review that. So as part of the quality gate as well, you need to make sure that 100% of your hotspots have been reviewed.

Then you also want to make sure your code is tested. So you need to make sure at least 80% of your code has been tested.

And finally, it's checking for duplicate code blocks, making sure that there aren't large areas that need refactoring. And so in this case, it makes sure that less than 3% of your code is duplicate code blocks.

Now this is the default quality gate that every installation of Sonar comes with, and we call it the Sonar way because it's Clean as You Code compliant. Every time you run a build, it's going to evaluate all of these criteria. If all of them pass, great, your quality gate passes. If any of them fail, it's going to fail that quality gate.

Also, let's jump in here and look at Eclipse real quick. Show you SonarLint.

So with SonarLint, a developer is going to be able to see the issues with their software. It's going to identify things like unused that the IDE is already identifying.

But here's a case, there's a hard-coded password. SonarLint is identifying that, saying, "Hey, you shouldn't be using hard-coded passwords." Developer could then click on this.

Sorry, my internet connection doesn't look like it's working here. This is a link to that rule, if this works. Okay, well that should be providing information on the rule itself. Tell them why this is the case. Show them examples of what bad code looks like. Show them examples of good code. Also give them links to third-party sites they could look at to try to fix this.

It'll do that for any issue that SonarLint has identified to provide that education for developers.

The real power in SonarLint, though, comes when it's connected in what we call connected mode, connecting it to a running instance of SonarQube or SonarCloud. That way all the projects that you have within your IDE are synchronized to projects that you have within Sonar.

And by doing this, it'll synchronize all of the issues that are found by other developers when they're pushing. But also it'll synchronize that rule set that you're using to make sure all developers are running this using the same rules that you've established within Sonar.

It also lets you take advantage of some of the commercial features, assuming you're using a commercial version of SonarQube or SonarCloud. It lets you take advantage of some of the commercial features. One of those is one that we call taint analysis.

What taint analysis will do is actually show you how a vulnerability propagates through your code.

And so in this case, sorry I can't blow up this text, so I hope you guys can read this. I know it's small. But it's showing you how this issue is propagating through the code.

So I started in the DBUtil class, and this is showing me, oh, well actually this all starts with an HTTP GET request where I'm reading in a parameter, saving that, passing that into DBUtil. And if I go here, okay, yep, reading that in, saving that variable, saving this into a select statement with that, then ultimately calling that select statement here.

But this is a SQL injection vulnerability. We're making this so someone can pass in something. They could be able to pass in something that could do some damaging stuff to our database.

So we're showing all of this to the developer to let them be able to understand what's going on. And yes, maybe they could just fix it with a prepared statement here instead of using a statement. Or maybe by seeing this and seeing how it's propagating through, they decide, you know what, I think this would actually be better fixed here. Or I actually see this request coming in through the HTTP request and throw an error to the user if they're passing in something that could cause this SQL injection attack.

So SonarLint will provide this functionality to them to be able to see all of this.

Let's talk about now a bigger company that's using .NET. They're using C#, VB.NET, maybe some Python too. And obviously then they're using Visual Studio and VS Code, and they're using Azure DevOps for their CI pipeline.

Well, let's talk about another feature of Sonar with all of these DevOps platforms: the ability to comment directly on the PR.

And it's going to look a little bit different for every DevOps platform that you're using just based on what their APIs allow us to see. But in this case, we can see for Azure DevOps, with this PR, when they ran it, we're telling them, "Hey, your Sonar quality gate failed." You want to be able to click on this. This will give you more information, and you can actually see exactly why it failed, because your maintainability rating was a B and there's a hotspot that hasn't been reviewed.

If we look over here, we can see that, yep, there is an issue here. Yes, there is a hotspot that needs to be reviewed. So it lets the developer see that within the DevOps platform, not even having to open up SonarQube.

I showed an earlier example of Maven. Here's an example as well where you can do this with MSBuild. So it's just adding a couple steps to your Azure pipeline: one to prepare that connection to SonarQube, and run your build just as you normally would. And then you just need to call this analyze step, which will actually look for those issues. And finally the publish step, which will push all of these results to Sonar to make them available there.

Finally, let's talk about one more big, huge, massive software company where they're using any programming language you can think of, infrastructure as code, mainframe applications, everything they're doing. And well, we can use all of the IDEs that they're going to be using. And maybe they've had dev teams that have homegrown their own DevOps solutions, and one team is using GitLab, and another team is using GitHub, and another team is using Jenkins, and another team is using Bitbucket, and whatever.

Great. Sonar can connect to all those, let you view all of those results in one place.

And so I've mentioned a few other DevOps platforms. We'll go in here and talk about GitHub. Now again, we can do this pull request here. In this case, we actually show you why this failed. In this case, because the security rating wasn't A. Again, there's a link to SonarQube here, so they could go in and see that within SonarQube itself.

Another thing, though, that I wanted to point out is that with GitHub, we actually can integrate with GitHub Advanced Security. And so vulnerabilities can show up in GitHub Advanced Security as your code scanning alerts. You can see this was detected by SonarQube, can open this up. This actually shows us exactly where in the code it is, tells us the rule.

If I actually open this, this would actually show me all this information that we should have seen on SonarLint earlier, but that wasn't working. Apologies there. But right, exactly telling you why is this an issue? What does bad code look like? What does good code look like? And provide links for other sites that you can go in to read more about this particular issue.

So I've given some examples of theoretical companies that could be using this. But now let's actually talk about a real company that really has achieved this.

So it's a German software provider called DATEV. They provide custom software to basically any industry across Germany. And they have hundreds of products now that they have developed for all of these different industries that they're working for.

They've been a Sonar customer since 2016. They have over 38 million lines of code that they're scanning with Sonar. And as a company, they've completely embraced this Clean as You Code methodology, but it hasn't always been that way.

It started back in 2018, where they heard about us talking about this, and they're like, alright, let's look into this a little bit more. And so they trialed it with one of their dev teams that was working on taking a COBOL mainframe application and making it a cloud-based application.

And the dev manager was a little bit hesitant because he's like, well, I don't know if my developers are going to like it if we're telling you, hey, you've got dirty code here. But to his surprise, they all took it as a challenge, and they were all like, I don't want to be the one to break the build. I want to make sure that I'm delivering good code.

And that project was released successfully.

So they pushed this out to everyone, and they kind of pulled back from those quality gate standards that I mentioned earlier, and they developed what they called the DATEV Quality Gate, and they pushed that out to every dev team.

Well, all of these teams then started taking this on and all started trying to make sure that they could pass that. But then they realized, well, this actually didn't meet those Clean as You Code standards. We want to make sure we're doing that. So they made it a little bit more rigid, made it actually follow that. And they created the DATEV Gold Gate.

Well, then they started saying, well, you know what? Yeah, our new code is great, but this actually isn't making sure that our old code is meeting standards. We want to make sure that there's no blocker-level vulnerability within the old code either.

So they created the DATEV Platinum Gate, which will add that. All the dev teams started passing that then. And they said, no, you know what? We want this more. We don't want any issues in any project.

And they created what they called the DATEV Insane Gate. And it became a race between all of their different dev teams to see who could pass that DATEV Insane Gate first.

Now all the teams are, and obviously this didn't happen overnight. It took them four years to be able to achieve this. But now every team releases every project with zero issues.

So if DATEV could do this, why can't you?

So I'd like to welcome you.