Klaus Baumgartner

Hello, and a warm welcome to the next session at the conference. In this session we would like to present an approach we call continuous quality assurance: what it is, how we implemented it, and how we integrated it into our organization.

My name is Klaus Baumgartner. I am the R&D manager for Digital Buildings at Siemens Smart Infrastructure Building Products. Today I am accompanied by my Siemens colleague Peter.

Dr. Peter Fassbinder

A warm welcome from my side as well. My name is Peter Fassbinder. I am a principal expert for PLM process innovation with Siemens Advanta Consulting. I have a long history in the lean-agile domain, and for more than five years I have dedicated my time fully to continuous delivery and DevOps, especially to the question of how those approaches can be applied in a company like ours.

As most of you probably know, Siemens is a very large company with close to 300,000 employees. We have a history of more than 170 years and come from a heritage of physical, technological, and electronic products. But digitalization, IoT, and the increasing importance of software are spreading across our portfolio and business. That is why Klaus and I are here talking about DevOps.

The slide shows the current Siemens setup with its operating businesses. A relatively new addition is Siemens Advanta, where I am located. The mission of Siemens Advanta is to look into digital topics and unlock the digital future of our clients. That is what we have been doing with Klaus and Digital Buildings, driving forward the topic we want to explain today. I will hand back to Klaus.

Klaus Baumgartner

Before we go into the subject, a few words about my organization. Building Products, or Digital Buildings, has been a pioneer in the building industry and provides a full set of products and services around buildings. With this offering, we are basically turning a building into a smart system.

If you look at a building, you can see several disciplines such as security, safety, power, lighting, and HVAC. Historically these were purely on-premise and predominantly hardware-based systems. Today they are increasingly connected to the cloud. A user wants to manage, operate, and monitor those systems, perhaps with a mobile phone, or manage the system through a cloud-based web service.

With this development we had to change not only the technology. We were also forced to change the way we were working, because we came from a process world with projects, development times, and release cycles of many months, up to a year or even longer. We had to move to a continuous way of delivering technology to leverage the possibilities of digitalization.

I am probably preaching to the converted, because continuous delivery and DevOps should be familiar to you. But if we look at an industrial environment, there are also adjacent regulatory and quality assurance aspects that must be fulfilled even in continuous delivery. The approach for embedding and integrating those aspects into continuous delivery is what we call continuous quality assurance. Peter will now go into more detail about what it is and how we implemented it in our continuous delivery pipeline.

Dr. Peter Fassbinder

Continuous quality assurance is our solution for driving this topic forward in a continuous world, also for industrial companies. From our point of view, it is a very important topic and one of the crucial success factors and enablers for bringing continuous DevOps to life in a company like ours.

In the next slides I will first give an overview of the core idea behind the continuous quality approach we have developed over the past couple of years and applied in a couple of Siemens businesses. Then I will talk about how to manage it efficiently in practice.

The core idea has to do with the mantra green to green. The bottom line of the picture, labeled continuous integration, should be familiar. Moving from project work toward a continuous value stream with CI/CD pipelines implies a different way of working. The uppermost priority is to always have an up-and-running system, a green system. A highly automated continuous integration tool chain is how you ensure that.

But if you want to move toward the more continuous value stream Klaus described, the continuous integration tool chain is not sufficient. Even with perfectly green working software changes, we cannot simply deploy them into production. There are many additional things we need to provide, which I call non-software artifacts, before we can release or deploy a product. This applies equally to deploying a very small change.

These artifacts include end-user documentation, service documentation, installation guidelines, regulated and legal documents such as export control and open source clearing, and internal topics such as risk assessment and patent applications. A huge variety of topics must be ready and available in a defined state together with the product and the software before we can deploy. That applies to a large one-year project, and it equally applies to a small change after a sprint.

In the past, when we had projects, this was handled by the familiar milestone quality-gate approach. But if you want very frequent deployments, you cannot run the whole milestone and quality-gate approach anymore. We had to think about how to replace it with an adequate yet continuous approach that ensures the same quality. That is what continuous quality assurance is about.

The core idea is to apply the green-to-green mantra from continuous integration and the software side also to non-software artifacts. We call this continuous conformance. It implies two things. First, if we have a system up and running in production, all of these non-software artifacts are already in a valid state. Second, when we want to implement a small change, we ask whether that change requires an update of any non-software artifact. If the answer is yes, we must make sure we update the artifact before deployment. We have evolved this iteratively and brought it more and more into practice.

At a high level, what we want to achieve is continuous, frequent deployment of software updates into existing products. We need two streams for that. We need a continuous integration stream that ensures the quality of the software. Existing off-the-shelf tool chains can implement this. But that is not sufficient. In addition, we need a continuous conformance stream that ensures the quality of the non-software artifacts. Only if both are implemented, and both are equally important, are we in a position to do frequent continuous deployment.

When we started implementing this approach, we found that quite a number of the non-software artifact criteria that were part of milestone checklists in the past are not relevant for individual deployment decisions. We lifted those to a higher level, shown in blue on the diagram. Examples are marketing or organizational topics. We still have to take care of them, but we do not have to integrate them into the high-frequency stream. That makes the yellow continuous conformance stream easier because it has fewer artifacts to handle.

The next learning was that it is not only the green-to-green mantra that must be applied to these non-software artifacts. Another familiar mantra from the software world is needed: shift left. Originally, we thought that in a two-week sprint we could think at the beginning about what non-software artifacts we needed, and have them at the end. That does not work for two reasons.

First, some non-software artifacts take a long time to update, so a two-week sprint is not sufficient. If the artifact is not updated by the end, you cannot deploy. Second, many non-software artifacts are required as inputs for development. If a change needs one of those artifacts updated, that update must be ready before development starts. So we need to shift this work left.

That does not mean huge upfront planning. It means a continuous rolling process for non-software artifacts, one, two, or three sprints ahead, so we always have enough non-software topics prepared and enough changes ready in the proper state to start developing.

When we look deeper at continuous conformance, implementation becomes more complicated because these non-software artifacts are heterogeneous. We identified four types depending on when the analysis can be done and when the update must be completed. Type one includes patent applications. If you have a change where you need to rerun a patent application before implementing it, that takes a long time and must happen upfront.

Type two is the most common in our typical scenarios: artifacts that are input for development. If an update is required, it must be completed before development starts. Examples are UX design, architecture updates, and risk assessment.

Type three includes artifacts such as user documentation, where the update can only happen in parallel with or after development because it needs an output from development. Type four includes topics such as open source clearing, where only during development do you identify that something needs to be updated, so analysis can happen only during the sprint.

This makes the process more complicated, but the classification is essential. We need a clear understanding of when we can analyze each non-software artifact and when its update needs to be available, so that continuous integration and software development run smoothly like a conveyor belt in a factory.

We introduced a ready-to-develop entry gate for each change. In addition to the normal definition of ready, such as test acceptance criteria and functional specification, we also need to have completed the non-software items required for this specific change. This is stronger and stricter than a typical definition of ready. It is crucial because by the end of the sprint we want to be ready to deploy. From the non-software conformance, compliance, and legal aspects, everything must already be prepared, ready, and ticked off. When the software is ready, we can deploy because we have taken care of the rest as well.

Then we had to ask whether this is manageable in practice. An early analysis gave us confidence that it is. The picture shows two teams with real data. Team A had eight features to develop in the next three months. Team B had 16 features. We had roughly 20 continuous conformance criteria or non-software artifacts. We analyzed, based on each feature or change, which criteria needed an update. Typically only around 10% to 20% of all possible updates were required. That means in 80% to 90% of cases the existing non-software artifact was still valid and did not need an update. But we cannot ignore it. We must always do the analysis, and where the analysis says yes, the update becomes a prerequisite for deployment.

It also became clear that we had many individual micro-items and atomic items to manage efficiently. We wanted to do that in a tool, and in the existing tools where we already have requirements, features, user stories, test cases, and continuous integration information.

In the projects we have done so far, we used the existing requirements breakdown and content-based user stories in the ALM tools, and integrated continuous conformance at the same level. Because the organizations used different ALM tools, we implemented this in different tools. So far, we have implemented it successfully in IBM Rational Team Concert, Jira, and Azure DevOps.

The tool structure follows the same principle: analysis, then update and tracking. In analysis, we use a predefined checklist with all criteria identified for the organization. When defining new features, in addition to functional user stories and content user stories, you go through the checklist, tick the ones that need to be updated because of the feature, and only then does the tool create a task and attach it to the feature. Then you track and update it.

We created dashboards in all of the tools. The dashboards look different, but the message is the same. Previously identified updates create tasks in the tool, and those tasks must be completed and set to done for the feature. Only then can you implement the feature. So we have two dashboards, or two traffic lights, for every change or feature. One addresses software, testing, and the continuous integration pipeline. The other addresses continuous conformance. Only if both traffic lights for a specific change are green are we in a position to deploy. This is a clear, transparent, and efficient way, and we have successfully applied it in several organizations.

I will hand back to Klaus to give an idea of the key success factors for a sustainable implementation of this continuous quality assurance concept.

Klaus Baumgartner

Thank you, Peter. We started by setting the clear vision that we wanted to move into continuous delivery mode with all aspects in the same quality as before. From the beginning we implemented change management so we would be ready for coaching and rollout afterward.

Then we had to define the new deployment strategy. The most important aspect was identifying and analyzing the continuous conformance criteria: where to put them in the process and how to be ready.

As Peter showed, these criteria and their visualization became part of the tool and were integrated into the tool chain. You have one dashboard where you can see not only the progress and status of software artifacts, but also the non-software artifacts, the continuous conformance aspects, and the continuous integration aspects.

We did not talk about it in this session, but the roles, responsibilities, and steps also need to be integrated into your ALM process. Ours is based on the SAFe Agile framework, and we mapped all of these steps into the process.

We have been doing this for quite some time, and it helped us a lot because releases became no longer a surprise. By the time software implementation was finished, all adjacent artifacts were also ready, and deployment could be done with the appropriate quality aspect as before in the project-driven approach we had used for many years.

With that, we are at the end of the session. Thank you for your attention. If you want to know more or have additional questions, feel free to reach out. Our contact details are included in the presentation.

Dr. Peter Fassbinder

Thank you very much from my side for listening. I hope we can get in touch and exchange on what you think about this, and also have further discussions over the course of the conference. We have attached a white paper on continuous delivery and DevOps in industrial environments, something I wrote some time ago. It is a high-level summary not only of the topic we just presented but of the overall topic of continuous delivery and DevOps in industrial environments, including similar other aspects that need to be addressed. Feel free to download it and get in touch if you would like to know more. Thank you very much, and enjoy the rest of the conference.