Rosalind Radcliffe

My name is Rosalind Radcliffe, and I am an IBM Distinguished Engineer. In my real job, I am responsible for making z/OS DevOps-able, as I put it, to help you have our capabilities so that you can do DevOps with your existing mainframe systems, helping improve our capabilities and our products.

But really, I spend most of my time with clients, helping them understand how you can do DevOps with a system.

At IBM, and I've told this story for a long time, we've transformed, and we do DevOps to build our systems. If I use CICS as an example, they are the best story we have that talks about DevOps transformation and their ability to deliver quality code to customers every day, which I know you don't want, so don't worry, we're not going to actually ship it every day. But we could.

In this discussion, I want to talk about the different things we've been doing to make this easier.

But I want to start with some fun facts, because it's always fun to think about this system we call the mainframe. These are some statistics: number of transactions a second. If you look at the mainframe, and you look at two of the transaction processors on the mainframe, just two of them, just CICS and IMS, I'm leaving out all of TPF, and I'm leaving out all of the airlines and a whole bunch of other things. It's only 4 billion. So the perspective is a little different.

It runs the mainframe. z/OS runs many of the systems we have today. But if you think about it and you think about z/OS, is this what you think about? A green screen that looked the same as it did maybe when I started in development.

And to be honest, I started in ISPF development on the mainframe 31 years ago. So this is what development is, right? This is what z/OS is.

No, this is what z/OS is.

z/OS has been transforming for a long time. If you look at the Z hardware itself, it's the most modern hardware that we have in the environment. But the way we treat it is as a system that was built in the 1960s or '70s with waterfall development practices, and we haven't necessarily transformed the way we do work.

Within IBM, we've been working to help remove those barriers, help make the changes so that when you think about z/OS, you don't think about it as different, but you think about it as just another platform.

So we continue to evolve. Now, the first picture I had to put on there because it was fun. It's the latest Z machine, the z14 single frame. For those of you who don't know a lot about the mainframe, the mainframe always sits in the corner of your data center in a different place because it was not built to sit in the same rack size as any of the other systems, and it wasn't built in the same hot and cooling aisle, and it wasn't a lot of other things.

With the z14 this year, it is now designed in the exact same configuration, standard rack size. You can drop it into any data center you have. So it can drop into any cloud data center in the planet.

To make it easier for you to do things, it needs to be less different. And you may think the hardware configuration, the actual physical frame, doesn't matter when you think about DevOps, but it does. Because when you think about the systems that you're running, the environment you're dealing with, everywhere it's different, it's harder to deal with.

So the way that the mainframe needs to fit in today's world is to not be different in any way that provides no value to you. So if it provides value, i.e., it has a high quality of service, it has the capabilities to run those high transaction throughput, that's value. A reliable system, that's value. The fact that it happens to be built backwards and be in the wrong-size frame, that's no value. So we're removing no value.

The other thing to think about when you're thinking about your development environment: you use lots of different languages. Java, Node, Python, Swift, take your choice of languages. If those languages run on z/OS, then z/OS is just a target of your DevOps pipeline. And all of those do run on z/OS, by the way. So if you are doing development with Swift or with Node or doing Python, it doesn't matter. They all run there. So you can just use it as a target for your pipeline.

But the other thing we want to do is make sure not only that these languages run, so if you're doing Java as part of your pipeline and Java is your language, why not? Let's make it run better there than anywhere else.

So you've got to be really careful with this first word, because it's "pauseless." Not that it does not pause, but it pauses less. It's always fun, garbage collection as an example. So if you can run your Java applications and just run them faster, more reliably, then you have a system that's at the target of your DevOps pipeline.

The idea with all of these changes is to make it not different. So Java's Java's Java. You run it on any platform you want, just target z/OS as one of those platforms.

But the biggest thing we've been doing, and you're going to notice on this chart a whole bunch of open source. The biggest thing we've been doing is saying, you have a pipeline. You have decided on some choice of pipeline. It might be GitHub, it might be GitLab, it might be Bitbucket, it might be... There are other choices. But let's say it's one of those.

Why shouldn't your COBOL or your PL/I be sitting in the same repository? There's no reason. Git runs on... Now, you're going to run your GitLab, your GitHub, your Bitbucket, wherever you run it, because you run it somewhere. And on your build server, you're going to put a Git client. Well, on z/OS, you're going to put a Git client, just like you put anywhere else. It runs there just like anything else. So you can download it, put it on your z/OS environment.

So now I've got Git. I have Git available to use. There's no reason to use a different SCM. There's no reason to separate the source code.

If you see on my picture, I have Jenkins. The Jenkins slave runs on z/OS. Runs there. There's an environment. I can just run it, just download it. It's open source, it's Java, it just runs. We make it run. So if your pipeline of choice is Jenkins, it runs there.

What is your pipeline? That's the question. Whatever your pipeline is, then probably it works. And if it doesn't, well, let me know. But we've been going through and porting open source tools to z/OS to allow you to do your pipeline there.

I have a sample pipeline that we demonstrate all the time that has the EGit client sitting in Eclipse with all the editors I need. Well, we have one with GitHub, and we have one with GitLab, actually. So we have Git as an SCM, and then we have this IBM Dependency Based Build, which happens to be a product, which is really Groovy, running on z/OS, with some extensions to allow you to do things you really have to do on z/OS because we needed to.

As the build tool, Sonar works there. Sonar allows you to scan your source, get your results. We have code coverage capabilities, so you can feed in your code coverage. There's something called zUnit. You've all heard of JUnit, right? Or xUnit? Well, there's zUnit. So you can do your zUnit testing.

And you'll notice up there that I have Nexus and Artifactory. One of the last problems that we had is if you really know z/OS very well, you'll probably realize that when you create an artifact, it's a load module, and it doesn't really easily come off the z/OS environment. I'm being a little polite. It doesn't, unless you do a lot of extra work.

And we know how to do it. You can XMIT it somewhere. You can do things, but it takes a lot of work. In order to solve that problem, we've added a fix, as I call it. Others in development called it enhancement request, but I call it a fix, to allow you to take your artifacts off z/OS and put them in Artifactory or Nexus just like the rest of your applications.

So if you want to store your z/OS application alongside all of the rest of your applications, you can now. So you have the capability of taking whatever your pipeline is and putting it in the picture. And so z/OS really is just another target. All of the processes and practices that you use today, test-driven development, take your choice of practice that you're using in the distributed side, you can just do it in the z/OS side.

Now, the two most important things about this picture to me. One is, take your own picture and it'll work, probably. And two, it really does work. It really is possible. And this message that this pipeline is possible doesn't seem to be being heard. I stand up at a lot of conferences, I speak to a lot of clients, but the message isn't getting out. It really is possible. There is nothing about z/OS that has to make it different.

Now, today, most z/OS customers use a library management system instead of this pipeline. They're using a system, and I'll pick on my own, IBM's SCLM, that controls the library structure, and it handles the source code management, and it handles the promotion process. This limits your ability to have dynamic environments. It limits your ability to create environments on the fly and have as many environments as you want. It gives you a static hierarchy that you move up.

If you move to this model, the other thing you get is the ability to dynamically create environments. I can, with this model, say I've got my own branch that I'm doing development on. I've got a feature branch or a personal branch. I want to build that branch, and I want to deploy it into my own environment to do my own testing.

If you have this conversation with your Z guys, other than the first thing they'll probably do is laugh, the second thing they'll probably do is say, "That's impossible." A third thing they'll say is, "It already works the way we're doing it now, so why should we change?"

I can give you more. All of those answers are wrong because that system that you're running there has value to your company, and you need to be able to get people younger than me willing to work on it.

I'm only 52 or whatever, somewhere in there, and I'm the young end of the average age, probably, of the mainframe developers. We need the 20-somethings and the 30-somethings willing to work in this environment. If you have a pipeline like this, why wouldn't they work there?

And if you're going to tell me that 20-somethings don't want to develop in COBOL, I will tell you that the easiest way to get a 20-something to develop in COBOL or PL/I is to tell them they can't. Tell a developer they can't develop in a new language. Yeah, that'll last about 12 hours, and they'll come up and say, "Nope, I can do that."

What they won't do is use that green screen thing with that limited ability to do anything. That's the issue in that environment. So if you're sticking them in 1980s technology to do their development process, there's no chance they're going to want to do it.

The other thing that's somewhat hiding in this picture, and I probably should have brought it out: by having Groovy on the system, your new kids can do the automation tasks, the capabilities that the old guys are going to say you have to write JCL for, without any JCL.

If you don't know what JCL is, job control language, I don't think you want to learn it if you don't know it. And apologies for all those of you who had to learn it. It's not the most friendly thing in the world. That's the best way to put it. And I'm not sure any new JCL has been written in the last 30 years. I think people have copied it. I know I copied all of mine. So it's not something people want to learn or do.

So how about Groovy? Is that good enough? Is that easy enough? Is that simple enough? I can do Groovy to do my automation.

Now, there is one piece on this picture that you'll see, and it's UrbanCode Deploy from a deployment standpoint. And I will say today that the only deployment tool, I'm picking on other vendors right now, the only deployment tool that deploys to z/OS as well as any other platform happens to be UrbanCode Deploy. It's because we're the only one that have chosen to implement a solution that runs on z/OS as well as. It's a modern deployment tool that allows you to do that.

There are other vendors that could do this when they decide to, and I wish they would. One of the reasons we have Git running on z/OS is because IBM, we built Rational Team Concert. We made it work on z/OS, and all of the clients said, "Well, that's only one choice, and that's only IBM, so give me another choice." So fine, I gave you Git.

I would love to get another choice. I will say that I'm sort of working on open source tools, and if you have one, we'll look at it. But deployment matters. Deployment is critical. Deployment is key.

One of the things that we found is audits in z/OS are slightly more stringent. Well, I'm going to say I'm not totally sure why they're more stringent, and then I'm going to say I know why. It's usually because z/OS is running the most critical parts of your business application. It's running the banking piece that matters or the insurance piece that matters. It's that back-end system that everybody seems to want to regulate.

And so the audits over there are more strenuous. Having tools that let you satisfy those audits is very critical. And in z/OS, one of the things that we found is you need control on z/OS from z/OS to do a deployment. So having the deployment not have an agent controlled by the security of the z/OS box seems to be a problem, which is why we need people to implement deployment on the box itself, which is why that works.

The other piece of this picture that's not totally clear, but I want to make clear is, in this end, when I'm deploying to these environments, I can deploy to real Z hardware. Yes. But I could also deploy z/OS image into an Intel Linux image running somewhere or running in, in my picture, IBM Cloud, or in any cloud environment.

So there's z/OS that can run on Intel Linux that allows you to dynamically provision systems into a cloud to run your tasks to do your testing. It's called zD&T. It's been available for quite a while: Z Development and Test Environment.

You could also use z/VM. I'm sure you've heard of VM or VMware. They got the name somewhere. They got it from VM, as in virtual machine. It's been running in z/OS for a very long time. We've had it forever. You could use z/VM to provision images on your z/OS system to do this, or you could really just provision it within the existing LPAR if you want to. It's up to you. There are multiple ways to do it, but you can dynamically provision into this back environment the same way you do in every other environment, and please do.

One of the biggest problems I find in the Z customers is they don't have the test environments to allow people to do the testing they need to when they want to. This morning, when the speaker spoke about why should somebody have to wait three to five weeks to get their feedback on their test, I was somewhat thinking, "Congratulations. It's only three to five weeks?" There's some customer shops I walk into that tell me it's six months before the developer might get their feedback. That's wrong.

If you can create your own environment, you can dynamically provision your own environment, you don't have to worry about that problem. I can get my development and test done.

The other thing that's critical in this picture is automated testing. And if you talk to many organizations, go home, ask, "How much of your z/OS application testing is automated?" And if the answer is anything greater than 10% or zero, I'll say, "Congratulations. You're in the rare minority."

Automated testing also works in this environment in the exact same way, by the way, it works everywhere else. Just for some reason, we don't do it.

Got a reason why? Big reason why. I have 30 years of legacy code that doesn't have a set of automated tests against it. You're right, you probably do. But you also have the ability to create an API for any of the functions that you have.

I'm going to say this, and every time I say it, I regret it. I'm hoping that you don't have users using 3270 green screens very often anymore. I'm hoping you've put something more friendly in front of it for most of your users. Okay. You may or may not have, but if you have, then there's an API there.

There is an interface. Many people are putting REST interfaces to their Z system because now why shouldn't I? I can REST-enable any of my z/OS applications simply using z/OS Connect. I've got a REST interface. I can do the calls that I need to do.

I can create an API test at that layer, and I can easily create at least a set of a regression bucket that can give you the code coverage you need to prove that you have enough automated tests to start out with.

So if you start out with this, build up your regression bucket using API tests because you don't have the time to go in and create unit tests for everything. In fact, there's just no way you're going to get enough unit tests on 30 years of legacy.

Now, I will say that that answer is probably true for your legacy Java too, or your legacy .NET, or your legacy everything else. Anything you don't have tests for, you can create API tests faster as at least a regression bucket to get started to allow you to move in this pipeline.

Because this pipeline is great. It's wonderful to have a pipeline. And if you don't have any automated tests, you're not going to move any faster. Sorry. You may get a little better with automated deployment, absolutely true, but you won't get faster until you have the automated testing.

The other thing that's important to recognize about this picture is that in many organizations, the answer is, "I already have this."

So if you talk to a historical Z person that has been in the industry doing the same thing for the last 30 years, they will say, "I already have a central SCM that manages all my source. I already have a common build tool that builds all my stuff, and I already have a common deployment tool." That's the production control tool they're currently using.

And yes, they have that. But that tool does not allow them to easily add automated tests. It does not allow them to easily add new environments. It does not allow them to do most of the functions you would expect a modern SCM to do, like branching. Doesn't have such a concept. So all of those things you're lacking in the system.

Yes, they started out by having a common way of doing deployment, and that's a huge step forward, but that's not the whole answer. You really need to bring modern technology into this pipeline and have the common shared backlog.

Now, what I need help with, and this is really what I need. The first one: what am I missing? What do you use in your pipelines that is open source today that you want to be able to use for z/OS? What am I missing? What's in your pipeline? Tell me. I'll get it there. It's not hard.

Well, okay, it depends on what language it's written in. I'm working on GitLab already, so that one, GitLab CI, I'm working on already. But tell me what's missing. Tell me what's not there.

And the second one is probably the most important message, because I come to DevOps Enterprise Summit, I speak at many events, and every time I come, I get the same, "I didn't know you could do that."

Help me get rid of the perception that the mainframe is that old thing sitting over there that can't do any of these new processes. Help me help yourselves by having the industry understand that really, z/OS is just another target platform. It happens to have a set of qualities of service that are very unique, but other than that, it's just a big server. Happens to be a very big server, but it's just a big server sitting in that environment.

And you can do the things that you do in every other environment on this one. The same languages, the same capabilities, the same things.

Now, this session has been all z/OS-focused, but I do have to say that the Z machine does actually run more than z/OS. So if any of you happen to be a TPF customer, I can tell you how to do the exact same thing with TPF, Transaction Processing Facility. TPF, if you don't know that, is another operating system that happens to run on z/OS, and it happens to run the major airlines of the world and a few other key critical things you might care about. It's the fastest transaction processing system.

And the other one I left out is, oh, by the way, you know the mainframe runs Linux, right? So it's just the largest Linux machine that can scale up in the environment. And oh, by the way, it provides secure containers, so you can have a secure container. And by the way, the mainframe has encryption available, so you can encrypt everything on the box at all times.

So the box itself provides a set of capabilities that allow you to do it as a target for anything you want.

For this, we're focused on z/OS because z/OS is the one that we had to do things to. If it's running Linux, eh, it's running Linux. You just do the same thing you're always doing.

For z/OS, we've been making changes. This last change of allowing you to take artifacts off the box was a key critical change because I can now make a package that looks the same as everything else. I can now make a package that doesn't confuse people. It'll make it easier for other vendors to implement deployment on z/OS and allow us to use things like Artifactory or Nexus.

So hopefully, this was helpful. Hopefully, this gave you some new insights in the environment. And if you didn't know that z/OS is just another target platform that you can use your standard pipeline for, hopefully you now know that and you will share my message.

And if you have open source tools you're using, please let me know. Thank you.