Dylan Barrell

So it's about 20% of the population has a disability of some sort. And if you take the halo of the people around them that are important to them, it's almost three quarters of the population that is somehow impacted or interested in things being accessible.

So that's a really good reason, because if you are accessible, that means more people are going to come to your site, if that's important. Your site being usable is important to the conversion rate, to customer acquisition, and to selling more of your product. Then that's going to have a positive impact on that.

The second thing is lower cost, because if somebody has to use your service, say you're a school and they have to use that and you can't do everything digitally, if they have to go to some other channel, that channel is generally much more costly. So if you're a bank and they have to come into your branch, servicing that person through the branch is going to be a lot more expensive. So being able to service people in a digital channel is going to save you costs.

And then finally, this one gets a lot of the attention, but really it's the last reason from our perspective that you should pay attention to this. There is brand damage potential and also the risk of fines or litigation and damages if you're not accessible. Over the last 20 years, we've seen a real rise in both legislation and litigation related to accessibility, and it's just going to increase.

So what about within development, though? Well, this is a famous study from IBM that shows what the cost of fixing a defect is if you find it in design versus other phases of a development lifecycle. And really this is one of the main reasons that DevOps has become as popular as it is, because in trying to do things in an agile way, we need to automate things as much as possible.

The same principles have gone into DevOps, as you can see up here. And that is that if you can find and fix something earlier on in the process, like if you can find and fix something in design and it costs you 1x, then if you can find that same thing and fix it in development, it's going to cost you about three times as much. If you find it in QA testing, 10 to 12 times as much, and in production, it's going to be anywhere between 30 and 100 times as much to fix that.

So what that means for accessibility is that if you have to do it, the earlier you do it in the lifecycle, the more quickly you're going to do it, the more cost-effective that's going to be. And to be honest, sometimes when things get to production and we find them there, they end up not ever getting fixed. And so you're exposed to all those negative side effects we talked about earlier.

So, okay, that's the justification for why you should do it. But what is accessibility? Accessibility is something that, if you're not familiar with it, you'll be like, where do I start? Luckily, there are some standards that you can go to. There's something called WCAG. We call it WCAG. It stands for the Web Content Accessibility Guidelines.

This is a set of international standards that started in the nineties. The first version was published in May 1999 and has been steadily developed since then. The one that we're currently on is WCAG 2.2. WCAG 2.0 was published in December 2008. It's a pretty stringent process to go through to get changes, so it took us another 10 years to get to 2.1, but now we've got more frequent updates. And so we're continuing to improve that.

And that can really guide you. It helps you understand what you need to do in order to be accessible and gives you some guidelines that can help you and your developers and your development teams. But that covers kind of the what you have to do. It doesn't cover a lot of the how, and there are some other standards that fill some of the gaps.

So, for example, HTML doesn't have a lot of the semantics that you need for a rich internet application. So WAI-ARIA is another important standard that was developed in 2014 and continues to be enhanced. That covers the ability to add semantics that aren't possible through native HTML.

All of these things really focus around four different principles. The principles we call perceivable, operable, understandable, and robust, which are the core principles about how you make some digital experience accessible.

So let's look at two of those in a little bit more detail. Perceivable is really about: if you're trying to convey some information to me through your user experience, through a web app or a mobile phone app, can I actually perceive the information that is being presented?

So an example of that is an image. Here we have an image of a crowd of people high-fiving each other, and you may recognize one or two faces in the crowd. One of them is in fact ex-president Bill Clinton, and he's about to high-five somebody else in the crowd. And you could describe this image with that text, and that would be a good description.

But what's actually missing there is kind of the context of why is Bill Clinton in this crowd? What is he doing? The additional context there is that this is a get-out-the-vote rally, and he's there to help that initiative. So that's really the context that you need to provide in a textual form so that somebody with a screen reader can actually perceive that information in the same way and get the same context out of it as somebody who can actually see that image. And this is a good example of how you can achieve perceivable for a blind person as it relates to visual content.

An operable example: operable is about, can I actually use the user interface controls in the user interface? So here, this is on Deque's website, our mega menu, where you can see a lot of different options. And this is what it looks like if you're using a pointer device. So it's just various different options that are there.

But if I'm a blind person using a screen reader, once again, which is a keyboard-equivalent device, or if I'm somebody who can see like Stephen Hawking, but I can't necessarily use a pointer device, I have to use a keyboard-equivalent device, then how do I interact with this?

So in this example here, you can see one of the options has been highlighted, and that's what the focus looks like when that element is focused using a keyboard, so that I can move through with the keyboard, and then by pressing the return key, I can actually select that option and it'll navigate me to that page.

So operable is about: with the different devices that different people have, given their different disabilities, can I actually use those user interface controls?

Some interesting statistics about WCAG. There are 50 success criteria in WCAG version 2.1 in levels A and AA, but only three of those account for 60% of the defects. At Deque, we do a lot of assessments for our customers. We collect that data and we use it in different ways statistically to drive and prioritize our product development, but we also look at it for things like this.

Six of those success criteria account for 80% of all the defects. And if you look at the top 15, that accounts for almost 95% of all defects. So as an organization, if you're starting out, one of the strategies you can use is to say, okay, let's reduce our risk, or let's get the most benefit. And if you focus on these 15 success criteria, you can get 95% of that benefit, or looked at the other way, reduce 95% of your risk.

Here is the table showing that same information. This is downloadable from our website. This is a free report you can download. Go to www.deque.com or visit us at the booth and you can get a link to that.

So, okay, what tools can help you? Because in DevOps it's really about being efficient, automating everything, and having the least negative impact on your developers.

There are three tools that I think are really great places to start. The first thing about accessibility is if you start with a design and the design thinking, and you think about people with disabilities, you generally end up with a design that's much easier to make accessible. And it's also easier to use for all your users, not only the ones with disabilities.

And a tool that's really critical here is a tool that can test that design for accessibility and also then help the designer communicate that information to the rest of the development team. axe for Designers is an example of a Figma plugin that can help you with that.

And then the next two tools are great, what I call set-and-forget DevOps CI/CD pipeline tools. The first one, axe DevTools Linter, integrates both into the IDE, so the developer can get feedback as they're typing, but also within something like GitHub Actions inside your CI/CD pipeline or inside Jenkins. You can have that linter look at the changes that are part of a pull request or part of a build process and actually surface those issues, so that you can present that information inside your GitHub pull request.

And then the pull request reviewer and the developer have the option and the information that they need to decide, are we going to fix this first, or are we going to let this go through?

And the second one is what we call the axe Developer Hub, which has two pieces. It's got this watcher mode thing, which integrates automatically into your test framework. And then any test that gets written automatically picks up accessibility testing. So as a developer, I don't need to think about, do I need to add a call to check accessibility here or there? It'll automatically detect all the UI changes and analyze them for accessibility.

It sends them to the Developer Hub at the back end, which provides that reporting and that same feedback back into your CI/CD pipeline once again. So those are great tools for you to look at.

Visit us in the expo hall. We also have going on for free during the conference, at the back of the expo hall, if you go through the tables, there's an accessibility awareness lab going on where you can actually see people using screen readers, use some of these devices yourself, and learn a lot more about accessibility.

And with that, I'm going to hand over to Steven from USAA. They're trying to embed accessibility into their DevOps pipeline.

Steven Rambach

Awesome. Good morning, everybody. I am Steven Rambach. I have been at USAA for about 10 years as an engineer. Most of my time there has been spent working on, I guess, testing infrastructure. And then the last couple years or so, I've been more focused on accessibility testing and the infrastructure behind that at USAA.

So I'm here today to kind of talk as the voice of someone who recently broke into kind of a DevOps and CI space with accessibility testing. And I'm obviously excited to talk to many here to learn how we can improve.

So for those who don't know, USAA is a financial services provider for our military personnel, right? So whether those are veterans, active duty, their families, anytime we have a discussion internally or externally, we like to show our mission slide. As you can see here, it talks about our core values, our behaviors, et cetera.

So what I would say too, the way this conversation ties back to our mission, I think, is pretty evident. We have a lot of disabled members. We have a lot of members that are visually impaired, right? So all of our digital channels need to be as accessible as possible. So the more we bake in accessibility testing to our business processes, our IT processes, et cetera, the more likely we're going to succeed at making those digital channels as strong as possible for those members.

So when I started at USAA, we had more waterfall projects than we do today. They were monolithic in testing scope. So there's a lot of manual consulting, right? We had groups of individuals that were very experienced with screen readers such as JAWS, NVDA, et cetera. And so they would take all of these pages of this application that was being built, scan them, give back a big CSV report or something of the nature. And as you can imagine, there's a lot of kind of clunky back and forth with that.

They also used browser extensions that scan against WCAG as well. And so what was happening was there's really limited hands-on manual tooling for the development community, right? So developers could request it, and they would if they had a passion for it. But as you can imagine, when it's not explicitly part of the process, that manual tooling isn't used as much as we would've liked to see.

Also, since everything wasn't centralized from a process perspective, you can imagine too, there's version mismatches in what rules are being run as well when everyone is kind of doing their own thing with it.

So fast forward a few years, at least into my career, we see USAA is fortunately transitioning to CI environments, right? We have pipeline stages, things of that sort. So this is great, because now we can streamline and automate things such as code review and deploy, vulnerability management, and then obviously testing, right? Unit testing, functional integration. And then I think we were in the works for security and performance. Accessibility wasn't quite there yet, right? And so we'll talk about that.

Also, the automated test evidencing was very important. Like I mentioned, everyone was using spreadsheets, kind of doing a big manual back and forth. There was really no reason why we couldn't automate this.

So that journey for CI continued for a few years. Then I would say a few years back we started getting the questions and the desire to, hey, why not put accessibility testing in there, right? With all the other testing types.

Because we were a few years already into developing the CI journey, standards were being established, things of that sort. We wanted minimal developer disruption, obviously, right? CI was already fairly new for everybody to an extent, depending on which area you worked in. So the more we can kind of keep this hands-off, the better, with the obvious exception of developers needing to learn kind of the ins and outs of accessibility testing in general.

And then like I mentioned before, hands-off evidencing. So at this time too, or around this time, as any big organization may face, there was a lot of kind of compliance desires to track testing more closely and carefully, how it tied back to the changes that were associated with that testing. Basically, kind of pushing everything from the pipeline down into our test management systems. So that's something we had to keep in mind with our strategy as well.

So in terms of minimal developer impact, what we noticed is many of these UI development or front-end development teams were already utilizing browser-based technologies for testing, right? So think WDIO, Cypress, Selenium, just to name a few. They're already getting in there launching browsers in stages or pipeline jobs, and they're already checking those pages for things, right? Is the content what they expected? Is it operating as expected? Are the components there?

So we have a DOM readily available for us already. Why can't we just scan it for accessibility defects?

We also noticed and knew at the time too that XMLs, think JUnit, really just any kind of standardized artifact from test execution, they were already being supported in terms of the test management systems I spoke of, right? So unit tests were being executed, reports and artifacts were being created. That was already going downstream. So why can't we piggyback off that as well for accessibility?

So execution-wise, and I'll show an example in a second, what we were able to accomplish was basically a one-line page scan, right? So what this meant is no new tests, for one. So if a team already had, let's say, 20 or 30 browser-based tests for their small, big, large application, there's not really a need for them to write new tests. They already wrote these tests in a framework that supported, so they didn't need a new framework.

There's a one-time setup in terms of dependencies, configuration files, things of that sort, but that's always to be expected. And then the biggest thing at the time, especially with CI being a relatively new journey, was we weren't adding execution time, right? So as you can imagine, everyone's fighting to get their jobs in that they insist need to be there. Execution time was kind of stacking up as we were figuring all of this out. So not adding to that was a big gain in the strategy.

And then mentioning again, we were actually successful in being able to export and evidence accessibility findings through the artifacts that were created from these one-line scans.

So for the sake of time, this is just kind of a simple example at the top of the testing calls, right? So very bare-bones example, here's a front-end test or browser-based test. We should click a link. And then at the bottom there, you can see we run an analyze call, and that analyze call is actually going to be named homepage.

So what's happening here is my browser already has an object up or a DOM up. It can observe it appropriately. We're going to analyze it against WCAG, and we're going to name any artifacts or reports based off that scan, homepage in this case.

So imagine if you had multiple tests here or a very long, lengthy test, for better or worse, you could run this two or three times, name it homepage dash expand, homepage dash collapse. You get the idea, right? You can kind of have control over all of the page states you want to evaluate and record.

And so this is a very simplified example, obviously, but what we would see too and how we integrated into where the pipeline was at the time, how we added presence to that would be kind of portrayed here.

So in this example, you would build your application, you would do pre-testing such as unit and linting, and then you would publish and deploy your app to some sort of test environment that can be hit via URL. Then via your functional testing, which is that browser-based testing, you already have that in place. It's already spinning up VMs or whatever your team wants to utilize, and then it's going through the pages, validating the pages, et cetera.

This is the perfect opportunity too for those scans to run with that one-liner solution I mentioned. We're going to take those artifacts from each of those one lines. We're going to save them as pipeline artifacts. We're not adding any execution time to this, maybe seconds or milliseconds, worst case. And now we have accessibility artifacts as well as the functional testing artifacts.

So when I spoke to the test management systems as well, as you can see, we would have individual publish jobs for the individual testing types, right? So unit ties back to published unit, whether that's a JUnit XML or whatever the sort. We didn't touch functional at all. That's still operating as intended.

And then now we've just added the additional artifacts and some of the USAA-specific things we may or may not do to get those accessibility artifacts in the state we need them in to push them downstream to our test management systems.

And then there accessibility is now prevalent and visible to whoever accesses the systems. Maybe it's product owners wanting to write up defects or check if everything looks good for a big effort, auditors, et cetera.

So what we solved so far was we were successful in getting accessibility scans automated in a CI setting, right? We were able to do so with minimal code changes, and no extra tests were required. And then like I mentioned, evidencing is present and fully automated just like it was for everything else.

I didn't include this specifically, but I believe we started to scrape metrics and set up data gathering for how this is being used earlier this year. And we're fortunately in, I think, hundreds of projects and thousands of scans. So we're excited to see where that goes in the future.

So what's next? I would say there's a lot of things. For one, currently we're just using WCAG scans, right? Which is a rule set that's been established for a while. We're aware that there may be opportunities in the industry to expand our coverage past that, right? The closer we can get to a hundred percent, the better. So that's something I'm personally excited about delving into for sure.

Admittedly, we do things a little late in the process. I think we did that because we felt we could get the best first-time coverage by doing what we did from a WCAG perspective. But we do have to build and deploy before we test. And so admittedly that's not always ideal from an automated perspective.

So there's opportunities for us to integrate linters in our process as well. So it could happen earlier in the pipeline or even locally. And then design stages, right? You may have UX designers or product owners that want to mock out a page as they're proofing out the requirements. Why not find out just then and there if something is inaccessible?

And then there could always be opportunities to streamline scans even further in the pipeline. So one line of code is obviously great, right? But if you think about it, a lot of these browser-based testing frameworks, they know when a page changes state and things of that nature. So can we delve into that further and make it zero, right? What's stopping us from doing that, per se?

So that's all I had to share today. I'd love to speak with anyone who's had similar experiences or anything to share.