Accessibility Testing Tools

Avatar of Chris Coyier
Chris Coyier on

There is a sentiment that accessibility isn’t a checklist, meaning that if you’re really trying to make a site accessible, you don’t just get to check some things off a list and call it perfect. The list may be imperfect and worse, it takes the user out of the equation, so it is said.

Karl Groves once argued against this:

I’d argue that a well-documented process which includes checklist-based evaluations are better at ensuring that all users’ needs are met, not just some users.

I mention this because you might consider an automated accessibility testing tool another form of a checklist. They have rules built into them, and they test your site against that list of rules.

I’m pretty new to the idea of these things, so no expert here, but there appears to be quite a few options! Let’s take a look at some of them.

aXe

The Accessibility Engine for automated testing of HTML-based user interfaces. Drop the aXe on your accessibility defects!

aXe can take a look at an HTML document and find potential accessibility problems and report them to you. For example, there are browser extensions (Firefox / Chrome) that you give you the ability to generate a report of accessibility errors on the page you’re looking at.

At it’s heart, it’s a script, so it can be used in all kinds of ways. For example, you could load up that script in a Pen and test that Pen for accessibility.

There is a CLI so you can integrate it into build processes or testing environments or deployment flows or whatnot.

Looks like maybe intern-a11y can help script aXe for extra functionality.

Pa11y

Pa11y is your automated accessibility testing pal. It runs HTML CodeSniffer from the command line for programmatic accessibility reporting.

Pa11y is another tool along these lines. It’s a script that can test a URL for accessibility issues. You can hit it with a file path or URL from the command line (pa11y http://example.com) and get a report.

As well as use it from a Node environment and configure it however needed. It’s actually intentionally meant to be used only programmatically, as it’s the programmatic version of HTML_CodeSniffer, the bookmarklet/visual version.

There is also a native app version called Koa11y if that makes usage easier.

Seren Davies recently wrote about a specific scenario where they picked Pa11y over aXe:

We began by investigating aXe CLI, but soon realised it wouldn’t fit our requirements. It couldn’t check pages that required a visitor to log in, so while we could test our product pages, we couldn’t test any customer account pages. Instead we moved over to Pa11y. Its beforeScript step meant we could log into the site and test pages such as the order history.

Google Accessibility Developer Tools

Google is in on the game with Accessibility Developer Tools.

Its main component is the accessibility audit: a collection of audit rules checking for common accessibility problems, and an API for running these rules in an HTML page.

It’s similar to the others in that it’s designed to be used different ways, like as Grunt task, from the command line, or the browser.

Addy Osmani has a11y, powered by Chrome Accessibility Tools, which appears to provide a nicer API and nicer reporting.

It seems like most of Google’s website auditing weight is thrown behind Lighthouse these days though, which include accessibility tests. For example, the “Buttons Have An Accessible Name” test, but that test is actually aXe under the hood.

It’s unclear to me if Lighthouse runs a complete and up-to-date aXe audit or not, and if the Accessibility Developer Tools are sort of deprecated in favor of that, or what.

Automated Accessibility Testing Tool (AATT)

PayPal is in on the game with AATT, a combination and extension of already-mentioned tools:

Browser-based accessibility testing tools and plugins require manually testing each page, one at a time. Tools that can crawl a website can only scan pages that do not require login credentials, and that are not behind a firewall. Instead of developing, testing, and using a separate accessibility test suite, you can now integrate accessibility testing into your existing automation test suite using AATT.

AATT includes HTML CodeSniffer, aXe, and Chrome developer tool with Express and PhantomJS, which runs on Node.

It spins up a server with an API you can use to test pages on other servers.

accessibilityjs

GitHub themselves recently released accessibilityjs, the tool they use themselves for accessibility testing. They use it on the client side, where when it finds an error, it applies a big huge red border and applies a click handler so you can click it to tell you what the problem is.

They scope it to these common errors:

  • ImageWithoutAltAttributeError
  • ElementWithoutLabelError
  • LinkWithoutLabelOrRoleError
  • LabelMissingControlError
  • InputMissingLabelError
  • ButtonWithoutLabelError
  • ARIAAttributeMissingError

Tenon.io

Tenen.io is perhaps the easiest of all of them to get started with, as the homepage has a validator right up top where you can copy and paste HTML or drop in a URL to validate.

Tenon.io can identify 508 and WCAG 2.0 issues in any environment – even on your developer’s laptop. Because production is a bad place to discover bugs.

It has a free 30 day / 500 API call trial, and is a paid product beyond that.

Tenon.io integrates in loads of places. Karl himself told me:

We have a CLI. We have Grunt & Gulp plugins, Node modules, and plugins for Chrome, Firefox, IE, and Opera. PHP Classes, Ruby Gems, CMS integrations for WordPress, Drupal, etc.

Honorable Mentions

I’m not intentionally trying to feature or hide any particular accessibility testing tool. All this stuff is new to me. It just seemed like these were a lot of the big players. But web searching around reveals plenty more!

  • Tanaguru: “Automated accessibility (a11y) testing tool, with emphasis on reliablity and automation”
  • The A11y Machine “is an automated accessibility testing tool which crawls and tests pages of any web application to produce detailed reports.”
  • tota11y: “an accessibility (a11y) visualization toolkit”