Accessibility Testing with Intern

Intern already has a wide array of capabilities and today we’re pleased to announce one more: accessibility testing. Thanks to a generous award from Mozilla Open Source Support we’ve created the intern-a11y plugin, which allows users to run accessibility tests on pages or components using Intern.

How it works

Web accessibility testing generally works by having a scanner check a web page or web page fragment for rule violations. The most commonly used rules are defined in the World Wide Web Consortium (W3C) Web Content Accessibility Guidelines (WCAG) and the General Services Administration (GSA) Section 508 Standards. Scanners can check for violations at different rule levels without a set and can typically be configured to only check a subset of rules.

Regardless of the scanner being used, automated accessibility checks are only a part of the solution. A human is still required to fully verify whether a site is accessible. For example, an automated tool can check that images have alt tags and that form elements have visible labels, but it can’t verify whether the information in those tags and elements is meaningful or accurate. Automated scanners can still save time by quickly detecting a large number of common design and layout issues, helping a manual reviewer focus on problem areas.

Intern doesn’t reinvent the wheel and performs accessibility checks itself as there are several great tools that already exist. Rather, intern-a11y provides an interface to external accessibility scanners. Two such scanners are supported in the initial release: Tenon.io, a cloud-based testing service, and aXe, an injectible client-side library. Both of these scanners can check entire pages or portions of a page or app, configured to use the rule sets necessary for the evaluation.

Testing with intern-a11y

Intern-a11y is an Intern plugin, so the first step is to install it (along with Intern):

$ npm install intern intern-a11y --save-dev

To actually use the plugin, you’ll import intern-a11y, using methods from one of its service modules. The package uses CommonJS modules rather than AMD, so it must be loaded with the Node.js loader plugin with Intern 3.4. However, that also means that no special loader configuration (i.e., a packages entry) is required in the test config file to use intern-a11y. Below is a simple suite that uses the tenon module:

define([
	'intern!object',
	'intern/dojo/node!intern-a11y'
], function (registerSuite, a11y) {
	var tenon = a11y.services.tenon;

	registerSuite({
		name: 'accessibility tests',

		'check page': function () {
			return tenon.check({
				source: 'http://mypage.com'
			});
		}
	});
});

In this example the tenon module will perform an accessibility check behind the scenes using Tenon.io. If the check reports any failures, the call to tenon.check will fail and throw an error. The check method is essentially an asynchronous assertion. This means that its result (a Promise) must be returned from the test so that Intern will wait for the check to be complete.

Tests using the axe module may be authored in a similar fashion, although the axe module works in a fundamentally different way. While the tenon module makes HTTP calls to an external service, the axe module uses Intern (specifically, the WebDriver call provided by Leadfoot) to load a test page and inject the aXe library into it. This means that axe can only be used in functional tests executed using intern-runner.

Reporting results

The package also includes a custom reporter, A11yReporter, that will save complete accessibility reports to a file or directory. When Intern runs a check using aXe or Tenon.io, a lot of data is typically returned. Intern only needs to see if there are errors to pass or fail a check. However, it is useful to see more information about what failed. A11yReporter creates HTML reports in a standardized format that lists test failures returned by the scanner, including information about specific rule violations.

To use the reporter, just add a descriptor to the reporters list in your intern config:

reporters: [
    {
        id: 'dojo/node!intern-a11y/src/A11yReporter',
        filename: 'a11y-report.html'
    },
    'Runner'
]

After the test run is finished, results from all the accessibility tests will be aggregated into the filename specified in the reporter descriptor (a11y-report.html by default). It is also possible to provide a directory for filename, in which case the results from each failing test will be written to a separate file in the target directory.

If there were any failing tests, you’ll end up with a report like the following. The report will contain a section for each failing test, and each test section will contain a list of cards describing specific rule failures.

What’s in store

More scanners! Two are supported in the initial release, but intern-a11y was designed to make adding new scanners easy. We’d also like to improve the reporting format to make parsing results even easier. Have other ideas? Let us know in the project issue tracker!