testing and automating accessibility
Debugging jest-axe Violations in CI
A jest-axe test that is green on your laptop and red in CI is almost never a flaky framework—it is a timing, scope, or environment difference that your local run happened to paper over. This guide is a triage manual: how to read the violations array axe returns, how to stop testing the DOM before it has finished rendering, when to scope axe to document instead of container, and which failures are jsdom artifacts you should stop chasing in jest-axe entirely. It builds on the patterns in Component Testing with jest-axe and feeds the pipeline rules covered in Gating Accessibility in CI/CD Pipelines.
The rules most often implicated are 4.1.2 Name, Role, Value (a control lost its name during an async render) and 1.3.1 Info and Relationships (a reference broke because the target mounted in a portal axe never scanned).
Reading the Violations Array
When toHaveNoViolations fails it prints a formatted summary, but the structured data behind it is what you debug from. Each entry in results.violations is a rule failure; each node inside it is a specific element that broke the rule.
const results = await axe(container);
results.violations.forEach((v) => {
console.log(v.id); // rule id, e.g. 'button-name'
console.log(v.impact); // 'minor' | 'moderate' | 'serious' | 'critical'
console.log(v.help); // one-line description of the rule
v.nodes.forEach((n) => {
console.log(n.target); // CSS selector path to the offending element
console.log(n.html); // the element's outer HTML at failure time
console.log(n.failureSummary); // exactly what to fix, e.g. "Fix any of the following..."
});
});
The four fields that resolve most cases: id tells you which rule (look it up in the axe-core ruleset), impact tells you severity, target is the selector that points at the element, and failureSummary spells out the remediation. In CI, the printed html is your most valuable clue—it shows the DOM as axe saw it, which is frequently a half-rendered state your local timing skipped past. Log the full violation in CI rather than only the matcher's summary when a failure is hard to reproduce.
Async Rendering: Wait Before You Run axe
The classic CI-only failure is a race. Your local machine renders the component fast enough that the DOM is settled by the time axe runs; CI runs slower (or faster, exposing a different order), and axe inspects an intermediate state—a button before its label loads, an input before its error mounts. The fix is to never call axe until the DOM you intend to test exists.
import { render, screen, waitFor } from '@testing-library/react';
import { axe } from 'jest-axe';
test('runs axe only after async content settles', async () => {
const { container } = render(<UserProfile id="42" />);
// BAD: axe here may inspect a loading skeleton with unnamed controls
// expect(await axe(container)).toHaveNoViolations();
// GOOD: wait for the real content to appear first
await screen.findByRole('heading', { name: /ada lovelace/i });
// Or wait on a stable post-load condition before scanning
await waitFor(() => expect(screen.queryByText(/loading/i)).not.toBeInTheDocument());
expect(await axe(container)).toHaveNoViolations();
});
findBy* queries retry until the element appears or time out; waitFor retries an arbitrary assertion. Use whichever expresses "the DOM is ready." Running axe behind an explicit wait removes the entire class of order-dependent CI failures—if the component has finished rendering before axe runs, the result is deterministic across machines.
Portals: Scope to document, Not container
React portals render outside the React subtree—typically to document.body. Modals, tooltips, toasts, and popovers commonly use them. If you scope axe to the container returned by render, the portal content lives elsewhere in the DOM and axe never sees it. Locally you might not notice; in CI a portal-rendered violation surfaces only when something else changes scope.
test('modal in a portal is scanned by axe', async () => {
const user = userEvent.setup();
render(<DeleteDialog />); // dialog portals to document.body
await user.click(screen.getByRole('button', { name: /delete/i }));
await screen.findByRole('dialog');
// container would MISS the portal—scan the whole document instead
expect(await axe(document.body)).toHaveNoViolations();
});
The asymmetry to remember: Testing Library queries like getByRole search the entire document by default, so they find portal content fine—which is exactly why a portal can pass your role/name assertions while axe(container) silently scanned nothing. When a component portals, scope axe to document.body (or the portal root). When it does not, keep the tighter container scope so axe stays focused on the component under test rather than test scaffolding.
Flaky Timing and Fake Timers
Some CI flake comes from timers, not rendering: a toast that auto-dismisses on a setTimeout, a debounced validation, an animation gate. If axe runs while a timer-driven element is mid-transition, the result is nondeterministic. Control time explicitly instead of hoping the wall clock cooperates.
test('scans the toast before its auto-dismiss timer fires', async () => {
jest.useFakeTimers();
const user = userEvent.setup({ advanceTimers: jest.advanceTimersByTime });
render(<SaveButton />);
await user.click(screen.getByRole('button', { name: /save/i }));
await screen.findByRole('status'); // toast is present
expect(await axe(document.body)).toHaveNoViolations();
jest.runOnlyPendingTimers(); // let the dismiss timer fire deterministically
jest.useRealTimers();
});
When you adopt fake timers, wire userEvent.setup({ advanceTimers }) so user interactions still flush microtasks—otherwise queries hang. The principle is the same as the async section: make the DOM state deterministic before axe inspects it, and CI stops disagreeing with your laptop.
What jsdom Simply Cannot Test
Some "CI failures" are not failures—they are jsdom limitations producing noise. jsdom has no layout or paint pipeline, so any rule needing computed geometry or color cannot run meaningfully:
color-contrast(1.4.3 Contrast (Minimum)) — no computed colors; disable it in jest-axe config and verify in a browser.target-size(2.5.8 Target Size (Minimum)) — no real pixel dimensions exist.- Visibility/occlusion rules — geometry is effectively zero, so "is this visible" is unanswerable.
Disable these explicitly so they never produce a misleading result:
const results = await axe(container, {
rules: { 'color-contrast': { enabled: false } }, // jsdom can't compute it—own it in the browser
});
Chasing these in jest-axe wastes triage time. Move them to a real-browser run and gate them there. The pipeline-level strategy for splitting structural component checks from rendered checks lives in Gating Accessibility in CI/CD Pipelines.
How to Verify
Reproduce the CI condition locally before declaring a fix. Force the slower, deterministic path and run the exact command CI runs:
CI=true npx jest --runInBand --ci
--runInBand removes worker parallelism (a frequent source of order-dependent flake), and --ci makes Jest treat the run like the pipeline does. If a test still passes locally but fails in CI, log the full violation—id, impact, target, html, failureSummary—from the CI run and read the html to see the DOM state axe captured; it usually reveals an async or portal scope gap. Confirm the fix by re-running the same command several times: a genuinely deterministic test passes every time. Finally, for any rule you disabled as a jsdom limitation, verify it in a real browser and add a manual NVDA/VoiceOver spot-check so the coverage you removed from jest-axe is not simply lost.
Common a11y Mistakes
- Scoping
axe(container)for portal content — axe scans nothing while your role queries still pass, hiding the violation. Scope todocument.bodyfor portals. - Running axe before async DOM settles — produces order-dependent CI failures. Gate axe behind
findBy*/waitFor. - Leaving
color-contrastenabled in jsdom — yields misleading results; disable it and verify in a browser. - Parallel workers masking flake — debug with
--runInBandso failures are reproducible. - Reading only the matcher summary — the structured
violationsarray (target,html,failureSummary) tells you exactly what and where.
Conclusion
CI-only jest-axe failures decompose into a short checklist: did axe run after the DOM settled, did it scope to the place the content actually rendered, were timers deterministic, and is the failing rule something jsdom can even evaluate. Read the violation nodes, fix the scope and timing, push layout and color to the browser, and your component suite becomes a reliable gate instead of an intermittent annoyance.
Frequently Asked Questions
Why does my jest-axe test fail in CI but pass locally?
Almost always timing or scope. CI runs at a different speed, so axe may inspect a half-rendered DOM your local run skipped past, or worker parallelism exposes an order your laptop didn't. Gate axe behind findBy*/waitFor, run with --runInBand to reproduce, and read the violation's html to see the state axe captured.
My modal passes getByRole but axe reports nothing—why?
The modal renders into a portal on document.body. Testing Library queries search the whole document, so getByRole finds it, but axe(container) only scans the render container and misses the portal entirely. Scope axe to document.body when a component portals.
Which violation fields should I log to debug CI failures?
Log id (the rule), impact (severity), target (selector to the element), html (the DOM as axe saw it), and failureSummary (the remediation). The html field is the most useful for CI-only failures because it reveals the exact intermediate state that triggered the rule.