Zero Errors, Yet Still Inaccessible: Why Accessibility Scanners Still Need Human Testing?
Accessibility scanners are incredibly useful, but they do not experience a website the way a person does. That is why human testing remains essential even when automated accessibility testing reports no errors.
I was reminded of this while looking for a website to demonstrate accessibility barriers. I came across two demo versions of the same page: one built to be accessible and the other intentionally built with barriers. I ran both through WAVE, a web accessibility evaluation tool. The two demo pages are designed to pass automated accessibility checks without detected errors. The difference becomes obvious only when you stop looking at the scan and start using the page.
When zero errors still mean barriers
The result of an automated check can feel reassuring. If there are no errors, it is easy to assume there is nothing left to find. Looking only at the result, there was nothing to suggest that one of these pages could create serious problems for some users. If I had treated the scanner result as the final verdict, the test would have ended there.
But the page told a different story once I actually used it. With a mouse, the inaccessible version seemed to work. With a keyboard, some controls could not be reached or operated properly. Nothing about the visual appearance had changed; only the way I interacted with the page had.
That small experiment captured something easy to forget when accessibility becomes a report or a score: the real question is not only whether a tool found an error. It is whether a person can complete what they came to do. If the only “Submit” button in a form cannot be reached with a keyboard, the problem is not simply one faulty element. The user may be unable to finish the form at all.
WAVE itself makes this distinction clear. Its documentation explains that the absence of detected errors does not mean that a page is accessible or compliant. WAVE can help identify accessibility issues, but only human evaluation can determine whether the content is truly accessible.
Zero errors detected can therefore be good news. It just is not the same as zero barriers exist.
Want to see it for yourself?
The two demo pages by AtomicA11y are publicly available:
- Accessible demo: AtomicA11y Basic Accessible Webpage
- Inaccessible demo: AtomicA11y Basic Inaccessible Webpage
Run both through WAVE and compare the results. Then put the scanner aside and try to navigate both pages using only your keyboard. Use Tab and Shift + Tab to move through interactive elements and Enter or Space to activate them. You do not need to know anything about accessibility to notice that the experience is not the same.
What accessibility scanners can and cannot tell us?
None of this is an argument against accessibility scanners. Quite the opposite. Automated accessibility testing is valuable because computers are good at certain kinds of checks. They can inspect code, identify programmatically detectable issues and repeat the same tests after every change.
The important part is understanding where that reliability ends
A scanner may identify that an image has no alternative text. If alternative text is present, another question remains: does it communicate the right information in that context? It may also find that all required controls exist, but that does not necessarily tell us whether someone can move through them in a logical order using only a keyboard.
This is why I think of automated results as evidence rather than a verdict. They tell us what the tool was able to detect, while human judgment helps us understand what that means for the person using the page.
The mistake is not using a scanner. The mistake is expecting it to answer questions it was never able to answer.
The difference between detecting an issue and experiencing a barrier
Finding an accessibility issue is only part of the story. The technical defect tells us what is wrong. The barrier tells us what that problem prevents someone from doing.
A button that cannot be reached with a keyboard may look like one small issue in a long test report. But if that button is the only way to submit an order, book an appointment or send an important form, its impact is much bigger than the size of the defect might suggest.
The same issue can also have very different consequences depending on where it appears. An unclear label on an optional filter may be inconvenient. An unclear label on a field required to complete a payment can block the entire process.
That is why accessibility findings should not be evaluated only by counting errors or looking at the technical effort required to fix them. What matters is the effect on the person using the product: Does the issue make something slightly harder, significantly more difficult, or completely impossible?
This is where human testing adds another layer to the result. It connects the technical finding to the real task behind it.
Because users do not experience accessibility issues as entries in a report. They experience them as barriers between themselves and what they are trying to do.
AI changes the tools, not the need for human judgment
AI makes this conversation more interesting. Newer tools can analyse language, interpret code, suggest fixes and work with context in ways that traditional rule-based scanners often cannot. That can reduce repetitive work and help testers investigate issues faster.
But “more capable” does not mean “able to make every accessibility decision”.
Alternative text is a simple example. AI can describe an image in seconds, and that description may be accurate. But useful alternative text depends on why the image is there. The same image can communicate important information in one context, be decorative in another or function as part of a link somewhere else. Knowing what is visible is not always the same as knowing what matters.
The same is true when AI suggests a fix. A technically reasonable change can still leave the overall journey confusing. Someone still has to verify the result and check whether the barrier has actually been removed.
That is why I do not see AI as a replacement for accessibility testers. I see it as another tool that can make us faster and help us investigate more.
My rule is simple: use the tools. Just know what they can and cannot tell you.
When human testing is not the opposite of automation
It is tempting to turn this into a competition: automated testing or human testing, AI or people. I do not think that framing helps.
Automation gives us speed, repeatability and scale. Human testing gives us context, interpretation and the ability to follow a real journey from beginning to end. Testing with assistive technologies can reveal interactions that are easy to miss, while involving disabled users brings lived experience that neither a scanner nor a tester without that experience can fully reproduce.
Those strengths complement each other. When a tool reports no errors, a tester can still check the journeys that matter most.
For me, that is also where the tester’s role becomes more interesting rather than less important. If tools become better at finding patterns, testers can spend more time interpreting impact. What does this issue prevent someone from doing? How serious is the barrier? Did the fix solve the actual problem, or did it simply make the automated check turn green?
For me, that is also where the tester’s role becomes more interesting rather than less important. If tools become better at finding patterns, testers can spend more time interpreting impact. What does this issue prevent someone from doing? How serious is the barrier? Did the fix solve the actual problem, or did it simply make the automated check turn green?
A green checkmark is useful. It is just not the happy ending.
From finding barriers to preventing them
Once we stop treating a clean scanner report as the finish line, the next question is how to prevent more accessibility problems from reaching the finished product in the first place.
That is where Shift Left becomes important. Accessibility can be considered while requirements are written, designs are reviewed, content is created and components are developed. Keyboard operation can be part of acceptance criteria. A contrast issue can be corrected before the same component is reused across many pages. An unclear error message can be improved while the content is still being written.
This does not remove the need to test the finished experience. Individual components can look correct in isolation and still create a broken journey when combined. Early checks and complete end-to-end testing solve different parts of the same problem.
The same applies after release. Websites and applications keep changing, so accessibility cannot realistically be treated as a one-time project. Continuous testing helps teams detect regressions instead of rediscovering the same barriers later.
At nextpertis, we have developed a shift-left accessibility strategy that brings accessibility testing into regular testing cycles. This makes accessibility part of the quality definition for a digital product or service and helps teams detect issues earlier, when they are easier and less costly to fix. We also apply continuous accessibility testing to help prevent regressions over time.
The point is not to trust accessibility scanners less. It is to understand what they are good at, combine them with human judgment and use both throughout the product lifecycle. Tools can help us find and prevent more barriers, but the reason we test in the first place is still human.
To learn more about how nextpertis integrates accessibility into regular QA processes, contact us for a free initial consultation.
