Passkey Testing and Conformance

Passkey code is unusually hard to test well: the interesting behaviour happens inside a browser dialog you cannot inspect, the failure modes are byte-level, and the errors are deliberately uninformative. This page covers the four layers of testing a production deployment needs, what each one actually catches, and where the FIDO conformance tooling fits. It sits under Backend Verification and Secure Credential Storage, and most of the specific checks it describes are detailed in the debugging and observability material.

Concept Definition and Spec Grounding

There is no single “test the passkey flow” step, because a passkey flow is four systems interacting: your server’s verification algorithm, the browser’s credential API, the operating system’s credential surface, and the authenticator itself. A test that exercises all four at once is an end-to-end test with a human in it, and it does not scale.

What does scale is layering. The verification algorithm is a pure function of a payload and a set of expectations, so it can be tested exhaustively with fixtures. The ceremony lifecycle is browser behaviour, so it can be driven with a virtual authenticator. Spec conformance is a defined suite published by the FIDO Alliance. Hardware behaviour is the residue, and it is small enough to cover manually.

The layering matters because the layers catch disjoint problems. A unit test of the decoder will never notice that your page arms two conditional ceremonies. A browser test will never notice that your DER-to-raw signature conversion strips a leading zero. Teams that pick one layer and invest heavily in it end up with a suite that is thorough about a quarter of the risk.

Four layers, four kinds of test Each layer of a passkey deployment fails differently, so each needs its own kind of coverage. No single layer catches more than a quarter of what goes wrong unit — decode and verify byte-level bugs: offsets, encodings, algorithm dispatch fixture — golden payloads regressions against real authenticator output browser — virtual authenticator ceremony lifecycle, error handling, UI state conformance — FIDO tooling spec compliance you would otherwise discover in certification

Architecture and Data Flow

The practical arrangement is a small pyramid with an unusual base. At the bottom sit fixture-driven unit tests over the verification path — fast, numerous, and grounded in payloads captured from real authenticators rather than generated by the code under test. Above them sit browser-driven tests using a virtual authenticator, covering the ceremony lifecycle and the interface states around it. Above those sits a thin layer of conformance runs, executed before a release rather than on every commit. Manual device testing sits outside the pyramid entirely, as a release checklist rather than as automation.

The base is unusual because the fixtures are the expensive part to create and the cheap part to run. Capturing a registration and an assertion from a real device takes a few minutes once; the resulting assertions run in milliseconds forever. Every encoding boundary in your stack — the base64url helper, the database column type, the serialiser, the proxy — is covered by a single test that hashes the payload at ingress and again at verification.

Where a golden fixture pays for itself One captured registration and assertion, stored with the expectations that verified them, guards every future refactor. The single highest-value artefact in a passkey test suite capture one real ceremony store payload plus expectations assert it still verifies protect every later refactor Synthetic payloads generated by your own code test the code against itself; a captured payload tests it against reality.

Implementation Guide

Step 1 — capture golden fixtures per algorithm. One registration and one assertion from a real authenticator for each algorithm you accept, stored with the challenge, origin, RP ID and public key that made them verify. Without the expectations, the payload is unusable.

Step 2 — assert the round trip, not just the parse. A decoder that does not throw proves nothing. The assertion that matters is that a captured assertion verifies against the stored key, end to end.

Step 3 — add the negative fixtures. An altered byte in authenticatorData; a signature from a different credential; an expired challenge; an origin not on the allow-list. Each should fail at a specific, named reason code rather than at a generic exception.

Step 4 — drive the browser with a virtual authenticator. Add one, configure its capabilities, run the ceremony, and assert both the happy path and the three interesting failures: refusal, no matching credential, and a ceremony that never answers.

Step 5 — test what remains usable after a failure. The most common shipped defect is a page that cannot recover: the passkey button stays disabled, or the password form has been hidden. Assert the idle state, not just the error.

Step 6 — run the conformance tools before a release. They exercise spec requirements your own tests will not think of, particularly around attestation formats and error handling, and they are the difference between believing you are compliant and knowing it.

Step 7 — keep a short manual checklist. One real platform authenticator, one hardware key, one cross-device flow. Ten minutes, once per release, covering everything automation cannot reach.

Validation Checklist

What a virtual authenticator can and cannot simulate It covers ceremony behaviour completely and hardware behaviour not at all. Complete for logic, silent on hardware it can simulate user verification present or absent resident-key support or its absence a ceremony that never answers a credential that does not exist multiple authenticators at once it cannot simulate real biometric prompts hardware storage exhaustion cross-device Bluetooth proximity vendor firmware quirks platform sync behaviour Which is why a small amount of manual testing on real devices still belongs in a release checklist.

Error Reference Table

Symptom in tests Likely cause Layer that should have caught it
Fixture verifies, production does not an encoding boundary in the request path fixture test with ingress/egress digests
Browser test passes, users report failures hardware or platform behaviour manual device checklist
Signature fails only for one algorithm dispatch or conversion bug per-algorithm fixtures
Ceremony hangs in tests virtual authenticator configured not to answer that is the intended behaviour — assert the timeout path
Conformance run fails on attestation a format you accept but do not fully verify conformance tooling

Platform and Library Notes

Browser automation frameworks expose the virtual authenticator through the underlying DevTools protocol, and the ergonomics differ substantially. Some expose a first-class API for adding an authenticator and configuring its capabilities; others require dropping to the raw protocol. Either works, and the capability set is the same underneath: attachment, transport, resident-key support, user-verification support, and whether user verification is automatically satisfied.

Server-side libraries usually ship their own test vectors, and those are worth running, but they are not a substitute for fixtures captured from your own stack. A library’s vectors prove the library is correct; your fixtures prove your integration is.

The FIDO conformance tooling is a separate download with its own workflow and is oriented towards certification rather than continuous integration. Treat it as a pre-release gate rather than something to wire into every pull request.

Pitfalls and Security Hardening

1. Generating test payloads with the code under test. The classic circular test: your signer and your verifier agree on a wrong interpretation, and every test passes.

2. Testing only the happy path. The failure paths are where the user-visible defects live, and they are the paths nobody exercises by hand.

3. Mocking the credential API. A mock returns what you told it to return, which means it cannot surprise you — and surprise is the entire point of testing this layer.

4. Skipping the no-JavaScript test. It takes one assertion and it protects the population most at risk from a passkey rollout.

5. Letting fixtures rot. A fixture whose expectations were edited to make a test pass has stopped being evidence. If a golden fixture starts failing, the change is what needs explaining.

6. Treating conformance as a one-off. It is run at certification and then forgotten, by which time the implementation has moved. A pre-release run costs little and catches drift.

Frequently Asked Questions

Can I test WebAuthn without a browser at all?

The verification half, yes — it is a pure function over bytes, and fixtures cover it completely. The ceremony half, no: the behaviour you need to test is the browser’s, and simulating it means writing a model of the browser that will agree with your assumptions rather than with the browser.

How many fixtures are enough?

One registration and one assertion per accepted algorithm, plus one negative fixture per verification reason code. That is typically eight to twelve payloads, and it covers more real risk than any number of generated cases.

Should tests run against a real database?

For the encoding-boundary tests, yes. The whole point is that the bytes survive the round trip through your actual column type and driver, and an in-memory substitute has different behaviour precisely where the bugs are.

Do I need conformance testing if I am not seeking certification?

Not strictly, but it is the cheapest external review available. The suite encodes spec requirements that are easy to overlook, and running it once will usually surface at least one check nobody had implemented.

What should a release checklist contain?

Sign in with a platform authenticator, sign in with a hardware key, complete one cross-device flow, and complete one enrolment on a device that has never seen the account. Four scenarios, ten minutes, and between them they cover everything the automation is blind to.

How do I test the cross-device flow?

Manually, and only manually. The Bluetooth proximity check and the relay tunnel are not simulatable, which is exactly why the flow needs a place on the manual checklist rather than an entry in the suite.

Capturing Fixtures Without Creating a Data Problem

The awkward part of fixture-driven testing is that the payloads are real, which means they came from a real device belonging to a real person — even if that person is you.

Capture them from a dedicated test account on a device you control, never from production traffic. A payload harvested from a live user carries their credential identifier and user handle, and those are personal data regardless of how obviously non-secret they are. A test account sidesteps the question entirely and costs nothing.

Store the fixtures in the repository alongside the code they protect. They are small — a few kilobytes each — and keeping them next to the decoder means a reader can see immediately what real inputs the implementation has been proven against. Storing them in a separate artefact store guarantees that within a year nobody will know which fixture corresponds to which behaviour.

Record the provenance of each one in a short comment: which authenticator produced it, which algorithm was negotiated, and what it demonstrates. When a fixture starts failing after a library upgrade, that comment is the difference between a five-minute diagnosis and an argument about whether the fixture was ever right.

Refresh them occasionally. Authenticator behaviour changes with firmware and platform releases, and a fixture set captured three years ago represents a population that no longer exists. Adding a new capture each time you meet unusual hardware builds a corpus that reflects your actual users rather than the state of the ecosystem when the project started.

One thing not to do: never edit a fixture to make a test pass. A golden payload that has been adjusted has stopped being evidence and become an assertion of what you believe. If a fixture fails, either the code regressed or the expectation was wrong, and both cases deserve an explanation rather than an edit.

Deciding What Not to Automate

An honest test strategy names what it does not cover, because the alternative is a suite that looks comprehensive and is silently blind.

Biometric prompts cannot be automated. A virtual authenticator can report that user verification succeeded, but nothing in the automation stack presses a fingerprint sensor, and the interface around that prompt — the timing, the copy, whether the user understands what is being asked — is untested by definition.

Hardware storage exhaustion cannot be automated. A virtual authenticator has no slot limit, so the full-key path exists only in production unless somebody deliberately fills a real key and tries it.

Cross-device flows cannot be automated. The Bluetooth proximity check is the security property, and simulating it would mean removing it.

Platform synchronisation cannot be automated. Whether a credential appears on a second device is a function of the user’s platform account, and no test harness has one.

Each of these belongs on a manual checklist rather than in a backlog of automation that will never be written. Four scenarios, ten minutes, once per release, is a proportionate answer — and writing them down converts an unknown gap into a known one, which is the whole point.

The corollary is that anything outside those four categories should be automated. A team that finds itself manually testing the same verification failure repeatedly has a missing fixture, not a hard-to-test system.

Testing the Failure Paths Users Actually Meet

Most passkey test suites cover registration and authentication succeeding, and stop. The defects that reach production almost all live in the paths after something goes wrong, because those are the paths nobody exercises by hand during development.

The refused ceremony. A user opens the prompt and dismisses it. The page must return to a state where both sign-in paths work without a reload, the passkey button must be clickable again, and nothing may have been logged as an error requiring investigation. Assert all three; the second is the one that regresses when somebody adds a loading state.

The ceremony with no matching credential. A user with no passkey for your relying party triggers the flow. The browser shows a prompt they cannot satisfy and eventually rejects. From your side this is indistinguishable from a cancellation, which is the point — assert that your code does not attempt to distinguish them, because a heuristic that leaks into security logic is worse than no heuristic.

The abandoned ceremony. Configure the virtual authenticator not to answer at all. Assert that your interface eventually recovers, that the server challenge is either still valid or cleaned up according to your documented policy, and that the test does not need to wait out a real multi-minute window to prove it.

The duplicate enrolment. Two registrations against one authenticator, the second correctly refused. This single test covers exclude-list construction, transmission and error handling together, and it is the one that catches a scoped query being loosened during a refactor.

The revoked credential. Revoke, then present the same credential. The lookup must fail closed with a generic response, the attempt must be recorded, and no distinguishing information may reach the client.

The stripped-script page. Load the sign-in page with JavaScript disabled and complete a password sign-in. One assertion, and it protects every user whose bundle failed to arrive.

Six scenarios, all automatable, and between them they cover the overwhelming majority of what users actually report. A suite that has these plus the happy path is in better shape than one with three times the assertions concentrated on success.

Related