The Challenge-Response Authentication Flow in WebAuthn

The challenge-response flow is the cryptographic core of every WebAuthn authentication ceremony. The Relying Party backend generates a high-entropy, single-use nonce; the authenticator signs it with a device-bound private key; and the server verifies the signature — all without the private key ever leaving the authenticator boundary. This page provides spec-grounded, implementation-ready guidance for every stage of the assertion pipeline, from challenge issuance through sign counter validation. For the broader protocol context this flow operates within, see WebAuthn & FIDO2 Protocol Fundamentals.

---

Concept Definition and Spec Grounding

The assertion ceremony is defined in W3C WebAuthn Level 3 §7.2 (“Verifying an Authentication Assertion”). It is distinct from the attestation ceremony (§7.1), which occurs during credential registration. The key data structures involved are:

PublicKeyCredentialRequestOptions — the options object passed to navigator.credentials.get(), containing:

• challenge: a BufferSource of at least 16 bytes (32 bytes recommended), never reused
• rpId: the effective domain of the RP, validated by the browser against the page origin
• allowCredentials: optional list of PublicKeyCredentialDescriptor entries restricting which credentials may respond
• userVerification: 'required', 'preferred', or 'discouraged'; use 'required' for AAL2/AAL3 flows

authenticatorData byte layout (WebAuthn §6.1):

COSE key format (RFC 8152): public keys are stored as CBOR maps. For ES256 (COSE algorithm -7): key type kty = 2 (EC2), curve crv = 1 (P-256), coordinates x and y. For RS256 (COSE algorithm -257): kty = 3 (RSA), modulus n, exponent e.

---

Architecture and Data Flow

The assertion handshake spans three actors: the RP backend, the browser (client), and the authenticator (platform or roaming). The browser enforces rpId binding and user gesture requirements; it never exposes private key material to JavaScript.

The browser’s rpId enforcement is described in Understanding WebAuthn vs FIDO2 Architecture: it validates the requesting origin against the rpId before forwarding the request to the OS credential manager or a roaming CTAP2 key.

---

Implementation Guide

Step 1 — Generate and store the challenge (RP backend)

Spec requirement: W3C WebAuthn §13.4.3 — challenge must be at least 16 bytes from a cryptographically secure random source; must be single-use and bound to the authenticated session.

import { randomBytes } from 'crypto';
import { createClient } from 'redis';

const redis = createClient({ url: process.env.REDIS_URL });

export async function issueChallenge(sessionId: string): Promise<string> {
  const challengeBuffer = randomBytes(32);                    // 256-bit entropy
  const challengeBase64url = challengeBuffer.toString('base64url');

  // NX prevents overwriting an in-progress challenge; EX = 120-second TTL
  const stored = await redis.set(
    `webauthn:challenge:${sessionId}`,
    challengeBase64url,
    { EX: 120, NX: true }
  );

  if (!stored) throw new Error('Active challenge already exists for this session');
  return challengeBase64url;
}

Never derive challenges from timestamps, user IDs, or other deterministic seeds. Predictable challenges enable precomputation attacks against the assertion signature.

Step 2 — Invoke navigator.credentials.get() (client)

Spec requirement: W3C WebAuthn §5.1.3 — options must include the challenge as a BufferSource; rpId must be a registrable domain suffix of the effective domain.

async function authenticate(
  challengeBase64url: string,
  allowCredentials: Array<{ id: string; type: 'public-key' }>
): Promise<PublicKeyCredential> {
  function base64urlToBuffer(b64: string): Uint8Array {
    const binary = atob(b64.replace(/-/g, '+').replace(/_/g, '/'));
    return Uint8Array.from(binary, c => c.charCodeAt(0));
  }

  const credential = await navigator.credentials.get({
    publicKey: {
      challenge: base64urlToBuffer(challengeBase64url),
      rpId: window.location.hostname,
      allowCredentials: allowCredentials.map(c => ({
        type: 'public-key' as const,
        id: base64urlToBuffer(c.id)
      })),
      userVerification: 'required'        // enforce UV for AAL2/AAL3
    }
  });

  if (!credential) throw new Error('No credential returned');
  return credential as PublicKeyCredential;
}

Omitting allowCredentials triggers a discoverable credential prompt (resident key flow). Populate it for server-side credential lookups to prevent account enumeration through timing differences.

Step 3 — Parse clientDataJSON (RP backend)

Spec requirement: W3C WebAuthn §7.2, step 11 — verify type, challenge, origin, and optionally topOrigin.

function parseAndVerifyClientData(
  clientDataJSONBase64url: string,
  expectedChallenge: string,
  expectedOrigin: string
): void {
  const buf = Buffer.from(clientDataJSONBase64url, 'base64url');
  const clientData = JSON.parse(buf.toString('utf8'));

  if (clientData.type !== 'webauthn.get') {
    throw new Error(`Invalid type: expected webauthn.get, got ${clientData.type}`);
  }
  if (clientData.origin !== expectedOrigin) {
    throw new Error(`Origin mismatch: ${clientData.origin} !== ${expectedOrigin}`);
  }
  // Timing-safe comparison prevents oracle attacks on the challenge
  if (!timingSafeEqual(
    Buffer.from(clientData.challenge),
    Buffer.from(expectedChallenge)
  )) {
    throw new Error('Challenge mismatch');
  }
}

Step 4 — Parse authenticatorData byte layout

Spec requirement: W3C WebAuthn §6.1 — byte layout is fixed; offset 32 is always the flags byte regardless of whether attested credential data is present.

interface ParsedAuthData {
  rpIdHash: Buffer;
  flags: number;
  userPresent: boolean;
  userVerified: boolean;
  signCount: number;
}

function parseAuthenticatorData(authDataBase64url: string): ParsedAuthData {
  const authData = Buffer.from(authDataBase64url, 'base64url');

  if (authData.length < 37) throw new Error('authenticatorData too short');

  const rpIdHash   = authData.subarray(0, 32);     // bytes  0–31
  const flags      = authData[32];                  // byte  32
  const signCount  = authData.readUInt32BE(33);     // bytes 33–36 big-endian

  return {
    rpIdHash,
    flags,
    userPresent:  (flags & 0x01) !== 0,            // bit 0 — UP
    userVerified: (flags & 0x04) !== 0,            // bit 2 — UV
    signCount
  };
}

The flags byte at offset 32 is a fixed position — it is not affected by whether optional attested credential data (AT flag, bit 6) is appended after byte 37 during registration assertions.

Step 5 — Validate rpIdHash and flags

Spec requirement: W3C WebAuthn §7.2, steps 17-20 — rpIdHash must equal SHA-256 of the stored rpId; UP must be set; UV must be set when userVerification was 'required'.

import { createHash, timingSafeEqual } from 'crypto';

function validateRpIdAndFlags(
  parsed: ParsedAuthData,
  expectedRpId: string,
  requireUserVerification: boolean
): void {
  const expectedHash = createHash('sha256').update(expectedRpId).digest();
  if (!timingSafeEqual(parsed.rpIdHash, expectedHash)) {
    throw new Error('rpIdHash mismatch — possible cross-origin credential theft attempt');
  }
  if (!parsed.userPresent) {
    throw new Error('UP flag not set — user presence not confirmed');
  }
  if (requireUserVerification && !parsed.userVerified) {
    throw new Error('UV flag not set — user verification required but not performed');
  }
}

Step 6 — Verify the assertion signature

Spec requirement: W3C WebAuthn §7.2, step 21 — signed data is authenticatorData || SHA-256(clientDataJSON); algorithm is determined by the COSE algorithm identifier (alg) stored at registration.

import { createVerify } from 'crypto';

function verifySignature(
  authDataBuffer: Buffer,
  clientDataJSONBase64url: string,
  signatureBase64url: string,
  publicKeySpki: Buffer,   // DER-encoded SubjectPublicKeyInfo from registration
  coseAlg: number          // -7 = ES256, -257 = RS256, -8 = EdDSA
): void {
  const clientDataBuffer = Buffer.from(clientDataJSONBase64url, 'base64url');
  const clientDataHash   = createHash('sha256').update(clientDataBuffer).digest();
  const signedData       = Buffer.concat([authDataBuffer, clientDataHash]);
  const signature        = Buffer.from(signatureBase64url, 'base64url');

  const hashAlg = coseAlg === -7   ? 'SHA256'        // ES256 / ECDSA P-256
                : coseAlg === -257 ? 'SHA256'        // RS256 / RSA-PKCS1v15
                : coseAlg === -37  ? 'SHA256'        // PS256 / RSA-PSS
                : null;

  if (!hashAlg) throw new Error(`Unsupported COSE algorithm: ${coseAlg}`);

  const keyOptions = coseAlg === -37
    ? { key: publicKeySpki, padding: 6 /* RSA_PKCS1_PSS_PADDING */ }
    : { key: publicKeySpki, dsaEncoding: 'der' };

  const verifier = createVerify(hashAlg);
  verifier.update(signedData);
  if (!verifier.verify(keyOptions, signature)) {
    throw new Error('Assertion signature verification failed');
  }
}

In production, prefer @simplewebauthn/server or fido2-lib over raw node:crypto — they handle COSE parsing, algorithm negotiation, and edge cases in the EC2 coordinate encoding that are easy to mis-implement.

Step 7 — Validate the sign counter and issue session

Spec requirement: W3C WebAuthn §7.2, step 23 — if storedSignCount is non-zero, the new count must be strictly greater; a match or decrease indicates a possible cloned authenticator.

async function finalizeAssertion(
  credentialId: string,
  newSignCount: number,
  db: DatabaseClient
): Promise<void> {
  const stored = await db.credential.findUnique({ where: { credentialId } });
  if (!stored) throw new Error('Credential not found');

  if (stored.signCount > 0 && newSignCount <= stored.signCount) {
    // Log as a security event; consider disabling the credential
    await logSecurityEvent({
      type: 'SIGN_COUNTER_ANOMALY',
      credentialId,
      storedCount: stored.signCount,
      receivedCount: newSignCount
    });
    throw new Error('signCount did not increase — possible cloned authenticator');
  }

  await db.credential.update({
    where: { credentialId },
    data: { signCount: newSignCount, lastUsedAt: new Date() }
  });
}

Platform authenticators that back up credentials to iCloud Keychain or Google Password Manager may return signCount = 0 for all assertions — this is per-spec for synced passkeys. If storedSignCount === 0 and newSignCount === 0, skip the counter check rather than rejecting.

---

Validation Checklist

• Challenge is 32 bytes from crypto.randomBytes() (never from Math.random() or timestamp-derived seeds)
• Challenge is stored with a 120-second TTL, bound to sessionId, and deleted immediately after first verification
• clientDataJSON.type === 'webauthn.get' (not webauthn.create)
• clientDataJSON.origin exactly matches the expected RP origin (scheme + host + port)
• clientDataJSON.challenge is compared with timingSafeEqual to prevent timing oracles
• rpIdHash (bytes 0-31) is compared with timingSafeEqual against SHA-256(rpId)
• Flags byte is read from offset 32, not offset 37
• UP flag (bit 0) is always required; UV flag (bit 2) required when userVerification: 'required'
• signCount checked for strict increase when storedSignCount > 0; synced passkeys returning 0 are handled separately
• Signature verified with the COSE algorithm from registration (alg field), not hardcoded ES256
• Challenge deleted from store on both success and failure to prevent retry attacks

---

Error Reference Table

---

Platform and Library Notes

@simplewebauthn/server (Node.js) verifyAuthenticationResponse() handles the full §7.2 verification pipeline, including COSE key parsing, DER signature normalisation, and sign counter logic. Pass requireUserVerification: true to mirror the userVerification: 'required' policy set on the client. The library treats signCount === 0 on both sides as a valid synced passkey and skips the counter check.

fido2-lib (Node.js) Requires manual construction of AssertionExpectations; call f2l.assertionResult(assertion, expectations). The library validates the challenge as a ArrayBuffer comparison — ensure you pass the raw bytes, not the base64url string.

py_webauthn (Python) verify_authentication_response() accepts base64url strings directly. Set require_user_verification=True for high-assurance flows. The library raises InvalidAuthenticationResponse with a descriptive message for each validation failure, making it straightforward to map to HTTP status codes.

WebAuthn4J (Java/Spring) WebAuthnManager.validate() performs full §7.2 verification. Wire UserVerificationRequirement.REQUIRED into the AuthenticationParameters. The library’s SignCountChecker raises SignCountException on counter anomalies — catch this specifically to distinguish cloned-authenticator alerts from other verification failures.

iOS Safari NotAllowedError fires if the WebAuthn API is called from a cross-origin <iframe>. The credential selection UI is a modal sheet; users cannot dismiss it programmatically. iCloud Keychain synced passkeys always return signCount = 0.

Android Chrome Falls back to device PIN if biometrics are unavailable. Android 9+ supports CTAP2 over hybrid transport (QR code / caBLE) for cross-device authentication. signCount increments on hardware-backed credentials (Titan M chip); Google Password Manager synced passkeys may return 0.

Windows Hello TPM-backed credentials increment signCount monotonically. The Windows WebAuthn API returns InvalidStateError if the user cancels the Windows Hello prompt, rather than NotAllowedError — handle both in your error map.

---

Pitfalls and Security Hardening

1. Reading flags from the wrong byte offset The flags byte is at offset 32 of authenticatorData. Offset 37 is the start of attested credential data (only present during registration when the AT flag is set). Reading from offset 37 causes all UP and UV flag checks to silently pass or fail with wrong values. Always use authData[32].

2. Non-timing-safe challenge comparison Using === or Buffer.equals() to compare the challenge or rpIdHash exposes a timing oracle. An attacker can measure response-time differences to infer matching bytes. Use crypto.timingSafeEqual() for all security-critical byte comparisons.

3. Failing to delete the challenge after first use If the server verifies the challenge but does not delete it from the store, an attacker who intercepts the assertion can replay it within the TTL window. Delete the challenge immediately on both successful and failed verification — the failure case is equally important.

4. Hardcoding ES256 as the verification algorithm Storing only the public key bytes without the COSE alg field means the server cannot reliably select the verification algorithm. A credential registered with RS256 will fail signature verification if ES256 is assumed. Store alg at registration and pass it explicitly to the verification function.

5. Rejecting synced passkeys due to signCount === 0 Passkeys synced via iCloud Keychain or Google Password Manager legitimately return signCount = 0 for every assertion, because the counter cannot be kept in sync across devices. Treat storedSignCount === 0 && newSignCount === 0 as valid; only raise an anomaly when storedSignCount > 0 and newSignCount <= storedSignCount.

6. Missing exponential backoff on NotAllowedError Retrying assertion immediately after a NotAllowedError (user cancellation or timeout) consumes a new challenge and may lock users out if the server enforces attempt limits. Implement exponential backoff starting at 1 second, capped at 30 seconds, and surface a clear user-facing message distinguishing cancellation from device unavailability.

---

Related

• Step-by-Step Breakdown of the FIDO2 Registration Flow — the enrollment ceremony that creates the key pair this flow authenticates against
• Attestation vs Assertion Explained — spec-level distinction between the registration and authentication ceremonies, including clientDataJSON.type values
• Relying Party and Authenticator Roles — trust model partitioning, rpId scope, and CTAP2 transport negotiation
• Cryptographic Algorithms Supported by WebAuthn — COSE algorithm identifiers, ES256 vs RS256 vs EdDSA selection, and key storage formats
• Implementing Authentication Verification Logic — server-side verification pipeline, including best practices for FIDO2 challenge generation and Node.js signature verification