Relying Party and Authenticator Roles in WebAuthn

Modern passwordless identity rests on a strict cryptographic separation between two actors: the Relying Party (RP) that owns the service and the Authenticator that holds the private key. Neither actor trusts the other implicitly — the browser acts as a mediator, and the W3C WebAuthn specification (Level 3) enforces this separation at the protocol level. Understanding where each entity’s authority begins and ends is the prerequisite for building production-safe passkey registration and authentication endpoints. For a broader treatment of the spec hierarchy, see WebAuthn & FIDO2 Protocol Fundamentals.

---

Concept Definition and Spec Grounding

The WebAuthn Level 3 specification (W3C, §4 Terminology) defines these three entities precisely:

Relying Party — The web application that calls the WebAuthn API and ultimately accepts or rejects credentials. The RP is identified by its RP ID, which must be the effective domain of the origin or a registrable domain suffix of it (§13.4.9). The RP generates challenges, stores public keys, and verifies assertions. It never touches private key material.

Authenticator — A cryptographic module that generates asymmetric key pairs, stores them in a secure boundary, and signs assertions on behalf of the user. The spec (§6.2.2) mandates that private keys must not leave the authenticator boundary. Authenticators divide into two physical categories — platform (built into the device: Secure Enclave on iOS/macOS, TPM on Windows Hello, Android StrongBox) and roaming (external tokens communicating via CTAP2 over USB HID, NFC, or BLE).

Client — The browser or OS layer that implements the navigator.credentials API surface, enforces origin binding, translates between the WebAuthn JS API and the CTAP2 transport protocol, and routes clientDataJSON to both parties.

authenticatorData byte layout (§6.1)

Every authenticator response includes a 37-byte-minimum authenticatorData blob. Knowing its layout is essential for flag validation:

The UP (User Presence) flag at bit 0 must be set in every valid assertion. The UV (User Verification) flag at bit 2 must be set when userVerification: 'required' is configured. Failing to enforce these bits is one of the most common phishing-resistance failures in production deployments.

---

Architecture and Data Flow

The diagram below shows the full message sequence for credential registration, including which messages cross which trust boundaries.

The browser enforces origin binding at the left boundary; the authenticator’s hardware secure element enforces key isolation at the right boundary. The RP server never communicates directly with the authenticator.

---

Implementation Guide

Step 1 — Derive and validate RP ID (W3C WebAuthn §13.4.9)

The RP ID must be the hostname or a registrable domain suffix of the page origin. Using window.location.hostname directly is safe for most deployments; for multi-tenant SaaS, compute the effective registrable domain:

// Safe RP ID derivation — strips subdomains to the registrable domain
export function deriveRpId(origin: string): string {
  const { hostname } = new URL(origin);
  const parts = hostname.split('.');
  // e.g. app.example.com → example.com
  return parts.length > 2 ? parts.slice(-2).join('.') : hostname;
}

Mismatching rp.id and the page origin at registration time causes SecurityError at authentication time — the stored rpIdHash in authenticatorData will not match the hash of the origin the browser computes at assertion.

Step 2 — Generate a cryptographically secure challenge (§7.1 step 4)

The challenge prevents replay attacks. The server must generate it, not the client. Bind it to the authenticated session and enforce a strict TTL. For the full challenge-response cycle, see the challenge-response authentication flow.

import crypto from 'crypto';
import { db } from './db';

export async function beginRegistration(sessionId: string, userId: string) {
  const challenge = crypto.randomBytes(32);
  const expiresAt = new Date(Date.now() + 300_000); // 5-minute TTL

  await db.query(
    `INSERT INTO pending_challenges (session_id, user_id, challenge, expires_at)
     VALUES ($1, $2, $3, $4)
     ON CONFLICT (session_id) DO UPDATE
       SET challenge = EXCLUDED.challenge, expires_at = EXCLUDED.expires_at`,
    [sessionId, userId, challenge.toString('base64url'), expiresAt]
  );

  return {
    challenge: challenge.toString('base64url'),
    rp: { id: process.env.RP_ID!, name: process.env.RP_NAME! },
    user: { id: Buffer.from(userId).toString('base64url'), name: '[email protected]', displayName: 'User' },
    pubKeyCredParams: [
      { alg: -7,   type: 'public-key' as const }, // ES256
      { alg: -257, type: 'public-key' as const }  // RS256
    ],
    authenticatorSelection: {
      residentKey: 'required',
      userVerification: 'required'
    },
    attestation: 'none' as const
  };
}

Step 3 — Select authenticator attachment and residentKey policy (§5.4.4, §5.4.6)

The authenticatorSelection object determines which physical category of authenticator the browser will prompt for and whether the credential must be discoverable (stored on the authenticator, enabling username-less flows).

const authenticatorSelection: AuthenticatorSelectionCriteria = {
  // 'platform'       → Touch ID / Face ID / Windows Hello
  // 'cross-platform' → YubiKey / security key via CTAP2
  // omit             → user's choice (recommended for broadest compatibility)
  authenticatorAttachment: 'platform',

  // 'required' → credential stored on device (passkey flow)
  // 'preferred' → store if supported, fall back otherwise
  // 'discouraged' → server-side credential (traditional FIDO2 key)
  residentKey: 'required',

  // 'required' → biometric / PIN mandatory; UP alone insufficient
  userVerification: 'required'
};

Step 4 — Parse authenticatorData flags on the server (§6.1)

After the browser POSTs the attestationObject, your server must decode the CBOR, extract authenticatorData, and verify both the rpIdHash and the flag byte at offset 32.

import { decode as cborDecode } from 'cbor';
import crypto from 'crypto';

export function verifyAuthenticatorDataFlags(
  authenticatorData: Buffer,
  rpId: string,
  requireUV: boolean
): void {
  const rpIdHash = authenticatorData.subarray(0, 32);
  const expected = crypto.createHash('sha256').update(rpId).digest();

  if (!rpIdHash.equals(expected)) {
    throw new Error('rpIdHash mismatch — origin binding violation');
  }

  const flags = authenticatorData[32];
  const UP = (flags & 0x01) !== 0; // bit 0
  const UV = (flags & 0x04) !== 0; // bit 2

  if (!UP) throw new Error('UP flag not set — user presence not confirmed');
  if (requireUV && !UV) throw new Error('UV flag not set — user verification required');
}

Step 5 — Validate signCount monotonicity (§7.2 step 17)

The authenticator increments signCount on every assertion. A non-increasing counter signals a cloned authenticator or a replayed assertion — except for synced passkeys, which are allowed to report 0.

export function validateSignCount(
  incoming: number,
  stored: number,
  credentialId: string,
  logger: { warn: (msg: object) => void }
): boolean {
  if (stored === 0 && incoming === 0) {
    // Synced passkeys legitimately report zero — allow but do not store
    return true;
  }
  if (incoming <= stored) {
    logger.warn({
      event: 'sign_count_regression',
      credentialId,
      incoming,
      stored,
      action: 'reject — possible clone or replay'
    });
    return false;
  }
  return true;
}

Step 6 — Store the public key and credential metadata (§6.5.3)

The RP stores only the public key, never private material. For full schema options and indexing strategies, see public key vs symmetric credential types.

CREATE TABLE webauthn_credentials (
  id              UUID        PRIMARY KEY DEFAULT gen_random_uuid(),
  credential_id   BYTEA       UNIQUE NOT NULL,
  user_id         UUID        NOT NULL REFERENCES users(id) ON DELETE CASCADE,
  public_key      BYTEA       NOT NULL,   -- COSE-encoded public key (raw bytes)
  cose_alg        INTEGER     NOT NULL,   -- -7 = ES256, -257 = RS256
  sign_count      BIGINT      NOT NULL DEFAULT 0,
  transports      TEXT[]      NOT NULL DEFAULT '{}',
  backup_eligible BOOLEAN     NOT NULL DEFAULT FALSE, -- BE flag
  backup_state    BOOLEAN     NOT NULL DEFAULT FALSE, -- BS flag
  aaguid          UUID,                              -- from attestedCredentialData
  created_at      TIMESTAMPTZ NOT NULL DEFAULT NOW(),
  last_used_at    TIMESTAMPTZ,
  revoked_at      TIMESTAMPTZ
);

CREATE INDEX ON webauthn_credentials (user_id) WHERE revoked_at IS NULL;
CREATE INDEX ON webauthn_credentials (credential_id) WHERE revoked_at IS NULL;

---

Validation Checklist

Use this checklist for every registration and authentication endpoint before promoting to production.

Registration endpoint

• rp.id matches the effective domain — never a path, port, or subdomain that differs from the page origin
• Challenge is 32 bytes from crypto.randomBytes, server-generated, with a session-bound TTL of ≤ 300 seconds
• type field in clientDataJSON equals "webauthn.create"
• origin in clientDataJSON matches the expected RP origin exactly (scheme + host + port)
• rpIdHash in authenticatorData matches SHA-256 of rp.id
• UP flag (bit 0) is set in authenticatorData[32]
• UV flag (bit 2) is set when userVerification: 'required' was requested
• credentialId is stored uniquely; reject duplicates with HTTP 409
• Public key is COSE-decoded and stored as raw bytes — not as a JSON string

Authentication endpoint

• Challenge from session is consumed exactly once (delete on first use)
• type field in clientDataJSON equals "webauthn.get"
• rpIdHash re-verified against SHA-256 of rp.id
• UP and UV flags re-checked on every assertion
• Signature verified against stored public key using the COSE algorithm identifier
• signCount is strictly greater than stored value (allow 0 = 0 for synced passkeys only)
• backup_eligible and backup_state flags persisted for policy enforcement

---

Error Reference Table

---

Platform and Library Notes

@simplewebauthn/server (Node.js) — The verifyRegistrationResponse and verifyAuthenticationResponse functions handle CBOR decoding, flag checking, and signCount validation automatically. Pass requireUserVerification: true to enforce UV. The library does not enforce counter regression as a hard rejection by default — you must check authenticationInfo.credentialDeviceType and apply your own policy for synced passkeys.

fido2-lib (Node.js) — Requires explicit rpId and origin configuration at library instantiation. Does not derive the registrable domain automatically, so multi-origin deployments need one instance per origin.

py_webauthn (Python) — verify_registration_response accepts a require_user_verification boolean. The expected_rp_id parameter must be the bare hostname string, not a URL.

WebAuthn4J (Java/Spring) — Flag verification is handled by CoreAuthenticatorDataValidator. Counter validation raises MaliciousCounterValueException on regression.

iOS Safari — Platform authenticator is Face ID / Touch ID via the Secure Enclave. backup_eligible (BE flag) is set for iCloud Keychain passkeys, meaning the credential is synced. Treat these as multi-device credentials; do not rely on signCount for clone detection.

Android Chrome — Google Password Manager passkeys set BE and BS flags. signCount is always 0 for synced Android passkeys. Roaming keys via NFC (YubiKey) do use signCount correctly.

Windows Hello — Uses the TPM. authenticatorAttachment reports "platform". backup_eligible is false (device-bound). signCount increments reliably.

For a deeper look at how these differences affect attestation vs assertion semantics across platforms, see the dedicated guide.

---

Pitfalls and Security Hardening

1. Accepting an RP ID that includes a path or port The spec requires rp.id to be a valid domain string — no https://, no :3000, no /app. Including any of these causes SecurityError in all browsers. Harden: strip the scheme and port in your configuration loader and assert the result matches /^[a-z0-9.-]+\.[a-z]{2,}$/i.

2. Sharing a challenge across concurrent registration sessions A single challenge stored per user (rather than per session) allows an attacker with two open tabs to consume the same challenge. Harden: key the pending_challenges table on session_id, not user_id, and delete the row on first successful use.

3. Treating signCount = 0 as a regression for synced passkeys iOS, Android, and some FIDO2 authenticators report signCount = 0 permanently for synced (multi-device) credentials. Blindly rejecting 0 breaks passkeys for the majority of consumer users. Harden: exempt credential IDs with backup_eligible = true from the strict monotonicity check; log but do not reject.

4. Not checking the UV flag when userVerification: 'required' is set Passing userVerification: 'required' in authenticatorSelection tells the browser what to ask for, but the server must independently verify the UV bit in authenticatorData[32]. Some older CTAP2 implementations silently succeed without UV. Harden: reject assertions where (flags & 0x04) === 0 when your policy requires verification.

5. Storing the COSE public key as a JSON string The COSE map is a CBOR binary structure. Storing it as a JSON-serialised object introduces encoding round-trip bugs that surface only under specific key types (e.g., Ed25519 keys with negative coordinate indices). Harden: store the raw Uint8Array bytes in a BYTEA column and decode with a CBOR library on every verification.

6. Hardcoding RP ID across staging and production If RP_ID=example.com in staging but the staging origin is https://staging.example.com, registrations will succeed but assertions will fail with rpIdHash mismatch. Harden: read RP_ID from an environment variable and validate at startup that it is a suffix of the ORIGIN variable.

---

Related

• WebAuthn & FIDO2 Protocol Fundamentals — parent overview covering the full spec hierarchy
• The challenge-response authentication flow — how CSPRNG challenge generation and assertion verification interact end-to-end
• Attestation vs assertion explained — the distinction between registration-time attestation formats and authentication-time assertion signatures
• WebAuthn security boundaries for enterprise apps — enterprise enforcement of RP ID scoping, AAGUID allowlists, and MDS3 metadata validation
• Public key vs symmetric credential types — trade-offs between discoverable and server-side credentials and COSE algorithm selection