Cryptographic Algorithms Supported by WebAuthn

Selecting the wrong algorithm — or negotiating it incorrectly — is one of the most common causes of signature verification failures in production WebAuthn deployments. This page covers every COSE algorithm the WebAuthn & FIDO2 protocol currently mandates or recommends, explains the CBOR key structures involved, and provides TypeScript and Python verification code grounded in W3C WebAuthn Level 3 (§ 6.5) and RFC 8152 (COSE).

---

Concept Definition and Spec Grounding

WebAuthn does not invent its own cryptographic vocabulary. It borrows COSE — CBOR Object Signing and Encryption (RFC 8152, superseded by RFC 9052) — to express algorithm identifiers as small negative integers inside CBOR maps. During credential creation the Relying Party (RP) sends an ordered pubKeyCredParams array; the authenticator picks the first entry it supports. The chosen algorithm determines how the COSE_Key map inside attestationObject.authData is structured, and it is the algorithm that must be used for every subsequent assertion signature.

COSE key map layout (W3C WebAuthn Level 3 § 6.5.1.1)

The public key is embedded inside authenticatorData starting at byte 55 (after the 32-byte rpIdHash, 1-byte flags, 4-byte signCount, and any AAGUID/credId prefix). The CBOR-encoded COSE_Key contains:

Core algorithm registry

ES256 is the only algorithm all FIDO2-certified authenticators must implement. Every other entry is negotiated; if an authenticator does not support any entry in pubKeyCredParams it throws NotSupportedError.

---

Architecture and Data Flow

The diagram below shows the algorithm negotiation path from RP to authenticator and back during registration, and the corresponding verification path during authentication.

---

Implementation Guide

Step 1 — Declare pubKeyCredParams with an ordered preference array

W3C WebAuthn Level 3 § 5.4 requires the RP to pass pubKeyCredParams to navigator.credentials.create(). The authenticator selects the first entry it supports, so list stronger or preferred algorithms first.

// registration-options.ts
const pubKeyCredParams: PublicKeyCredentialParameters[] = [
  { type: 'public-key', alg: -7   }, // ES256  — ECDSA P-256 + SHA-256 (mandatory per FIDO2)
  { type: 'public-key', alg: -8   }, // EdDSA  — Ed25519 (modern platform authenticators)
  { type: 'public-key', alg: -257 }, // RS256  — RSA PKCS#1 v1.5 + SHA-256 (enterprise HSMs)
  { type: 'public-key', alg: -37  }, // PS256  — RSA-PSS + SHA-256 (FIPS 140-3 environments)
];

const options: PublicKeyCredentialCreationOptions = {
  challenge: crypto.getRandomValues(new Uint8Array(32)), // server-generated CSPRNG
  rp: { name: 'Example Corp', id: 'example.com' },
  user: {
    id: crypto.getRandomValues(new Uint8Array(16)),
    name: '[email protected]',
    displayName: 'Alice',
  },
  pubKeyCredParams,
  authenticatorSelection: {
    residentKey: 'preferred',       // discoverable credential
    userVerification: 'required',   // UV flag must be set in authData
  },
  timeout: 60_000,
};

Satisfies: W3C WebAuthn Level 3 § 5.4 (PublicKeyCredentialCreationOptions), FIDO2 Security Requirements § 5.1 (ES256 mandatory).

Step 2 — Parse the COSE_Key from authenticatorData

After registration, the RP receives an attestationObject. The public key lives inside authData at a variable byte offset after the attestedCredentialData prefix. Use an audited CBOR library — never hand-roll COSE parsing.

// parse-cose-key.ts
import { decode } from 'cbor-x'; // RFC 7049 / RFC 8949 CBOR decoder

interface CoseKey {
  1: number;  // kty  — 2 = EC2, 3 = RSA, 1 = OKP (EdDSA)
  3: number;  // alg  — -7, -8, -257, -37 …
  '-1': number | Buffer; // crv (EC/OKP) or n (RSA)
  '-2': Buffer;          // x (EC) or e (RSA)
  '-3'?: Buffer;         // y (EC only)
}

function extractCoseKey(authData: Buffer): CoseKey {
  // authData layout (W3C WebAuthn Level 3 § 6.1):
  //   [0..31]  rpIdHash (32 bytes)
  //   [32]     flags (1 byte) — bit 6 = AT (attested credential data present)
  //   [33..36] signCount (4 bytes, big-endian uint32)
  //   [37..52] AAGUID (16 bytes)
  //   [53..54] credIdLength (2 bytes, big-endian uint16)
  //   [55 .. 55+credIdLength-1]  credentialId
  //   [55+credIdLength ..]       COSE_Key (CBOR map)

  const flags = authData[32];
  if (!(flags & 0x40)) throw new Error('AT flag not set — no attested credential data present');

  const credIdLength = authData.readUInt16BE(53);
  const coseStart = 55 + credIdLength;
  return decode(authData.subarray(coseStart)) as CoseKey;
}

Satisfies: W3C WebAuthn Level 3 § 6.5.1.1 (COSE_Key parsing), RFC 9052 § 7 (key type parameters).

Step 3 — Store alg alongside the SPKI-encoded public key

The alg integer is the routing key for all future assertion verifications. Store it in the same row as the credential.

-- credentials table (PostgreSQL)
CREATE TABLE webauthn_credentials (
  credential_id     BYTEA        PRIMARY KEY,
  user_id           UUID         NOT NULL REFERENCES users(id),
  cose_alg          SMALLINT     NOT NULL,     -- -7, -8, -257, -37 …
  public_key_spki   BYTEA        NOT NULL,     -- SubjectPublicKeyInfo (DER)
  sign_count        INTEGER      NOT NULL DEFAULT 0,
  aaguid            UUID,
  created_at        TIMESTAMPTZ  NOT NULL DEFAULT now(),
  last_used_at      TIMESTAMPTZ
);
CREATE INDEX ON webauthn_credentials (user_id);

For background on how stored public keys relate to overall credential storage design, see handling public key storage and rotation.

Step 4 — Route assertion verification by cose_alg

The challenge-response authentication flow requires the RP to verify the assertion signature using the exact algorithm negotiated at registration. Route by the stored cose_alg, not by inspecting the public key’s ASN.1 OID at runtime.

// verify-assertion.ts
import { createVerify } from 'node:crypto';

interface StoredCredential {
  coseAlg: number;          // from DB column cose_alg
  publicKeySpki: Buffer;    // DER-encoded SPKI
  signCount: number;
}

interface AssertionResponse {
  authenticatorData: string; // base64url
  clientDataJSON: string;    // base64url
  signature: string;         // base64url
}

async function verifyAssertion(
  assertion: AssertionResponse,
  stored: StoredCredential,
): Promise<{ verified: boolean; newCounter: number }> {
  const authData   = Buffer.from(assertion.authenticatorData, 'base64url');
  const clientData = Buffer.from(assertion.clientDataJSON,     'base64url');
  const signature  = Buffer.from(assertion.signature,          'base64url');

  // Signed payload per WebAuthn Level 3 § 7.2 step 20:
  // authData ‖ SHA-256(clientDataJSON)
  const clientDataHash  = createHash('sha256').update(clientData).digest();
  const signedPayload   = Buffer.concat([authData, clientDataHash]);

  // Algorithm routing — must use OpenSSL digest names, not COSE labels
  const { digestAlg, dsaEncoding } = resolveAlgParams(stored.coseAlg);

  const ok = createVerify(digestAlg)
    .update(signedPayload)
    .verify({ key: stored.publicKeySpki, format: 'der', type: 'spki', dsaEncoding }, signature);

  if (!ok) throw new Error('Signature verification failed');

  // signCount is bytes 33–36 of authData (big-endian uint32)
  const newCounter = authData.readUInt32BE(33);
  if (newCounter !== 0 && newCounter <= stored.signCount) {
    throw new Error('signCount did not increase — possible cloned authenticator');
  }

  return { verified: true, newCounter };
}

function resolveAlgParams(coseAlg: number): { digestAlg: string; dsaEncoding: 'der' | 'ieee-p1363' } {
  switch (coseAlg) {
    case  -7:  return { digestAlg: 'SHA256',    dsaEncoding: 'der'         }; // ES256
    case  -35: return { digestAlg: 'SHA384',    dsaEncoding: 'der'         }; // ES384
    case  -36: return { digestAlg: 'SHA512',    dsaEncoding: 'der'         }; // ES512
    case  -8:  return { digestAlg: 'SHA512',    dsaEncoding: 'ieee-p1363'  }; // EdDSA / Ed25519
    case -257: return { digestAlg: 'RSA-SHA256', dsaEncoding: 'der'        }; // RS256
    case  -37: return { digestAlg: 'RSA-SHA256', dsaEncoding: 'der'        }; // PS256 (use createVerify with 'RSA-PSS')
    default:   throw new Error(`Unsupported COSE alg: ${coseAlg}`);
  }
}

Satisfies: W3C WebAuthn Level 3 § 7.2 steps 20–21 (signature and signCount validation).

Step 5 — Handle EdDSA (Ed25519) as a special case

Ed25519 keys have kty = 1 (OKP) and crv = 6. Node.js crypto.createVerify('SHA512') does not work for Ed25519; use crypto.verify(null, …) with a key object typed as ed25519.

// ed25519-verify.ts
import { createPublicKey, verify as cryptoVerify } from 'node:crypto';

function verifyEdDSA(signedPayload: Buffer, signature: Buffer, spkiDer: Buffer): boolean {
  const pubKey = createPublicKey({ key: spkiDer, format: 'der', type: 'spki' });
  // Ed25519 uses null as the algorithm parameter — the key type implies the hash function
  return cryptoVerify(null, signedPayload, pubKey, signature);
}

---

Validation Checklist

• pubKeyCredParams is a non-empty array with at least { type: 'public-key', alg: -7 } (ES256)
• Algorithm entries are ordered from most preferred to least preferred — the authenticator picks the first match
• After registration, cose_alg integer is persisted to the database alongside credential_id
• COSE_Key is decoded with a spec-compliant CBOR library; raw byte parsing is not used
• Assertion signed payload is exactly authData || SHA-256(clientDataJSON) — not just clientDataJSON
• Digest algorithm passed to crypto.verify() is the OpenSSL name (SHA256, RSA-SHA256), not the COSE label
• ECDSA signatures are decoded as DER (ASN.1) by default; dsaEncoding is explicit in the verify call
• Ed25519 signatures are verified with crypto.verify(null, …), not createVerify('SHA512')
• signCount is read as a 4-byte big-endian uint32 from authData[33..36]
• A signCount that did not increase triggers at minimum an alert; a strict RP rejects the assertion
• RS256 / PS256 key size is validated to be at least 2048 bits before storage
• RS1 (SHA-1 RSA, COSE alg -65535) is explicitly rejected in the RP’s allowlist

---

Error Reference Table

---

Platform and Library Notes

@simplewebauthn/server (TypeScript)

verifyRegistrationResponse and verifyAuthenticationResponse handle COSE_Key parsing and algorithm routing automatically. To restrict allowed algorithms, pass supportedAlgorithmIDs to generateRegistrationOptions:

import { generateRegistrationOptions } from '@simplewebauthn/server';

const options = await generateRegistrationOptions({
  rpName: 'Example Corp',
  rpID: 'example.com',
  userID: userId,
  userName: '[email protected]',
  supportedAlgorithmIDs: [-7, -8], // ES256 + EdDSA only; exclude RS256
});

py_webauthn (Python)

verify_authentication_response accepts expected_credential_public_key as a CBOR-encoded COSE_Key. The library reads the alg field from the key map and selects the verifier internally — no manual routing needed:

from webauthn import verify_authentication_response

verification = verify_authentication_response(
    credential=assertion,
    expected_challenge=expected_challenge,
    expected_rp_id="example.com",
    expected_origin="https://example.com",
    credential_public_key=stored_credential.cose_public_key,
    credential_current_sign_count=stored_credential.sign_count,
)

WebAuthn4J (Java)

CoreAuthenticationDataValidator uses the COSEKey class hierarchy. ES256 maps to EC2COSEKey, RS256 maps to RSACOSEKey, and EdDSA maps to OKPCOSEKey. Key type mismatches throw VerificationException with a descriptive message.

Platform authenticator quirks

The attestation vs assertion explained page covers how the algorithm choice affects attestation statement formats (packed, tpm, apple) during registration.

---

Pitfalls and Security Hardening

1. Passing the COSE label string to crypto.verify() instead of the OpenSSL digest name

Root cause: COSE labels like ES256 are not valid OpenSSL algorithm names. Node.js silently accepts the string and returns false for every signature. Mitigation: always route through a resolveAlgParams() function that maps COSE integer → OpenSSL string (SHA256 for ES256, RSA-SHA256 for RS256).

2. Treating ECDSA signatures as raw r || s instead of DER

Root cause: some authenticators (notably some FIDO U2F tokens) return raw 64-byte signatures; others return DER-encoded ASN.1. Using the wrong format produces consistent verification failures. Mitigation: set dsaEncoding: 'der' explicitly and normalise raw signatures to DER before calling verify().

3. Not persisting cose_alg with the credential

Root cause: some implementations assume ES256 universally and do not store the negotiated algorithm. When an enterprise device enrolls with RS256, subsequent authentication silently fails. Mitigation: store cose_alg as a non-nullable column alongside credential_id at registration time; never infer the algorithm from the public key’s ASN.1 OID at verification time.

4. Accepting credentials with an alg not in the RP’s declared pubKeyCredParams

Root cause: the browser may have exercised a platform default that the RP did not explicitly offer. Accepting it bypasses the RP’s algorithm policy. Mitigation: after registration, validate that the returned cose_alg appears in the original pubKeyCredParams array before storing the credential.

5. Including RS1 (COSE alg -65535, SHA-1) as a legacy fallback

Root cause: some old enterprise integrations include SHA-1-based RSA to support aging tokens. SHA-1 collision attacks (SHAttered, 2017) make this unacceptable for identity assertions. Mitigation: maintain an explicit denylist ([-65535]) in the RP’s algorithm policy and reject any credential presenting this identifier.

6. Using a single crypto.createVerify() call for both ECDSA and EdDSA

Root cause: Ed25519 does not use a separate digest step — createVerify('SHA512') does not work; you must call crypto.verify(null, …) with an Ed25519 key object. Mitigation: branch on coseAlg === -8 before verification and use the correct API path.

---

Related

• WebAuthn & FIDO2 Protocol Fundamentals — parent section covering the full protocol hierarchy
• The challenge-response authentication flow — how COSE algorithm selection integrates with the full assertion pipeline
• Attestation vs assertion explained — how algorithm choice determines valid attestation statement formats
• Handling signature verification in Node.js — deep-dive into server-side ECDSA and RSA verification paths
• Handling public key storage and rotation — storing SPKI-encoded public keys alongside the cose_alg identifier