fix(passkeys): pass challenge as Buffer to simplewebauthn to fix lookup

[dschom]

Because

• simplewebauthn v13 treats string challenges as UTF-8 text and re-encodes them to base64url, producing a different value than what was stored in Redis. This caused challenge lookup to fail on registration/authentication finish.
• Calls to the sipmlewebauthn lib were being unnecessarily mocked, so we didn't catch this.
• We had a few spots where we were not buffer encoding correctly. Strings are hex strings, and UID are 16 bit length buffers.

This pull request

• Passes challenge as Buffer.from(challenge, 'base64url') to generateRegistrationOptions and generateAuthenticationOptions so simplewebauthn base64url-encodes the raw bytes (roundtrip-safe).
• Adds webauthn-adapter.in.spec.ts with a virtual authenticator that exercises real simplewebauthn verifyRegistrationResponse (no mocks).
• Adds happy-path integration test for registration start → finish.
• Fixes uid buffer encodings

Issue that this pull request solves

Closes: FXA-13343

Checklist

Put an x in the boxes that apply

• My commit is GPG signed.
• If applicable, I have modified or added tests which pass locally.
• I have added necessary documentation (if appropriate).
• I have verified that my changes render correctly in RTL (if appropriate).
• I have manually reviewed all AI generated code.

How to review (Optional)

• Key files/areas to focus on: Take a good look at the web authn tests now. These were updated by claude.
• Suggested review order: Webauthn then other parts
• Risky or complex parts: The virutal authenticator was all gen ai code.

Screenshots (Optional)

Please attach the screenshots of the changes made in case of change in user interface.

Other information (Optional)

At first it appeared that challenges output by libGenerateAuthenticationOptions were always changing from the original challenge that is provided to the library call. I assumed that this was by design, however, after adding more tests that actually exercised the library call and doing some debugging it appears the core issue was just the encoding on the challenge. (Which is what Claude thought.)

There was one red herring in previous tests worth noting. When using an arbitrary challenge like test-challenge in tests, failures would occur since it doesn't encode cleanly, e.g. Buffer.from('test-challenge', 'base64url').toString('base64url'); -> 'test-challengQ'. Using a actual challenge value like const challenge = randomBytes(32).toString('base64url'); won't have this problem since the challenge is already base64 encoded.

[vpomerleau]

This is test scaffolding — should it be included in the public barrel? Noting a similar question about passkey.repository and webauthn-adapter, perhaps a quick follow-up?

[vpomerleau]

Thank you for spotting and resolving the challenge failure. The addition of new tests with virtual authenticator (instead of mocks) is much appreciated. I could see potential for expanding tests to cover more options (like backupState, prfEnabled, signCount regression), but major paths all look covered. No blockers other than ensuring that tests pass, most comments are questions/non-blocking.

[vpomerleau]

Is there a reason to use discouraged and not required here?

[dschom]

There is not. This gen-ai, but it seemed okay to me. I can switch to required if you think that's more typical.

[vpomerleau]

We're planning to always require user verification

[vpomerleau]

Do we need a ticket to add similar tests for authentication?

[dschom]

I think it'd be helpful. That would ensure wiring it all up goes smoothly.