Skip to content

Passway architecture

Jump to bottom
Jeremy Kahn edited this page Jul 24, 2024 · 41 revisions

Status:

Passway

Passway provides anonymous, secure user authentication and end-to-end encrypted cloud storage. It is meant to be a hosted web service that apps can integrate with. The Direct Users of Passway are app developers who want to build anonymous authentication and end-to-end encrypted cloud-based data storage into their projects. The users of those apps are Passway's Downstream Users.

High level architecture goals

  • All code is open source
  • All data transfer is end-to-end encrypted
    • Unencrypted data is never transmitted from clients
      • Exception: Functionally necessary, non-sensitive metadata will be transmitted from client to server unencrypted
  • Authentication is done via WebAuthn
  • Stack can be self-hosted
  • Data can be securely shared between users
  • Can be run on a laptop for local development
  • Can be deployed in a data center and horizontally scaled out

Registration and authentication strategy

Registration and authentication will be based on an unorthodox use of WebAuthn. WebAuthn is meant for authenticating with server-based web services via passkeys (implemented with public/private key cryptography). Though effective, this does not meet Passway's needs because the passkey's private key is not accessible to application code to encrypt data with. To work around this, an ECDSA signature keys and an AES encryption key will be generated on the client, collectively wrapped with AES, and then stored on the server for later retrieval and decryption by a client.

Registration

  1. User initiates registration ceremony and provides their desired user name
  2. User name and a client-side generated random ID string are provided to navigator.credentials.create() as publicKey. The user will be prompted to authenticate (typically via a biometric device) and the passkey will then be created.
  3. The user will then be prompted to "log in" for the first time by re-authenticating (typically via a biometric device). This will call navigator.credentials.get() and result in a PublicKeyCredential that contains a AuthenticatorAssertionResponse.
    • PublicKeyCredential.id (id for short) is treated as a user ID to identify the client to the server
    • AuthenticatorAssertionResponse.userHandle (userHandle for short) is treated as a secret key. It must never be sent to the server.
    • Note: id and userHandle are only available in the navigator.credentials.get() response and not navigator.credentials.create(), thus necessitating the user to re-authenticate
  4. The client generates an ECDSA signature key pair and an AES encryption key. These keys are then wrapped with userHandle as the AES secret key, and client-generated salt and initialization vector
  5. The client sends a PUT /api/v1/user request to the server with the id, the wrapped keys, and unencrypted public signature key in the payload
    • The server will store the public key, salt, initialization vector, and wrapped keys

Note

Why userHandle as the encryption key: Though counterintuitive semantically, application code has influence over the size of userHandle data but not the size of id. Bigger is better when it comes to encryption keys, so that is what's used to secure data before sending it to the server.

id and userHandle are the only stable strings that are stored with the passkey, so a more semantically sensible property of the passkey cannot be used for identification and decryption purposes.

It should be safe to depend on these properties as Passway uses them, as they are a part of W3C specifications:

Registration flow diagram

Authentication

  1. User initiates an authentication ceremony and provides their user name
  2. User name is provided to navigator.credentials.get(), prompting the user to provide biometric identification. If successful, this will result in a PublicKeyCredential that contains a AuthenticatorAssertionResponse.
  3. The client sends a GET /api/v1/user request with the user id as the x-passway-id header. If a user record is found, its metadata (including wrapped keys, encryption salt and initialization vector) is returned to the user in a 200 response.
  4. The client unwraps the keys with the passkey userHandle, salt, and initialization vector to reveal the private signature key
  5. The client sends a GET /api/v1/session request with the user id as the x-passway-id header and a private key-signed version of a shared message between the client and server as the x-passway-signature header
  6. The server looks up the user data associated with id. If found, the server validates the signed message with the public key, creates a session cookie, and returns a 200 response

Authentication flow diagram

Client overview

Requirements

  • Must handle user authentication
  • Must never expose user's private keys
  • Must manage user data encryption key (their WebAuthn public key)
    • It is up to the user to manage and safeguard this key
      • WebAuthn should make this relatively seamless and can largely automate backups and redundancy via service providers like Bitwarden, Apple, and Google
  • Must be easily integrated into consuming apps
  • Must support arbitrarily large data transfers
  • Must support many concurrent users

Design

  • TypeScript-based SDK that gets installed into integrating client apps
  • Provides an API for interacting with a Passway server
  • Leverages tus for streamed content upload
  • Unencrypted user data is never transmitted from the client

Note

Why tus: tus appears to provide a clean abstraction for streaming data from a client to server. Data must be streamed to avoid loading large content data objects in client memory and keep resource usage low.

Data upload
  • Encrypts each data chunk with user's public key before sending it to the server
  • For large data uploads, chunks are stored to IndexedDB prior to uploading when it is available (this avoids loading the entire content in memory)
  • tus is used to transfer data to the server
Data download
  • Decrypts each data chunk with the user's encryption key as they are received from the server
  • Chunks are cached to IndexedDB when it is available (this avoids storing the entire content in memory)

Server overview

Requirements

  • Must support arbitrarily large data transfers
  • Must support low latency data transfer
  • Must support extremely large data storage (many terabytes)

Design

System architecture

System design diagram

Authentication component
  • Node-based application for passkey registration and authentication
    • Implemented as an API with Fastify
    • Stores session data in a Valkey data store that can be accessed by other back end components
  • Set up behind NGINX + modsecurity to guard against DDoS attacks
  • Stores user data (such as passkey ID) in the data management component (outlined below)

Note

Why Fastify: The authentication API is pretty unexotic, so a conventional web app framework should suffice. Fastify has particularly good support for Swagger, which is a helpful addition to an API such as this.

Note

Why NGINX: NGINX is a tried-and-true solution for managing high traffic load to web servers. The request load management needs of Passway are not particularly unique.

Note

Why Valkey: Valkey is the open source successor to Redis. Redis would also work well, but its license terms are not a good fit for Passway.

Data transfer
  • Uses tus-node-server to receive content upload streams and pipe them to MinIO data storage
  • Authenticates requests by comparing Authorization header to what's available in the Authentication system's session store
Data storage
  • User account information and content metadata is stored in PostGres
    • This PostGres DB also contains tables that associate user accounts to user content objects
  • User content is stored in MinIO by default
    • The data storage back end should be treated as an implementation detail. There should be a common interface that allows users to swap MinIO for another data store such as S3 or Backblaze B2
    • Stores both metadata on individual users (such as their username and public key) and their application data
    • Scaling is done horizontally via MinIO Server Pools
    • When quota is exceeded, write operations fail with a descriptive error message
    • Data write operations are atomic upserts
      • When a data stream is received, it is stored in a temporary MinIO object and then moved to the object specified by the associated content-key once all chunks are received
        • This is done to ensure that data is not corrupted if streams are interrupted
  • Defines a quota of available content storage for a given user ID (configurable, but a good default could be 10 GB)
  • Content associated with a stale (unused) Passkey can be set to be evicted at a configurable interval, along with the passkey
    • This is intended to be an abuse mitigation (e.g. a malicious user storing useless data to consume DB space) and resource management mechanism

Note

Why PostGres: User account CRUD operations are likely to be most straightforward with a relational database, and PostGres seems to be as good of an option as any.

Note

Why MinIO: The user content that Passway stores will be encrypted before it reaches the server. This reduces the server to being little more than a dumb data store for which object storage seems like a good fit. MinIO also supports streamed uploads, which is necessary for managing memory usage. MinIO is also S3-compatible, so it should also work nicely with tus's S3/Node adapter.

Registration UI

The registration experience depends on the user agent. Passway defers this concern to navigator.credentials.create.

For example, here's the registration flow on Firefox with Bitwarden (a password manager) integrated:

WebAuthn registration UI

Here's the registration flow for Chromium (with no third-party password manager integration):

WebAuthn registration with Chromium

Client-server interaction data contract

Registration

Route:

PUT /api//v1/user

Headers:

x-passway-id: "<user ID>"

Payload:

{
	"iv": "<encryption key initialization vector>",
	"salt": "<encryption key salt>",
	"keys": "<wrapped encryption keys>",
	"publicKey": "<public signature key>",
}

Authentication

Route:

GET /api/v1/user

Headers:

x-passway-id: "<user ID>"

Response:

{

	"iv": "<encryption key initialization vector>",
	"salt": "<encryption key salt>",
	"keys": "<wrapped encryption keys>",
}

Route:

GET /api/v1/session

Headers:

x-passway-id: "<user ID>"
x-passway-signature: "<signed version of shared string>"

Response:

A session cookie called passwaySessionId.

Content retrieval

Route:

GET /api/v1/content/:contentKey

Headers:

passwaySessionId: "<session token>"
X-Passway-User-Id: "<user ID>"

If the request is valid, the server would then respond with an HTTP 200 response along with the encrypted content as binary data. If the request is invalid, the server would respond with the appropriate HTTP status code and error message.

Content uploading

This is done via tus-js-client.

Route:

POST /api/v1/content/:contentKey

Headers:

passwaySessionId: "<session token>"
x-passway-id: "<user ID>"

The request payload is the encrypted content as binary data. The specific details and behavior of this API may change as a result of tus implementation details and requirements.

User data retrieval

Route:

GET /api/v1/user

Headers:

passwaySessionId: "<session token>"
x-passway-id: "<user ID>"

200 response if user exists:

{
    "user-id": "<some user ID>",
    "error": null,
    "publicKey": "<public key for user-id>"
}

Or an error with a message and appropriate HTTP status.

Data sharing

Users should be able to securely share data with each other through Passway. This can be deferred to a post-MVP iteration, but the initial design should account for it.

Flow

Assuming User A wants to share data with User B:

  • User A produces a one-time use message containing a random key and content metadata (such as its ID), and encrypts their data with it
  • User A sends the encrypted message to the server to be stored (like any other data)
  • User A retrieves the public key of User B from the server
  • User A encrypts a message containing the one-time message key and nonsensitive metadata about the shared data with User B's public key
  • User A sends the encrypted metadata message to the server to an "inbox" on the server for user B
  • User B retrieves the encrypted metadata message from their inbox on the server
  • User B decrypts the metadata message with their private key which contains content ID and decryption password
  • User B retrieves the encrypted content from the server
  • User B decrypts the downloaded content

Pitfalls with this design

  • User loses access to passkey
    • User's access to their data is lost
    • Data may be evicted after being stale for a configurable period of time
Clone this wiki locally