diff --git a/.github/workflows/check.yml b/.github/workflows/check.yml index c91a601..8dd21ce 100644 --- a/.github/workflows/check.yml +++ b/.github/workflows/check.yml @@ -8,6 +8,23 @@ on: branches: [ "**" ] jobs: + build: + name: Run build + runs-on: ubuntu-latest + permissions: + contents: read + pull-requests: read + steps: + - name: Set up NodeJS + uses: actions/setup-node@v6 + with: + node-version: 'lts/*' + - name: Checkout repository + uses: actions/checkout@v6 + - name: Install dependencies + run: npm i + - name: Run build + run: npm run build eslint: name: Run ESLint runs-on: ubuntu-latest diff --git a/README.md b/README.md index bc33c60..329de32 100644 --- a/README.md +++ b/README.md @@ -1,14 +1,18 @@ # SecureReq ๐Ÿ” -**SecureReq** is a lightweight TypeScript utility for making secure HTTPS requests with strict TLS defaults and typed response parsing. +**SecureReq** is a lightweight TypeScript utility for secure HTTP requests with strict TLS defaults, automatic http/1.1 to http/2 negotiation, streaming I/O, and typed response parsing. --- ## ๐Ÿš€ Quick Summary -- **Small, dependency-light** wrapper around Node's `https` for typed responses and safer TLS defaults. +- **Class-first** API that probes each origin with `http/1.1` first, then upgrades future requests to `http/2` when appropriate. +- Also exposes `SimpleSecureReq`, a shared client instance for one-off requests without manual construction. +- Automatic HTTP/2 probing is conservative: only safe body-less auto requests are retried from negotiation failure to `http/1.1`. +- Supports **response compression** with `zstd`, `gzip`, and `deflate`. +- Supports optional **redirect following** with configurable redirect limits. +- Supports **streaming uploads and streaming downloads**. - Defaults to **TLSv1.3**, Post Quantum Cryptography key exchange, a limited set of strongest ciphers, and a `User-Agent` header. -- Supports typed response parsing: `JSON`, `String`, or raw `ArrayBuffer`. --- @@ -24,36 +28,89 @@ npm install @typescriptprime/securereq ## Usage Examples ๐Ÿ”ง -Import and call the helper: +Create a client and reuse it per origin: ```ts -import { HTTPSRequest } from '@typescriptprime/securereq' +import { Readable } from 'node:stream' +import { SecureReq } from '@typescriptprime/securereq' + +const client = new SecureReq() + +// First request to an origin uses http/1.1 probing. +const first = await client.Request(new URL('https://api64.ipify.org?format=json'), { + ExpectedAs: 'JSON', +}) + +// Later safe requests to the same origin can probe and establish http/2 automatically. +const second = await client.Request(new URL('https://api64.ipify.org?format=json'), { + ExpectedAs: 'JSON', +}) + +console.log(first.Protocol) // 'http/1.1' +console.log(second.Protocol) // 'http/2' when available after the safe probe + +// Follow redirects automatically +const redirected = await client.Request(new URL('https://example.com/old-path'), { + ExpectedAs: 'String', + FollowRedirects: true, + MaxRedirects: 5, +}) + +console.log(redirected.Body) + +// Stream upload + stream download +const streamed = await client.Request(new URL('https://example.com/upload'), { + HttpMethod: 'POST', + Payload: Readable.from(['chunk-1', 'chunk-2']), + ExpectedAs: 'Stream', +}) + +for await (const chunk of streamed.Body) { + console.log(chunk) +} +``` + +For quick one-off requests, you can use the exported shared client: -// JSON (auto-detected by .json path) or explicit -const url = new URL('https://api64.ipify.org?format=json') -const res = await HTTPSRequest(url) -console.log(res.StatusCode) // number -console.log(res.Body) // ArrayBuffer or parsed JSON depending on `ExpectedAs` and URL +```ts +import { SimpleSecureReq } from '@typescriptprime/securereq' -// Force string -const html = await HTTPSRequest(new URL('https://www.example.com/'), { ExpectedAs: 'String' }) -console.log(typeof html.Body) // 'string' +const response = await SimpleSecureReq.Request(new URL('https://api64.ipify.org?format=json'), { + ExpectedAs: 'JSON', +}) -// Force ArrayBuffer -const raw = await HTTPSRequest(new URL('https://example.com/'), { ExpectedAs: 'ArrayBuffer' }) -console.log(raw.Body instanceof ArrayBuffer) +console.log(response.Body) ``` --- ## API Reference ๐Ÿ“š -### HTTPSRequest(Url, Options?) +### `new SecureReq(Options?)` + +- Recommended entry point. +- Keeps per-origin capability state: + - first request is sent with `http/1.1` + - `Accept-Encoding: zstd, gzip, deflate` + - later safe requests can probe `http/2`, and capability updates only reflect observed protocol/compression evidence +- `Close()` closes cached http/2 sessions. +- `OriginCapabilityCacheLimit` bounds remembered origin capability entries with LRU-style eviction. +- Invalid constructor options fail fast during initialization. + +### `SimpleSecureReq` + +- An exported shared `SecureReq` instance for simple or occasional requests. +- Useful when you do not need to manage your own client lifecycle manually. +- Supports the same `.Request()`, `.GetOriginCapabilities()`, and `.Close()` methods as a manually created `SecureReq` instance. + +### `client.Request(Url, Options?)` - `Url: URL` โ€” Target URL (must be an instance of `URL`). - `Options?: HTTPSRequestOptions` โ€” Optional configuration object. -Returns: `Promise>` where `T` is determined by `ExpectedAs`. +Returns: +- If `ExpectedAs` is specified, `Promise>` +- If `ExpectedAs` is omitted, `Promise>` Throws: - `TypeError` when `Url` is not a `URL` instance. @@ -62,33 +119,55 @@ Throws: ### HTTPSRequestOptions Fields: -- `TLS?: { IsHTTPSEnforced?: boolean, MinTLSVersion?: 'TLSv1.2'|'TLSv1.3', MaxTLSVersion?: 'TLSv1.2'|'TLSv1.3', Ciphers?: string[], KeyExchanges?: string[] }` +- `TLS?: { IsHTTPSEnforced?: boolean, MinTLSVersion?: 'TLSv1.2'|'TLSv1.3', MaxTLSVersion?: 'TLSv1.2'|'TLSv1.3', Ciphers?: string[], KeyExchanges?: string[], RejectUnauthorized?: boolean }` - Defaults: `IsHTTPSEnforced: true`, both Min and Max set to `TLSv1.3`, a small secure cipher list and key exchange choices. - When `IsHTTPSEnforced` is `true`, a non-`https:` URL will throw. + - `KeyExchanges` is forwarded to Node.js as the TLS supported groups / curve list. For strict `TLSv1.2` + ECDSA servers, overly narrow values such as only `X25519` may fail; include a compatible certificate curve such as `P-256` when needed. - `HttpHeaders?: Record` โ€” Custom headers. A `User-Agent` header is provided by default. -- `ExpectedAs?: 'JSON'|'String'|'ArrayBuffer'` โ€” How to parse the response body. +- `HttpMethod?: 'GET'|'POST'|'PUT'|'DELETE'|'PATCH'|'HEAD'|'OPTIONS'` +- `Payload?: string | ArrayBuffer | Uint8Array | Readable | AsyncIterable` +- `ExpectedAs?: 'JSON'|'String'|'ArrayBuffer'|'Stream'` โ€” How to parse the response body. + - Omitting `ExpectedAs` keeps the runtime extension heuristic (`.json`, `.txt`, fallback `ArrayBuffer`) but the body type is intentionally `unknown`. +- `PreferredProtocol?: 'auto'|'http/1.1'|'http/2'|'http/3'` + - `http/3` is currently a placeholder preference and uses the same TCP/TLS negotiation path as `http/2` until native HTTP/3 transport is added. +- `EnableCompression?: boolean` โ€” Enables automatic `Accept-Encoding` negotiation and transparent response decompression. +- `FollowRedirects?: boolean` โ€” Follows redirect responses with a `Location` header. +- `MaxRedirects?: number` โ€” Maximum redirect hops when `FollowRedirects` is enabled. Default: `5`. +- `TimeoutMs?: number` โ€” Aborts the request if headers or body transfer exceed the given number of milliseconds. +- `Signal?: AbortSignal` โ€” Cancels the request using a standard abort signal. ### HTTPSResponse -- `{ StatusCode: number, Headers: Record, Body: T }` +- `{ StatusCode: number, Headers: Record, Body: T, Protocol: 'http/1.1'|'http/2', ContentEncoding: 'identity'|'zstd'|'gzip'|'deflate', DecodedBody: boolean }` Notes: -- If `ExpectedAs` is omitted, a heuristic is used: `.json` โ†’ `JSON`, `.txt` โ†’ `String`, otherwise `ArrayBuffer`. +- If `ExpectedAs` is omitted, a heuristic is still used at runtime: `.json` โ†’ `JSON`, `.txt` โ†’ `String`, otherwise `ArrayBuffer`. +- Because omitted `ExpectedAs` may produce different runtime body shapes, the TypeScript return type is `unknown`. Prefer explicit `ExpectedAs` in application code. - When `ExpectedAs` is `JSON`, the body is parsed and an error is thrown if parsing fails. +- When `ExpectedAs` is `Stream`, the body is returned as a Node.js readable stream. +- Redirects are returned as-is by default. Set `FollowRedirects: true` to follow them. +- `301`/`302` convert `POST` into `GET`, `303` converts non-`HEAD` methods into `GET`, and `307`/`308` preserve method and payload. +- Redirects that require replaying a streaming payload are rejected instead of silently re-sending the stream. --- ## Security & Behavior Notes ๐Ÿ” - Strict TLS defaults lean on **TLSv1.3** and a reduced cipher list to encourage secure transport out of the box. -- TLS options are forwarded to Node's HTTPS layer (`minVersion`, `maxVersion`, `ciphers`, `ecdhCurve`). +- TLS options are forwarded to Node's HTTPS or http/2 TLS layer (`minVersion`, `maxVersion`, `ciphers`, `ecdhCurve`). +- When SecureReq performs an ALPN probe for HTTPS, the negotiated `TLSSocket` is reused for the actual `http/2` or `http/1.1` request instead of opening a second TLS connection. - The library uses `zod` for runtime validation of options. +- Compression negotiation is origin-scoped. Subdomains are tracked independently. +- `GetOriginCapabilities().PreferredProtocol` is updated from actual observed transport, and automatic fallback only occurs for safe negotiation failures before request bytes are sent. +- `GetOriginCapabilities().SupportedCompressions` is only narrowed when the response provided actual compression evidence. +- `GetOriginCapabilities().PreferredProtocol` reflects the currently usable transport (`http/1.1` or `http/2`), while `HTTP3Advertised` records whether the origin advertised `h3`. +- http/3 advertisement points are recorded from response headers, but Node.js built-in http/3 transport is not yet used. --- ## Development & Testing ๐Ÿงช -- Build: `npm run build` (uses `esbuild` + `tsc` for types) +- Build: `npm run build` (uses `tsc -p sources/tsconfig.json`) - Test: `npm test` (uses `ava`) - Lint: `npm run lint` @@ -102,4 +181,4 @@ Contributions, bug reports and PRs are welcome โ€” please follow the repository' ## License -This project is licensed under the **Apache-2.0** License. See the `LICENSE` file for details. \ No newline at end of file +This project is licensed under the **Apache-2.0** License. See the `LICENSE` file for details. diff --git a/eslint.config.js b/eslint.config.js index 88d2fd2..2a52aac 100644 --- a/eslint.config.js +++ b/eslint.config.js @@ -17,11 +17,11 @@ const config = [ "quotes": ["error", "single"], "@typescript-eslint/no-unused-vars": "warn", '@typescript-eslint/naming-convention': ['error', { - selector: ['variableLike', 'parameterProperty', 'classProperty', 'typeProperty'], + selector: ['variableLike', 'parameterProperty', 'classProperty', 'typeProperty', 'classMethod'], format: ['PascalCase'] }] } } ] -export default config \ No newline at end of file +export default config diff --git a/package.json b/package.json index 6e72b0d..92e61a6 100644 --- a/package.json +++ b/package.json @@ -18,10 +18,10 @@ "exports": { ".": { "node": "./dist/index.js", - "types": "./dist/types/index.d.ts" + "types": "./dist/index.d.ts" } }, - "types": "./dist/types/index.d.ts", + "types": "./dist/index.d.ts", "files": [ "dist/**/*", "LICENSE", diff --git a/sources/constants.ts b/sources/constants.ts new file mode 100644 index 0000000..66dbbee --- /dev/null +++ b/sources/constants.ts @@ -0,0 +1,22 @@ +import * as Process from 'node:process' +import * as TLS from 'node:tls' +import type { HTTPCompressionAlgorithm, HTTPMethod, HTTPSRequestOptions } from './type.js' + +export const DefaultTLSOptions = { + IsHTTPSEnforced: true, + MinTLSVersion: 'TLSv1.3', + MaxTLSVersion: 'TLSv1.3', + Ciphers: ['TLS_AES_256_GCM_SHA384', 'TLS_CHACHA20_POLY1305_SHA256'], + KeyExchanges: ['X25519MLKEM768', 'X25519'], + RejectUnauthorized: true, +} as const satisfies NonNullable + +export const DefaultHTTPHeaders = { + 'user-agent': `node/${Process.version} ${Process.platform} ${Process.arch} workspace/false`, +} as const + +export const DefaultSupportedCompressions: HTTPCompressionAlgorithm[] = ['zstd', 'gzip', 'deflate'] +export const ConnectionSpecificHeaders = new Set(['connection', 'host', 'http2-settings', 'keep-alive', 'proxy-connection', 'te', 'transfer-encoding', 'upgrade']) +export const PayloadEnabledMethods = new Set(['GET', 'POST', 'PUT', 'PATCH', 'OPTIONS']) +export const AutomaticHTTP2ProbeMethods = new Set(['GET', 'HEAD']) +export const AvailableTLSCiphers = new Set(TLS.getCiphers().map(Cipher => Cipher.toLowerCase())) diff --git a/sources/index.ts b/sources/index.ts index 3d33491..5ed7b80 100644 --- a/sources/index.ts +++ b/sources/index.ts @@ -1,280 +1,18 @@ -import * as HTTPS from 'node:https' -import * as HTTP2 from 'node:http2' -import * as TLS from 'node:tls' -import * as Process from 'node:process' -import * as Zod from 'zod' -import { ConcatArrayBuffers } from './utils.js' -import type { HTTPSRequestOptions, HTTPSResponse, ExpectedAsMap, ExpectedAsKey } from './type.js' - - -/** - * Perform an HTTPS GET request with strict TLS defaults and typed response parsing. - * - * @param {URL} Url - The target URL. Must be an instance of `URL`. - * @param {HTTPSRequestOptions} [Options] - Request options including TLS settings, headers, and `ExpectedAs`. Defaults to secure TLS v1.3 and a default `User-Agent` header. - * @returns {Promise>} Resolves with `{ StatusCode, Headers, Body }`, where `Body` is parsed according to `ExpectedAs`. - * @throws {TypeError} If `Url` is not an instance of `URL`. - * @throws {Error} When the request errors or if parsing the response body (e.g. JSON) fails. - * - * Notes: - * - TLS options are forwarded to the underlying Node.js HTTPS request (minVersion, maxVersion, ciphers, ecdhCurve). - */ -export async function HTTPSRequest(Url: URL, Options?: HTTPSRequestOptions): Promise> { - const DefaultOptions = { - TLS: { - IsHTTPSEnforced: true, - MinTLSVersion: 'TLSv1.3', - MaxTLSVersion: 'TLSv1.3', - Ciphers: ['TLS_AES_256_GCM_SHA384', 'TLS_CHACHA20_POLY1305_SHA256'], - KeyExchanges: ['X25519MLKEM768', 'X25519'], - }, - HttpMethod: 'GET', - HttpHeaders: { - 'User-Agent': `node/${Process.version} ${Process.platform} ${Process.arch} workspace/false`, - }, - } as const - - const MergedOptions = { ...DefaultOptions, ...(Options ?? {}) } as HTTPSRequestOptions - if (Url instanceof URL === false) { - throw new TypeError('Url must be an instance of URL') - } - - await Zod.strictObject({ - TLS: Zod.strictObject({ - IsHTTPSEnforced: Zod.boolean().optional(), - MinTLSVersion: Zod.enum(['TLSv1.2', 'TLSv1.3']).optional(), - MaxTLSVersion: Zod.enum(['TLSv1.2', 'TLSv1.3']).optional(), - Ciphers: Zod.array(Zod.string().refine(Cipher => TLS.getCiphers().map(C => C.toLowerCase()).includes(Cipher.toLowerCase()))).optional(), - KeyExchanges: Zod.array(Zod.string()).optional() - }).partial().optional(), - HttpHeaders: Zod.record(Zod.string(), Zod.string()).optional(), - HttpMethod: Zod.enum(['GET', 'POST', 'PUT', 'DELETE', 'PATCH', 'HEAD', 'OPTIONS']).optional(), - Payload: Zod.union([Zod.string(), Zod.instanceof(ArrayBuffer), Zod.instanceof(Uint8Array)]).optional(), - ExpectedAs: Zod.enum(['JSON', 'String', 'ArrayBuffer']).optional() - }).parseAsync(Options ?? {}) - - if (MergedOptions.TLS?.IsHTTPSEnforced && Url.protocol !== 'https:') { - throw new Error('HTTPS is enforced, but the URL protocol is not HTTPS') - } - - if (MergedOptions.Payload && !['GET', 'POST', 'PUT', 'PATCH', 'OPTIONS'].includes(MergedOptions.HttpMethod ?? 'GET')) { - throw new Error('Request payload is only supported for GET, POST, PUT, PATCH, and OPTIONS methods') - } - - const ExpectedAs = (Options?.ExpectedAs ?? (Url.pathname.endsWith('.json') ? 'JSON' : Url.pathname.endsWith('.txt') ? 'String' : 'ArrayBuffer')) as E - - const HTTPSResponse = await new Promise>((Resolve, Reject) => { - const HTTPSRequestInstance = HTTPS.request({ - protocol: Url.protocol, - hostname: Url.hostname, - port: Url.port, - path: Url.pathname + Url.search, - headers: MergedOptions.HttpHeaders, - minVersion: MergedOptions.TLS?.MinTLSVersion, - maxVersion: MergedOptions.TLS?.MaxTLSVersion, - ciphers: MergedOptions.TLS?.Ciphers?.join(':'), - ecdhCurve: MergedOptions.TLS?.KeyExchanges?.join(':'), - method: MergedOptions.HttpMethod, - }, (Res) => { - const Chunks: ArrayBuffer[] = [] - Res.on('data', (Chunk) => { - Chunks.push(Chunk.buffer.slice(Chunk.byteOffset, Chunk.byteOffset + Chunk.byteLength)) - }) - Res.on('end', () => { - const BodyBuffer = ConcatArrayBuffers(Chunks) - let Body: unknown - switch (ExpectedAs) { - case 'JSON': - try { - Body = JSON.parse(new TextDecoder('utf-8').decode(BodyBuffer)) - } catch (Err) { - return Reject(new Error('Failed to parse JSON response body', { cause: Err })) - } - break - case 'String': - Body = new TextDecoder('utf-8').decode(BodyBuffer) - break - case 'ArrayBuffer': - Body = BodyBuffer - break - } - Resolve({ - StatusCode: Res.statusCode ?? 0, - Headers: Res.headers as Record, - Body, - } as HTTPSResponse) - }) - }) - - HTTPSRequestInstance.on('error', (Error) => { - Reject(Error) - }) - - if (MergedOptions.Payload !== undefined) { - if (typeof MergedOptions.Payload === 'string') { - HTTPSRequestInstance.write(MergedOptions.Payload) - } else if (MergedOptions.Payload instanceof ArrayBuffer) { - HTTPSRequestInstance.write(MergedOptions.Payload) - } else if (MergedOptions.Payload instanceof Uint8Array) { - HTTPSRequestInstance.write(MergedOptions.Payload) - } - } - - HTTPSRequestInstance.end() - }) - - return HTTPSResponse -} - -/** - * Perform an HTTP request over TLS using Node's `http` and `tls` modules. - * - * @param {URL} Url - The target URL. Must be an instance of `URL`. - * @param {HTTPSRequestOptions} [Options] - Request options including TLS settings, headers, and `ExpectedAs`. Defaults to secure TLS v1.3 and a default `User-Agent` header. - * @returns {Promise>} Resolves with `{ StatusCode, Headers, Body }`, where `Body` is parsed according to `ExpectedAs`. - * @throws {TypeError} If `Url` is not an instance of `URL`. - * @throws {Error} When the request errors or if parsing the response body (e.g. JSON) fails. - * - * Notes: - * - Uses `node:http` with a custom TLS socket from `node:tls` (HTTP over TLS). - * - TLS options are forwarded to the underlying TLS connection (minVersion, maxVersion, ciphers, ecdhCurve). - */ -export async function HTTPS2Request(Url: URL, Options?: HTTPSRequestOptions): Promise> { - const DefaultOptions = { - TLS: { - IsHTTPSEnforced: true, - MinTLSVersion: 'TLSv1.3', - MaxTLSVersion: 'TLSv1.3', - Ciphers: ['TLS_AES_256_GCM_SHA384', 'TLS_CHACHA20_POLY1305_SHA256'], - KeyExchanges: ['X25519MLKEM768', 'X25519'], - }, - HttpHeaders: { - 'User-Agent': `node/${Process.version} ${Process.platform} ${Process.arch} workspace/false`, - }, - HttpMethod: 'GET' - } as const - - const MergedOptions = { ...DefaultOptions, ...(Options ?? {}) } as HTTPSRequestOptions - if (Url instanceof URL === false) { - throw new TypeError('Url must be an instance of URL') - } - - await Zod.strictObject({ - TLS: Zod.strictObject({ - IsHTTPSEnforced: Zod.boolean().optional(), - MinTLSVersion: Zod.enum(['TLSv1.2', 'TLSv1.3']).optional(), - MaxTLSVersion: Zod.enum(['TLSv1.2', 'TLSv1.3']).optional(), - Ciphers: Zod.array(Zod.string().refine(Cipher => TLS.getCiphers().map(C => C.toLowerCase()).includes(Cipher.toLowerCase()))).optional(), - KeyExchanges: Zod.array(Zod.string()).optional() - }).partial().optional(), - HttpHeaders: Zod.record(Zod.string(), Zod.string()).optional(), - ExpectedAs: Zod.enum(['JSON', 'String', 'ArrayBuffer']).optional(), - HttpMethod: Zod.enum(['GET', 'POST', 'PUT', 'DELETE', 'PATCH', 'HEAD', 'OPTIONS']).optional(), - Payload: Zod.union([Zod.string(), Zod.instanceof(ArrayBuffer), Zod.instanceof(Uint8Array)]).optional() - }).parseAsync(Options ?? {}) - - if (MergedOptions.TLS?.IsHTTPSEnforced && Url.protocol !== 'https:') { - throw new Error('HTTPS is enforced, but the URL protocol is not HTTPS') - } - - if (MergedOptions.Payload && !['GET', 'POST', 'PUT', 'PATCH', 'OPTIONS'].includes(MergedOptions.HttpMethod ?? 'GET')) { - throw new Error('Request payload is only supported for GET, POST, PUT, PATCH, and OPTIONS methods') - } - - const ExpectedAs = (Options?.ExpectedAs ?? (Url.pathname.endsWith('.json') ? 'JSON' : Url.pathname.endsWith('.txt') ? 'String' : 'ArrayBuffer')) as E - - const HTTPSResponse = await new Promise>((Resolve, Reject) => { - const NormalizedHeaders = Object.fromEntries(Object.entries(MergedOptions.HttpHeaders ?? {}).map(([Key, Value]) => [Key.toLowerCase(), Value])) - const HTTP2Session = HTTP2.connect(`https://${Url.hostname}${Url.port ? `:${Url.port}` : ''}`, { - createConnection: () => TLS.connect({ - host: Url.hostname, - port: Number(Url.port || 443), - servername: Url.hostname, - minVersion: MergedOptions.TLS?.MinTLSVersion, - maxVersion: MergedOptions.TLS?.MaxTLSVersion, - ciphers: MergedOptions.TLS?.Ciphers?.join(':'), - ecdhCurve: MergedOptions.TLS?.KeyExchanges?.join(':'), - ALPNProtocols: ['h2'], - }) - }) - - HTTP2Session.on('error', (Error) => { - Reject(Error) - }) - - const RequestHeaders: HTTP2.OutgoingHttpHeaders = { - ':method': MergedOptions.HttpMethod, - ':path': Url.pathname + Url.search, - ':scheme': 'https', - ':authority': Url.host, - ...NormalizedHeaders, - } - - const Request = HTTP2Session.request(RequestHeaders) - const Chunks: ArrayBuffer[] = [] - let StatusCode = 0 - let ResponseHeaders: Record = {} - - Request.on('response', (Headers) => { - StatusCode = Number(Headers[':status'] ?? 0) - ResponseHeaders = Object.fromEntries(Object.entries(Headers) - .filter(([Key]) => !Key.startsWith(':')) - .map(([Key, Value]) => { - if (Array.isArray(Value)) { - return [Key, Value.map(Item => Item?.toString())] - } - return [Key, Value?.toString()] - })) as Record - }) - - Request.on('data', (Chunk) => { - Chunks.push(Chunk.buffer.slice(Chunk.byteOffset, Chunk.byteOffset + Chunk.byteLength)) - }) - - Request.on('end', () => { - const BodyBuffer = ConcatArrayBuffers(Chunks) - let Body: unknown - switch (ExpectedAs) { - case 'JSON': - try { - Body = JSON.parse(new TextDecoder('utf-8').decode(BodyBuffer)) - } catch (Err) { - HTTP2Session.close() - return Reject(new Error('Failed to parse JSON response body', { cause: Err })) - } - break - case 'String': - Body = new TextDecoder('utf-8').decode(BodyBuffer) - break - case 'ArrayBuffer': - Body = BodyBuffer - break - } - HTTP2Session.close() - Resolve({ - StatusCode, - Headers: ResponseHeaders, - Body, - } as HTTPSResponse) - }) - - Request.on('error', (Error) => { - HTTP2Session.close() - Reject(Error) - }) - - if (MergedOptions.Payload !== undefined) { - if (typeof MergedOptions.Payload === 'string') { - Request.write(MergedOptions.Payload) - } else if (MergedOptions.Payload instanceof ArrayBuffer) { - Request.write(MergedOptions.Payload) - } else if (MergedOptions.Payload instanceof Uint8Array) { - Request.write(MergedOptions.Payload) - } - } - - Request.end() - }) - - return HTTPSResponse -} \ No newline at end of file +import { SecureReq } from './secure-req.js' + +export { SecureReq } + +export const SimpleSecureReq = new SecureReq() + +export type { + AutoDetectedResponseBody, + ExpectedAsKey, + ExpectedAsMap, + HTTPCompressionAlgorithm, + HTTPMethod, + HTTPProtocol, + HTTPSRequestOptions, + HTTPSResponse, + OriginCapabilities, + SecureReqOptions, +} from './type.js' \ No newline at end of file diff --git a/sources/request-helpers.ts b/sources/request-helpers.ts new file mode 100644 index 0000000..4ac76d1 --- /dev/null +++ b/sources/request-helpers.ts @@ -0,0 +1,31 @@ +import type { ExpectedAsKey, HTTPMethod, HTTPSRequestOptions } from './type.js' + +export function DetermineExpectedAs(Url: URL, Options: HTTPSRequestOptions): E { + return ( + Options.ExpectedAs + ?? (Url.pathname.endsWith('.json') + ? 'JSON' + : Url.pathname.endsWith('.txt') + ? 'String' + : 'ArrayBuffer') + ) as E +} + +export class HTTP2NegotiationError extends Error { + public constructor(Message: string, Options?: ErrorOptions) { + super(Message, Options) + this.name = 'HTTP2NegotiationError' + } +} + +export function ToError(Value: unknown): Error { + return Value instanceof Error ? Value : new Error(String(Value)) +} + +export function IsHTTP2NegotiationError(Value: unknown): Value is HTTP2NegotiationError { + return Value instanceof HTTP2NegotiationError +} + +export function IsAutomaticHTTP2ProbeMethod(Method?: HTTPMethod): boolean { + return Method === 'GET' || Method === 'HEAD' +} diff --git a/sources/request-schema.ts b/sources/request-schema.ts new file mode 100644 index 0000000..a936bfc --- /dev/null +++ b/sources/request-schema.ts @@ -0,0 +1,53 @@ +import * as Zod from 'zod' +import { AvailableTLSCiphers, DefaultSupportedCompressions } from './constants.js' +import { IsAbortSignal, IsStreamingPayload } from './utils.js' +import type { HTTPSRequestOptions, SecureReqOptions } from './type.js' + +const HTTPCompressionAlgorithmSchema = Zod.enum(DefaultSupportedCompressions) + +const TLSOptionsSchema = Zod.strictObject({ + IsHTTPSEnforced: Zod.boolean(), + MinTLSVersion: Zod.enum(['TLSv1.2', 'TLSv1.3']), + MaxTLSVersion: Zod.enum(['TLSv1.2', 'TLSv1.3']), + Ciphers: Zod.array( + Zod.string().refine(Cipher => AvailableTLSCiphers.has(Cipher.toLowerCase()), 'Unsupported TLS cipher'), + ), + KeyExchanges: Zod.array(Zod.string()), + RejectUnauthorized: Zod.boolean(), +}).partial() + +export const RequestOptionsSchema = Zod.strictObject({ + TLS: TLSOptionsSchema.optional(), + HttpHeaders: Zod.record(Zod.string(), Zod.string()).optional(), + HttpMethod: Zod.enum(['GET', 'POST', 'PUT', 'DELETE', 'PATCH', 'HEAD', 'OPTIONS']).optional(), + Payload: Zod.union([ + Zod.string(), + Zod.instanceof(ArrayBuffer), + Zod.instanceof(Uint8Array), + Zod.custom>(Value => IsStreamingPayload(Value), { + message: 'Payload must be a string, ArrayBuffer, Uint8Array, Readable stream, or AsyncIterable', + }), + ]).optional(), + ExpectedAs: Zod.enum(['JSON', 'String', 'ArrayBuffer', 'Stream']).optional(), + PreferredProtocol: Zod.enum(['auto', 'http/1.1', 'http/2', 'http/3']).optional(), + EnableCompression: Zod.boolean().optional(), + FollowRedirects: Zod.boolean().optional(), + MaxRedirects: Zod.number().finite().int().min(0).optional(), + TimeoutMs: Zod.number().finite().positive().optional(), + Signal: Zod.custom(Value => IsAbortSignal(Value), { + message: 'Signal must be an AbortSignal', + }).optional(), +}) + +export const SecureReqDefaultOptionsSchema = RequestOptionsSchema.omit({ + Payload: true, + ExpectedAs: true, + Signal: true, +}) + +export const SecureReqOptionsSchema = Zod.strictObject({ + DefaultOptions: SecureReqDefaultOptionsSchema.optional(), + SupportedCompressions: Zod.array(HTTPCompressionAlgorithmSchema).optional(), + HTTP2SessionIdleTimeout: Zod.number().finite().positive().optional(), + OriginCapabilityCacheLimit: Zod.number().finite().int().positive().optional(), +}) satisfies Zod.ZodType diff --git a/sources/secure-req.ts b/sources/secure-req.ts new file mode 100644 index 0000000..21f8ead --- /dev/null +++ b/sources/secure-req.ts @@ -0,0 +1,1127 @@ +import * as HTTP from 'node:http' +import * as HTTP2 from 'node:http2' +import * as HTTPS from 'node:https' +import { Readable } from 'node:stream' +import { pipeline } from 'node:stream/promises' +import * as TLS from 'node:tls' +import { + ConnectionSpecificHeaders, + DefaultHTTPHeaders, + DefaultSupportedCompressions, + DefaultTLSOptions, + PayloadEnabledMethods, +} from './constants.js' +import { + DetermineExpectedAs, + HTTP2NegotiationError, + IsAutomaticHTTP2ProbeMethod, + ToError, +} from './request-helpers.js' +import { RequestOptionsSchema, SecureReqOptionsSchema } from './request-schema.js' +import { + CreateDecodedBodyStream, + GetHeaderValue, + GetOriginKey, + GetPayloadByteLength, + IntersectCompressionAlgorithms, + IsStreamingPayload, + NormalizeHeaders, + NormalizeIncomingHeaders, + ParseCompressionAlgorithms, + PayloadChunkToUint8Array, + ReadableToArrayBuffer, + ResolveContentEncoding, + SerializeTLSOptions, + ToReadableStream, +} from './utils.js' +import type { + AutoDetectedResponseBody, + ExpectedAsKey, + ExpectedAsMap, + HTTPCompressionAlgorithm, + HTTPSRequestOptions, + HTTPSResponse, + OriginCapabilities, + SecureReqOptions, +} from './type.js' + +interface FinalizeResponseContext { + Url: URL, + Options: HTTPSRequestOptions, + ExpectedAs: E, + Protocol: 'http/1.1' | 'http/2', + StatusCode: number, + Headers: Record, + ResponseStream: Readable, + RequestedCompressions: HTTPCompressionAlgorithm[] +} + +interface NegotiatedSecureTransport { + Protocol: 'http/1.1' | 'http/2', + Socket: TLS.TLSSocket +} + +interface CachedOriginCapabilities extends OriginCapabilities { + HTTP2Support: 'unknown' | 'supported' | 'unsupported' +} + +export class SecureReq { + private readonly DefaultOptions: Omit + private readonly SupportedCompressions: HTTPCompressionAlgorithm[] + private readonly HTTP2SessionIdleTimeout: number + private readonly OriginCapabilityCacheLimit: number + private readonly OriginCapabilityCache = new Map() + private readonly HTTP2SessionCache = new Map() + private readonly PendingHTTP2SessionCache = new Map>() + + public constructor(Options: SecureReqOptions = {}) { + const ParsedOptions = SecureReqOptionsSchema.parse(Options) + + this.DefaultOptions = { + TLS: { + ...DefaultTLSOptions, + ...(ParsedOptions.DefaultOptions?.TLS ?? {}), + }, + HttpHeaders: { + ...DefaultHTTPHeaders, + ...NormalizeHeaders(ParsedOptions.DefaultOptions?.HttpHeaders), + }, + HttpMethod: ParsedOptions.DefaultOptions?.HttpMethod ?? 'GET', + PreferredProtocol: ParsedOptions.DefaultOptions?.PreferredProtocol ?? 'auto', + EnableCompression: ParsedOptions.DefaultOptions?.EnableCompression ?? true, + FollowRedirects: ParsedOptions.DefaultOptions?.FollowRedirects ?? false, + MaxRedirects: ParsedOptions.DefaultOptions?.MaxRedirects ?? 5, + TimeoutMs: ParsedOptions.DefaultOptions?.TimeoutMs, + } + + this.SupportedCompressions = (ParsedOptions.SupportedCompressions?.length ? ParsedOptions.SupportedCompressions : DefaultSupportedCompressions) + .filter((Value, Index, Values) => Values.indexOf(Value) === Index) + + this.HTTP2SessionIdleTimeout = ParsedOptions.HTTP2SessionIdleTimeout ?? 30_000 + this.OriginCapabilityCacheLimit = ParsedOptions.OriginCapabilityCacheLimit ?? 256 + } + + public async Request(Url: URL): Promise> + public async Request(Url: URL, Options: Omit & { ExpectedAs?: undefined }): Promise> + public async Request(Url: URL, Options: HTTPSRequestOptions & { ExpectedAs: E }): Promise> + public async Request(Url: URL, Options?: HTTPSRequestOptions): Promise> { + return await this.RequestInternal(Url, Options, 0) + } + + private async RequestInternal( + Url: URL, + Options: HTTPSRequestOptions | undefined, + RedirectCount: number, + ): Promise> { + if (Url instanceof URL === false) { + throw new TypeError('Url must be an instance of URL') + } + + RequestOptionsSchema.parse(Options ?? {}) + + const MergedOptions = this.MergeOptions(Options) + const ExpectedAs = DetermineExpectedAs(Url, MergedOptions) + this.ValidateRequest(Url, MergedOptions) + + const Protocol = this.ResolveTransportProtocol(Url, MergedOptions) + const PreconnectedTransport = this.ShouldNegotiateSecureTransport(Url, MergedOptions, Protocol) + ? await this.NegotiateSecureTransport(Url, MergedOptions) + : undefined + + if (PreconnectedTransport?.Protocol === 'http/1.1') { + if (MergedOptions.PreferredProtocol === 'http/2' || MergedOptions.PreferredProtocol === 'http/3') { + PreconnectedTransport.Socket.destroy() + throw new HTTP2NegotiationError('Origin did not negotiate HTTP/2 via ALPN') + } + + this.MarkOriginAsHTTP1Only(Url) + return await this.RequestWithHTTP1(Url, MergedOptions, ExpectedAs, RedirectCount, PreconnectedTransport.Socket) + } + + if (Protocol !== 'http/2') { + return await this.RequestWithHTTP1(Url, MergedOptions, ExpectedAs, RedirectCount) + } + + return await this.RequestWithHTTP2(Url, MergedOptions, ExpectedAs, RedirectCount, PreconnectedTransport?.Socket) + } + + public GetOriginCapabilities(Url: URL): OriginCapabilities | undefined { + const Capabilities = this.GetCachedOriginCapabilities(Url) + + if (Capabilities === undefined) { + return undefined + } + + return { + Origin: Capabilities.Origin, + ProbeCompleted: Capabilities.ProbeCompleted, + PreferredProtocol: Capabilities.PreferredProtocol, + SupportedCompressions: [...Capabilities.SupportedCompressions], + HTTP3Advertised: Capabilities.HTTP3Advertised, + } + } + + public Close(): void { + for (const Session of this.HTTP2SessionCache.values()) { + Session.close() + } + + this.PendingHTTP2SessionCache.clear() + this.HTTP2SessionCache.clear() + } + + private MergeOptions(Options?: HTTPSRequestOptions): HTTPSRequestOptions { + return { + ...this.DefaultOptions, + ...Options, + TLS: { + ...this.DefaultOptions.TLS, + ...(Options?.TLS ?? {}), + }, + HttpHeaders: { + ...this.DefaultOptions.HttpHeaders, + ...NormalizeHeaders(Options?.HttpHeaders), + }, + HttpMethod: Options?.HttpMethod ?? this.DefaultOptions.HttpMethod, + PreferredProtocol: Options?.PreferredProtocol ?? this.DefaultOptions.PreferredProtocol, + EnableCompression: Options?.EnableCompression ?? this.DefaultOptions.EnableCompression, + FollowRedirects: Options?.FollowRedirects ?? this.DefaultOptions.FollowRedirects, + MaxRedirects: Options?.MaxRedirects ?? this.DefaultOptions.MaxRedirects, + TimeoutMs: Options?.TimeoutMs ?? this.DefaultOptions.TimeoutMs, + Payload: Options?.Payload, + ExpectedAs: Options?.ExpectedAs, + Signal: Options?.Signal, + } + } + + private ValidateRequest(Url: URL, Options: HTTPSRequestOptions): void { + if (Url.protocol !== 'http:' && Url.protocol !== 'https:') { + throw new Error(`Unsupported URL protocol: ${Url.protocol}`) + } + + if (Options.TLS?.IsHTTPSEnforced !== false && Url.protocol !== 'https:') { + throw new Error('HTTPS is enforced, but the URL protocol is not HTTPS') + } + + if ((Options.PreferredProtocol === 'http/2' || Options.PreferredProtocol === 'http/3') && Url.protocol !== 'https:') { + throw new Error('http/2 and http/3 negotiation require an HTTPS URL') + } + + if (Options.Payload !== undefined && PayloadEnabledMethods.has(Options.HttpMethod ?? 'GET') === false) { + throw new Error('Request payload is only supported for GET, POST, PUT, PATCH, and OPTIONS methods') + } + } + + private ResolveTransportProtocol(Url: URL, Options: HTTPSRequestOptions): 'http/1.1' | 'http/2' { + if (Url.protocol !== 'https:') { + return 'http/1.1' + } + + switch (Options.PreferredProtocol) { + case 'http/1.1': + return 'http/1.1' + case 'http/2': + return 'http/2' + case 'http/3': + return 'http/2' + default: + break + } + + const OriginCapabilities = this.GetCachedOriginCapabilities(Url) + if (OriginCapabilities?.ProbeCompleted !== true) { + return 'http/1.1' + } + + if (OriginCapabilities.HTTP2Support === 'supported') { + return 'http/2' + } + + if (OriginCapabilities.HTTP2Support === 'unsupported') { + return 'http/1.1' + } + + if (this.CanAttemptAutomaticHTTP2Probe(Options)) { + return 'http/2' + } + + return 'http/1.1' + } + + private BuildRequestHeaders(Url: URL, Options: HTTPSRequestOptions): { + Headers: Record, + RequestedCompressions: HTTPCompressionAlgorithm[] + } { + const Headers = { + ...(Options.HttpHeaders ?? {}), + } + + if (Options.EnableCompression !== false && Headers['accept-encoding'] === undefined) { + const AcceptedCompressions = this.GetPreferredCompressions(Url) + if (AcceptedCompressions.length > 0) { + Headers['accept-encoding'] = AcceptedCompressions.join(', ') + } + } + + if (Options.Payload !== undefined && IsStreamingPayload(Options.Payload) === false && Headers['content-length'] === undefined) { + Headers['content-length'] = String(GetPayloadByteLength(Options.Payload)) + } + + return { + Headers, + RequestedCompressions: ParseCompressionAlgorithms(Headers['accept-encoding']), + } + } + + private GetPreferredCompressions(Url: URL): HTTPCompressionAlgorithm[] { + const OriginCapabilities = this.GetCachedOriginCapabilities(Url) + if (OriginCapabilities?.SupportedCompressions.length) { + return [...OriginCapabilities.SupportedCompressions] + } + + return [...this.SupportedCompressions] + } + + private async RequestWithHTTP1( + Url: URL, + Options: HTTPSRequestOptions, + ExpectedAs: E, + RedirectCount: number, + PreconnectedSocket?: TLS.TLSSocket, + ): Promise> { + const { Headers, RequestedCompressions } = this.BuildRequestHeaders(Url, Options) + + return await new Promise>((Resolve, Reject) => { + let Settled = false + let CleanupCancellation = () => {} + const CancellationTarget: { Cancel: (Cause: Error) => void } = { + Cancel: Cause => { + void Cause + }, + } + + const ResolveOnce = (Value: HTTPSResponse) => { + if (Settled === false) { + Settled = true + Resolve(Value) + } + } + + const RejectOnce = (Error: unknown) => { + if (Settled === false) { + Settled = true + CleanupCancellation() + Reject(this.EnhanceTransportError(Error, Options)) + } + } + + const Request = this.CreateHTTP1Request(Url, Options, Headers, Response => { + const ResponseHeaders = NormalizeIncomingHeaders(Response.headers as Record) + const StatusCode = Response.statusCode ?? 0 + + if (this.ShouldFollowRedirect(StatusCode, ResponseHeaders, Options)) { + CleanupCancellation() + this.UpdateOriginCapabilities(Url, 'http/1.1', ResponseHeaders, RequestedCompressions) + this.DiscardResponseStream(Response) + void this.FollowRedirect(Url, Options, StatusCode, ResponseHeaders, RedirectCount) + .then(ResponseValue => { + ResolveOnce(ResponseValue as HTTPSResponse) + }, RejectOnce) + return + } + + void this.FinalizeResponse({ + Url, + Options, + ExpectedAs, + Protocol: 'http/1.1', + StatusCode, + Headers: ResponseHeaders, + ResponseStream: Response, + RequestedCompressions, + }).then(ResponseValue => { + if (ExpectedAs === 'Stream') { + const ResponseBody = ResponseValue.Body as ExpectedAsMap['Stream'] + + CancellationTarget.Cancel = Cause => { + ResponseBody.destroy(Cause) + Request.destroy(Cause) + RejectOnce(Cause) + } + + this.BindRequestCleanupToResponseStream(ResponseBody, CleanupCancellation) + ResolveOnce(ResponseValue) + return + } + + CleanupCancellation() + ResolveOnce(ResponseValue) + }, RejectOnce) + }, PreconnectedSocket) + + const CancelRequest = (Cause: Error) => { + Request.destroy(Cause) + RejectOnce(Cause) + } + + CancellationTarget.Cancel = CancelRequest + CleanupCancellation = this.AttachRequestCancellation(Options, Cause => { + CancellationTarget.Cancel(Cause) + }) + + Request.once('error', RejectOnce) + + void this.WritePayload(Request, Options.Payload).catch(Cause => { + Request.destroy(ToError(Cause)) + RejectOnce(Cause) + }) + }) + } + + private CreateHTTP1Request( + Url: URL, + Options: HTTPSRequestOptions, + Headers: Record, + OnResponse: (Response: HTTP.IncomingMessage) => void, + PreconnectedSocket?: TLS.TLSSocket, + ): HTTP.ClientRequest { + const BaseOptions = { + protocol: Url.protocol, + hostname: Url.hostname, + port: Url.port || undefined, + path: Url.pathname + Url.search, + headers: Headers, + method: Options.HttpMethod, + } + + if (Url.protocol === 'https:') { + const Agent = PreconnectedSocket ? new HTTPS.Agent({ keepAlive: false }) : undefined + + if (Agent) { + Agent.createConnection = () => PreconnectedSocket + } + + return HTTPS.request({ + ...BaseOptions, + agent: Agent, + servername: Url.hostname, + minVersion: Options.TLS?.MinTLSVersion, + maxVersion: Options.TLS?.MaxTLSVersion, + ciphers: Options.TLS?.Ciphers?.join(':'), + ecdhCurve: Options.TLS?.KeyExchanges?.join(':'), + rejectUnauthorized: Options.TLS?.RejectUnauthorized, + }, Response => { + Response.once('close', () => { + Agent?.destroy() + }) + OnResponse(Response) + }).once('error', () => { + Agent?.destroy() + }) + } + + return HTTP.request(BaseOptions, OnResponse) + } + + private async RequestWithHTTP2( + Url: URL, + Options: HTTPSRequestOptions, + ExpectedAs: E, + RedirectCount: number, + PreconnectedSocket?: TLS.TLSSocket, + ): Promise> { + const { Headers, RequestedCompressions } = this.BuildRequestHeaders(Url, Options) + const Session = await this.GetOrCreateHTTP2Session(Url, Options, PreconnectedSocket) + let Request: HTTP2.ClientHttp2Stream + + try { + Request = Session.request({ + ':method': Options.HttpMethod, + ':path': Url.pathname + Url.search, + ':scheme': 'https', + ':authority': Headers.host ?? Url.host, + ...this.FilterHTTP2Headers(Headers), + }) + } catch (Cause) { + throw new HTTP2NegotiationError('Failed to start HTTP/2 request', { cause: Cause }) + } + + return await new Promise>((Resolve, Reject) => { + let Settled = false + let CleanupCancellation = () => {} + const CancellationTarget: { Cancel: (Cause: Error) => void } = { + Cancel: Cause => { + void Cause + }, + } + + const ResolveOnce = (Value: HTTPSResponse) => { + if (Settled === false) { + Settled = true + Resolve(Value) + } + } + + const RejectOnce = (Error: unknown) => { + if (Settled === false) { + Settled = true + CleanupCancellation() + this.InvalidateHTTP2Session(Url, Options, Session) + Reject(this.EnhanceTransportError(Error, Options)) + } + } + + Request.once('response', ResponseHeaders => { + const NormalizedHeaders = NormalizeIncomingHeaders(ResponseHeaders as Record) + const StatusCode = Number(ResponseHeaders[':status'] ?? 0) + + if (this.ShouldFollowRedirect(StatusCode, NormalizedHeaders, Options)) { + CleanupCancellation() + this.UpdateOriginCapabilities(Url, 'http/2', NormalizedHeaders, RequestedCompressions) + this.DiscardResponseStream(Request) + void this.FollowRedirect(Url, Options, StatusCode, NormalizedHeaders, RedirectCount) + .then(ResponseValue => { + ResolveOnce(ResponseValue as HTTPSResponse) + }, RejectOnce) + return + } + + void this.FinalizeResponse({ + Url, + Options, + ExpectedAs, + Protocol: 'http/2', + StatusCode, + Headers: NormalizedHeaders, + ResponseStream: Request, + RequestedCompressions, + }).then(ResponseValue => { + if (ExpectedAs === 'Stream') { + const ResponseBody = ResponseValue.Body as ExpectedAsMap['Stream'] + + CancellationTarget.Cancel = Cause => { + ResponseBody.destroy(Cause) + + if (ResponseBody !== Request) { + Request.destroy(Cause) + } + + RejectOnce(Cause) + } + + this.BindRequestCleanupToResponseStream(ResponseBody, CleanupCancellation) + ResolveOnce(ResponseValue) + return + } + + CleanupCancellation() + ResolveOnce(ResponseValue) + }, RejectOnce) + }) + + const CancelRequest = (Cause: Error) => { + Request.destroy(Cause) + RejectOnce(Cause) + } + + CancellationTarget.Cancel = CancelRequest + CleanupCancellation = this.AttachRequestCancellation(Options, Cause => { + CancellationTarget.Cancel(Cause) + }) + + Request.once('error', RejectOnce) + + void this.WritePayload(Request, Options.Payload).catch(Cause => { + Request.destroy(ToError(Cause)) + RejectOnce(Cause) + }) + }) + } + + private async GetOrCreateHTTP2Session( + Url: URL, + Options: HTTPSRequestOptions, + PreconnectedSocket?: TLS.TLSSocket, + ): Promise { + const SessionKey = this.GetHTTP2SessionKey(Url, Options) + const PendingSession = this.PendingHTTP2SessionCache.get(SessionKey) + + if (PendingSession) { + return await PendingSession + } + + const ExistingSession = this.HTTP2SessionCache.get(SessionKey) + + if (ExistingSession && ExistingSession.closed === false && ExistingSession.destroyed === false) { + return ExistingSession + } + + const SessionPromise = (async () => { + const NegotiatedTransport = PreconnectedSocket + ? { + Protocol: 'http/2', + Socket: PreconnectedSocket, + } satisfies NegotiatedSecureTransport + : await this.NegotiateSecureTransport(Url, Options) + + if (NegotiatedTransport.Protocol !== 'http/2') { + NegotiatedTransport.Socket.destroy() + throw new HTTP2NegotiationError('Origin did not negotiate HTTP/2 via ALPN') + } + + const Session = HTTP2.connect(GetOriginKey(Url), { + createConnection: () => NegotiatedTransport.Socket, + }) + + this.ConfigureHTTP2Session(SessionKey, Session) + this.HTTP2SessionCache.set(SessionKey, Session) + + return await new Promise((Resolve, Reject) => { + let Connected = false + + const Cleanup = () => { + Session.off('connect', HandleConnect) + Session.off('error', HandleError) + Session.off('close', HandleClose) + } + + const HandleConnect = () => { + Connected = true + Cleanup() + Resolve(Session) + } + + const HandleError = (Cause: unknown) => { + Cleanup() + this.InvalidateHTTP2Session(Url, Options, Session) + Reject( + Connected + ? this.EnhanceTransportError(Cause, Options) + : (Cause instanceof HTTP2NegotiationError + ? Cause + : new HTTP2NegotiationError('Failed to establish HTTP/2 session', { cause: this.EnhanceTransportError(Cause, Options) })), + ) + } + + const HandleClose = () => { + Cleanup() + Reject( + Connected + ? new Error('HTTP/2 session closed before it became ready') + : new HTTP2NegotiationError('HTTP/2 session negotiation closed before it became ready'), + ) + } + + Session.once('connect', HandleConnect) + Session.once('error', HandleError) + Session.once('close', HandleClose) + }) + })() + + this.PendingHTTP2SessionCache.set(SessionKey, SessionPromise) + + try { + return await SessionPromise + } finally { + if (this.PendingHTTP2SessionCache.get(SessionKey) === SessionPromise) { + this.PendingHTTP2SessionCache.delete(SessionKey) + } + } + } + + private GetHTTP2SessionKey(Url: URL, Options: HTTPSRequestOptions): string { + return `${GetOriginKey(Url)}|${SerializeTLSOptions(Options.TLS)}` + } + + private InvalidateHTTP2Session(Url: URL, Options: HTTPSRequestOptions, Session?: HTTP2.ClientHttp2Session): void { + const SessionKey = this.GetHTTP2SessionKey(Url, Options) + const SessionToClose = Session ?? this.HTTP2SessionCache.get(SessionKey) + + this.PendingHTTP2SessionCache.delete(SessionKey) + this.HTTP2SessionCache.delete(SessionKey) + + if (SessionToClose && SessionToClose.closed === false && SessionToClose.destroyed === false) { + SessionToClose.close() + } + } + + private FilterHTTP2Headers(Headers: Record): HTTP2.OutgoingHttpHeaders { + return Object.fromEntries( + Object.entries(Headers).filter(([Key]) => ConnectionSpecificHeaders.has(Key) === false), + ) + } + + private async WritePayload(Request: NodeJS.WritableStream, Payload?: HTTPSRequestOptions['Payload']): Promise { + if (Payload === undefined) { + Request.end() + return + } + + if (IsStreamingPayload(Payload)) { + await pipeline(ToReadableStream(Payload), Request) + return + } + + Request.end(Buffer.from(PayloadChunkToUint8Array(Payload))) + } + + private async FinalizeResponse(Context: FinalizeResponseContext): Promise> { + this.UpdateOriginCapabilities(Context.Url, Context.Protocol, Context.Headers, Context.RequestedCompressions) + + let ResponseStream = Context.ResponseStream + let ContentEncoding: HTTPCompressionAlgorithm | 'identity' = 'identity' + let DecodedBody = false + + if (Context.Options.EnableCompression !== false) { + const DecodedResponse = CreateDecodedBodyStream(ResponseStream, GetHeaderValue(Context.Headers, 'content-encoding')) + ResponseStream = DecodedResponse.Stream + ContentEncoding = DecodedResponse.ContentEncoding + DecodedBody = DecodedResponse.DecodedBody + } else { + ContentEncoding = ResolveContentEncoding(GetHeaderValue(Context.Headers, 'content-encoding')) + } + + const Headers = DecodedBody + ? { + ...Context.Headers, + 'content-encoding': undefined, + 'content-length': undefined, + } + : Context.Headers + + if (Context.ExpectedAs === 'Stream') { + return { + StatusCode: Context.StatusCode, + Headers, + Body: ResponseStream as ExpectedAsMap[E], + Protocol: Context.Protocol, + ContentEncoding, + DecodedBody, + } + } + + const BodyBuffer = await ReadableToArrayBuffer(ResponseStream) + let Body: ExpectedAsMap[E] + + switch (Context.ExpectedAs) { + case 'JSON': + try { + Body = JSON.parse(new TextDecoder('utf-8').decode(BodyBuffer)) as ExpectedAsMap[E] + } catch (Cause) { + throw new Error('Failed to parse JSON response body', { cause: Cause }) + } + break + case 'String': + Body = new TextDecoder('utf-8').decode(BodyBuffer) as ExpectedAsMap[E] + break + case 'ArrayBuffer': + default: + Body = BodyBuffer as ExpectedAsMap[E] + break + } + + return { + StatusCode: Context.StatusCode, + Headers, + Body, + Protocol: Context.Protocol, + ContentEncoding, + DecodedBody, + } + } + + private UpdateOriginCapabilities( + Url: URL, + Protocol: 'http/1.1' | 'http/2', + Headers: Record, + RequestedCompressions: HTTPCompressionAlgorithm[], + ): void { + const Origin = GetOriginKey(Url) + const ExistingCapabilities = this.GetCachedOriginCapabilities(Origin) + const NegotiatedCompressions = this.ResolveNegotiatedCompressions(Headers, RequestedCompressions) + const HTTP2Support = Protocol === 'http/2' + ? 'supported' + : (ExistingCapabilities?.HTTP2Support ?? 'unknown') + const HTTP3Advertised = this.IsHTTP3Advertised(Headers) || (ExistingCapabilities?.HTTP3Advertised ?? false) + + this.SetOriginCapabilities({ + Origin, + ProbeCompleted: true, + PreferredProtocol: Url.protocol === 'https:' && HTTP2Support === 'supported' ? 'http/2' : 'http/1.1', + SupportedCompressions: NegotiatedCompressions !== undefined + ? NegotiatedCompressions + : [...(ExistingCapabilities?.SupportedCompressions ?? [])], + HTTP3Advertised, + HTTP2Support, + }) + } + + private ResolveNegotiatedCompressions( + Headers: Record, + RequestedCompressions: HTTPCompressionAlgorithm[], + ): HTTPCompressionAlgorithm[] | undefined { + const ServerAcceptEncoding = ParseCompressionAlgorithms(GetHeaderValue(Headers, 'accept-encoding')) + if (ServerAcceptEncoding.length > 0) { + return IntersectCompressionAlgorithms(RequestedCompressions, ServerAcceptEncoding) + } + + const ContentEncoding = ParseCompressionAlgorithms(GetHeaderValue(Headers, 'content-encoding')) + if (ContentEncoding.length > 0) { + return IntersectCompressionAlgorithms(RequestedCompressions, ContentEncoding) + } + + return undefined + } + + private IsHTTP3Advertised(Headers: Record): boolean { + const AltSvcHeader = GetHeaderValue(Headers, 'alt-svc') + return /\bh3(?:-\d+)?\s*=/.test(AltSvcHeader ?? '') + } + + private MarkOriginAsHTTP1Only(Url: URL): void { + const Origin = GetOriginKey(Url) + const ExistingCapabilities = this.GetCachedOriginCapabilities(Origin) + + this.SetOriginCapabilities({ + Origin, + ProbeCompleted: true, + PreferredProtocol: 'http/1.1', + SupportedCompressions: [...(ExistingCapabilities?.SupportedCompressions ?? [])], + HTTP3Advertised: ExistingCapabilities?.HTTP3Advertised ?? false, + HTTP2Support: 'unsupported', + }) + } + + private GetCachedOriginCapabilities(UrlOrOrigin: URL | string): CachedOriginCapabilities | undefined { + const Origin = typeof UrlOrOrigin === 'string' ? UrlOrOrigin : GetOriginKey(UrlOrOrigin) + const Capabilities = this.OriginCapabilityCache.get(Origin) + + if (Capabilities === undefined) { + return undefined + } + + this.OriginCapabilityCache.delete(Origin) + this.OriginCapabilityCache.set(Origin, Capabilities) + return Capabilities + } + + private SetOriginCapabilities(Capabilities: CachedOriginCapabilities): void { + if (this.OriginCapabilityCache.has(Capabilities.Origin)) { + this.OriginCapabilityCache.delete(Capabilities.Origin) + } + + this.OriginCapabilityCache.set(Capabilities.Origin, Capabilities) + + while (this.OriginCapabilityCache.size > this.OriginCapabilityCacheLimit) { + const OldestOrigin = this.OriginCapabilityCache.keys().next().value + + if (OldestOrigin === undefined) { + break + } + + this.OriginCapabilityCache.delete(OldestOrigin) + } + } + + private ConfigureHTTP2Session(SessionKey: string, Session: HTTP2.ClientHttp2Session): void { + Session.setTimeout(this.HTTP2SessionIdleTimeout, () => { + Session.close() + }) + + if (typeof Session.unref === 'function') { + Session.unref() + } + + Session.on('close', () => { + if (this.HTTP2SessionCache.get(SessionKey) === Session) { + this.HTTP2SessionCache.delete(SessionKey) + } + + this.PendingHTTP2SessionCache.delete(SessionKey) + }) + + Session.on('error', () => { + if (Session.closed || Session.destroyed) { + this.HTTP2SessionCache.delete(SessionKey) + this.PendingHTTP2SessionCache.delete(SessionKey) + } + }) + + Session.on('goaway', () => { + this.HTTP2SessionCache.delete(SessionKey) + this.PendingHTTP2SessionCache.delete(SessionKey) + }) + } + + private EnhanceTransportError(Cause: unknown, Options: HTTPSRequestOptions): Error { + const TransportError = ToError(Cause) + const NormalizedMessage = TransportError.message.toLowerCase() + const UsesExplicitTLS12 = Options.TLS?.MinTLSVersion === 'TLSv1.2' && Options.TLS?.MaxTLSVersion === 'TLSv1.2' + const UsesECDSACipher = Options.TLS?.Ciphers?.some(Cipher => Cipher.toUpperCase().includes('ECDHE-ECDSA-')) ?? false + + if ( + UsesExplicitTLS12 + && Options.TLS?.KeyExchanges?.length + && ( + NormalizedMessage.includes('wrong curve') + || (UsesECDSACipher && NormalizedMessage.includes('handshake failure')) + ) + ) { + return new Error( + `TLS handshake failed while using TLSv1.2 with KeyExchanges (${Options.TLS.KeyExchanges.join(', ')}). This commonly means the server certificate needs a compatible curve such as P-256 in addition to X25519. Add P-256, remove the KeyExchanges restriction, or allow TLSv1.3.`, + { cause: TransportError }, + ) + } + + return TransportError + } + + private async NegotiateSecureTransport(Url: URL, Options: HTTPSRequestOptions): Promise { + return await new Promise((Resolve, Reject) => { + const Socket = TLS.connect({ + host: Url.hostname, + port: Number(Url.port || 443), + servername: Url.hostname, + minVersion: Options.TLS?.MinTLSVersion, + maxVersion: Options.TLS?.MaxTLSVersion, + ciphers: Options.TLS?.Ciphers?.join(':'), + ecdhCurve: Options.TLS?.KeyExchanges?.join(':'), + rejectUnauthorized: Options.TLS?.RejectUnauthorized, + ALPNProtocols: ['h2', 'http/1.1'], + }) + + const Cleanup = () => { + Socket.off('secureConnect', HandleSecureConnect) + Socket.off('error', HandleError) + Socket.off('close', HandleClose) + } + + const RejectWithNegotiationError = (Message: string, Cause?: unknown) => { + Cleanup() + + if (Socket.destroyed === false) { + Socket.destroy() + } + + Reject(new HTTP2NegotiationError(Message, Cause === undefined ? undefined : { cause: Cause })) + } + + const HandleSecureConnect = () => { + Cleanup() + Resolve({ + Protocol: Socket.alpnProtocol === 'h2' ? 'http/2' : 'http/1.1', + Socket, + }) + } + + const HandleError = (Cause: unknown) => { + RejectWithNegotiationError('Failed to negotiate HTTP/2 session', this.EnhanceTransportError(Cause, Options)) + } + + const HandleClose = () => { + RejectWithNegotiationError('HTTP/2 session negotiation closed before it became ready') + } + + Socket.once('secureConnect', HandleSecureConnect) + Socket.once('error', HandleError) + Socket.once('close', HandleClose) + }) + } + + private HasReusableHTTP2Session(Url: URL, Options: HTTPSRequestOptions): boolean { + const Session = this.HTTP2SessionCache.get(this.GetHTTP2SessionKey(Url, Options)) + return Session !== undefined && Session.closed === false && Session.destroyed === false + } + + private ShouldNegotiateSecureTransport( + Url: URL, + Options: HTTPSRequestOptions, + Protocol: 'http/1.1' | 'http/2', + ): boolean { + if (Url.protocol !== 'https:' || Protocol !== 'http/2') { + return false + } + + return this.HasReusableHTTP2Session(Url, Options) === false + } + + private CanAttemptAutomaticHTTP2Probe(Options: HTTPSRequestOptions): boolean { + return IsAutomaticHTTP2ProbeMethod(Options.HttpMethod) + && Options.Payload === undefined + && Options.PreferredProtocol === 'auto' + } + + private ShouldFollowRedirect( + StatusCode: number, + Headers: Record, + Options: HTTPSRequestOptions, + ): boolean { + return Options.FollowRedirects === true + && StatusCode >= 300 + && StatusCode <= 308 + && GetHeaderValue(Headers, 'location') !== undefined + } + + private async FollowRedirect( + Url: URL, + Options: HTTPSRequestOptions, + StatusCode: number, + Headers: Record, + RedirectCount: number, + ): Promise> { + const LocationHeader = GetHeaderValue(Headers, 'location') + + if (LocationHeader === undefined) { + throw new Error('Redirect response did not include a location header') + } + + const MaxRedirects = Options.MaxRedirects ?? 5 + if (RedirectCount >= MaxRedirects) { + throw new Error(`Maximum redirect limit exceeded (${MaxRedirects})`) + } + + const RedirectUrl = new URL(LocationHeader, Url) + const RedirectOptions = this.BuildRedirectOptions(Url, RedirectUrl, Options, StatusCode) + + return await this.RequestInternal(RedirectUrl, RedirectOptions, RedirectCount + 1) + } + + private BuildRedirectOptions( + Url: URL, + RedirectUrl: URL, + Options: HTTPSRequestOptions, + StatusCode: number, + ): HTTPSRequestOptions { + const OriginalMethod = Options.HttpMethod ?? 'GET' + const RedirectMethod = this.ResolveRedirectMethod(StatusCode, OriginalMethod) + const DropPayload = RedirectMethod !== OriginalMethod || RedirectMethod === 'GET' + + if (DropPayload === false && Options.Payload !== undefined && IsStreamingPayload(Options.Payload)) { + throw new Error('Cannot automatically follow redirects that require replaying a streaming payload') + } + + const RedirectHeaders = { + ...(Options.HttpHeaders ?? {}), + } + + delete RedirectHeaders.host + delete RedirectHeaders['content-length'] + delete RedirectHeaders['transfer-encoding'] + + if (DropPayload) { + delete RedirectHeaders['content-type'] + } + + if (GetOriginKey(Url) !== GetOriginKey(RedirectUrl)) { + delete RedirectHeaders.authorization + delete RedirectHeaders.cookie + delete RedirectHeaders['proxy-authorization'] + } + + return { + ...Options, + HttpMethod: RedirectMethod, + Payload: DropPayload ? undefined : Options.Payload, + HttpHeaders: RedirectHeaders, + } + } + + private ResolveRedirectMethod(StatusCode: number, Method: HTTPSRequestOptions['HttpMethod']): HTTPSRequestOptions['HttpMethod'] { + if (StatusCode === 303 && Method !== 'HEAD') { + return 'GET' + } + + if ((StatusCode === 301 || StatusCode === 302) && Method === 'POST') { + return 'GET' + } + + return Method + } + + private DiscardResponseStream(Stream: Readable): void { + Stream.resume() + } + + private AttachRequestCancellation( + Options: HTTPSRequestOptions, + Cancel: (Cause: Error) => void, + ): () => void { + const CleanupCallbacks: Array<() => void> = [] + + if (Options.TimeoutMs !== undefined) { + const Timer = setTimeout(() => { + Cancel(this.CreateTimeoutError(Options.TimeoutMs ?? 0)) + }, Options.TimeoutMs) + + if (typeof Timer.unref === 'function') { + Timer.unref() + } + + CleanupCallbacks.push(() => { + clearTimeout(Timer) + }) + } + + if (Options.Signal) { + const Signal = Options.Signal + const HandleAbort = () => { + Cancel(this.CreateAbortError(Signal.reason)) + } + + if (Signal.aborted) { + queueMicrotask(HandleAbort) + } else { + Signal.addEventListener('abort', HandleAbort, { once: true }) + CleanupCallbacks.push(() => { + Signal.removeEventListener('abort', HandleAbort) + }) + } + } + + return () => { + for (const Cleanup of CleanupCallbacks.splice(0)) { + Cleanup() + } + } + } + + private BindRequestCleanupToResponseStream(Stream: ExpectedAsMap['Stream'], Cleanup: () => void): void { + if (Stream.destroyed || Stream.readableEnded) { + Cleanup() + return + } + + let CleanedUp = false + + const CleanupOnce = () => { + if (CleanedUp) { + return + } + + CleanedUp = true + Stream.off('close', CleanupOnce) + Stream.off('end', CleanupOnce) + Stream.off('error', CleanupOnce) + Cleanup() + } + + Stream.once('close', CleanupOnce) + Stream.once('end', CleanupOnce) + Stream.once('error', CleanupOnce) + } + + private CreateAbortError(Cause?: unknown): Error { + const RequestError = Cause === undefined + ? new Error('Request was aborted') + : new Error('Request was aborted', { cause: Cause }) + + RequestError.name = 'AbortError' + return RequestError + } + + private CreateTimeoutError(TimeoutMs: number): Error { + const RequestError = new Error(`Request timed out after ${TimeoutMs}ms`) + RequestError.name = 'TimeoutError' + return RequestError + } +} diff --git a/sources/type.ts b/sources/type.ts index c8d2894..b843806 100644 --- a/sources/type.ts +++ b/sources/type.ts @@ -1,28 +1,67 @@ -interface TLSOptions { +import type { Readable } from 'node:stream' + +export type HTTPCompressionAlgorithm = 'zstd' | 'gzip' | 'deflate' +export type HTTPProtocol = 'http/1.1' | 'http/2' | 'http/3' +export type HTTPMethod = 'GET' | 'POST' | 'PUT' | 'DELETE' | 'PATCH' | 'HEAD' | 'OPTIONS' +export type HTTPProtocolPreference = 'auto' | HTTPProtocol +export type AutoDetectedResponseBody = unknown + +export interface TLSOptions { IsHTTPSEnforced?: boolean, MinTLSVersion?: 'TLSv1.2' | 'TLSv1.3', MaxTLSVersion?: 'TLSv1.2' | 'TLSv1.3', Ciphers?: string[], - KeyExchanges?: string[] -} + KeyExchanges?: string[], + RejectUnauthorized?: boolean +} + +export type HTTPSRequestPayloadChunk = string | ArrayBuffer | Uint8Array +export type HTTPSRequestPayloadStream = NodeJS.ReadableStream | AsyncIterable +export type HTTPSRequestPayload = HTTPSRequestPayloadChunk | HTTPSRequestPayloadStream export interface HTTPSRequestOptions { TLS?: TLSOptions, HttpHeaders?: Record, - HttpMethod?: 'GET' | 'POST' | 'PUT' | 'DELETE' | 'PATCH' | 'HEAD' | 'OPTIONS', - Payload?: string | ArrayBuffer | Uint8Array, - ExpectedAs?: E + HttpMethod?: HTTPMethod, + Payload?: HTTPSRequestPayload, + ExpectedAs?: E, + PreferredProtocol?: HTTPProtocolPreference, + EnableCompression?: boolean, + FollowRedirects?: boolean, + MaxRedirects?: number, + TimeoutMs?: number, + Signal?: AbortSignal } -export interface HTTPSResponse { +export interface SecureReqOptions { + DefaultOptions?: Omit, + SupportedCompressions?: HTTPCompressionAlgorithm[], + HTTP2SessionIdleTimeout?: number + OriginCapabilityCacheLimit?: number +} + +export interface HTTPSResponse { StatusCode: number, Headers: Record, - Body: T + Body: T, + Protocol: 'http/1.1' | 'http/2', + ContentEncoding: HTTPCompressionAlgorithm | 'identity', + DecodedBody: boolean +} + +export interface OriginCapabilities { + Origin: string, + ProbeCompleted: boolean, + PreferredProtocol: HTTPProtocol, + SupportedCompressions: HTTPCompressionAlgorithm[], + HTTP3Advertised: boolean } export type ExpectedAsMap = { - 'JSON': unknown, - 'String': string, - 'ArrayBuffer': ArrayBuffer + JSON: unknown, + String: string, + ArrayBuffer: ArrayBuffer, + Stream: Readable } -export type ExpectedAsKey = keyof ExpectedAsMap \ No newline at end of file + +export type ExpectedAsKey = keyof ExpectedAsMap diff --git a/sources/utils.ts b/sources/utils.ts index cebbc66..c27c62e 100644 --- a/sources/utils.ts +++ b/sources/utils.ts @@ -1,6 +1,9 @@ +import { Readable } from 'node:stream' +import * as ZLib from 'node:zlib' +import type { HTTPCompressionAlgorithm, HTTPSRequestPayload, HTTPSRequestPayloadChunk, TLSOptions } from './type.js' + export function ConcatArrayBuffers(Buffers: ArrayBuffer[]): ArrayBuffer { const TotalLength = Buffers.reduce((Sum, Block) => Sum + Block.byteLength, 0) - const Result = new Uint8Array(TotalLength) let Offset = 0 @@ -11,3 +14,207 @@ export function ConcatArrayBuffers(Buffers: ArrayBuffer[]): ArrayBuffer { return Result.buffer } + +export function BufferToArrayBuffer(Value: Uint8Array): ArrayBuffer { + const Result = new Uint8Array(Value.byteLength) + Result.set(new Uint8Array(Value.buffer, Value.byteOffset, Value.byteLength)) + return Result.buffer +} + +export async function ReadableToArrayBuffer(Stream: Readable): Promise { + const Chunks: ArrayBuffer[] = [] + + for await (const Chunk of Stream) { + Chunks.push(BufferToArrayBuffer(PayloadChunkToUint8Array(Chunk))) + } + + return ConcatArrayBuffers(Chunks) +} + +export function NormalizeHeaders(Headers?: Record): Record { + return Object.fromEntries( + Object.entries(Headers ?? {}).map(([Key, Value]) => [Key.toLowerCase(), Value]), + ) +} + +export function NormalizeIncomingHeaders(Headers: Record): Record { + return Object.fromEntries( + Object.entries(Headers) + .filter(([Key]) => Key.startsWith(':') === false) + .map(([Key, Value]) => { + if (Array.isArray(Value)) { + return [Key.toLowerCase(), Value.map(Item => Item?.toString())] + } + + if (Value === undefined || Value === null) { + return [Key.toLowerCase(), undefined] + } + + return [Key.toLowerCase(), Value.toString()] + }), + ) +} + +export function GetOriginKey(Url: URL): string { + const Port = Url.port || GetDefaultPort(Url.protocol) + return `${Url.protocol}//${Url.hostname}:${Port}` +} + +export function GetDefaultPort(Protocol: string): string { + switch (Protocol) { + case 'http:': + return '80' + case 'https:': + return '443' + default: + return '' + } +} + +export function SerializeTLSOptions(Options?: TLSOptions): string { + if (Options === undefined) { + return 'default' + } + + return JSON.stringify({ + MinTLSVersion: Options.MinTLSVersion, + MaxTLSVersion: Options.MaxTLSVersion, + Ciphers: Options.Ciphers ?? [], + KeyExchanges: Options.KeyExchanges ?? [], + RejectUnauthorized: Options.RejectUnauthorized, + }) +} + +export function GetHeaderValue(Headers: Record, Name: string): string | undefined { + const Value = Headers[Name.toLowerCase()] + + if (Array.isArray(Value)) { + return Value.join(', ') + } + + return Value +} + +export function ParseTokenList(Value?: string): string[] { + return (Value ?? '') + .split(',') + .map(Item => Item.trim().toLowerCase()) + .filter(Boolean) +} + +export function ParseCompressionAlgorithms(Value?: string): HTTPCompressionAlgorithm[] { + return ParseTokenList(Value).filter((Item): Item is HTTPCompressionAlgorithm => { + return Item === 'zstd' || Item === 'gzip' || Item === 'deflate' + }) +} + +export function IntersectCompressionAlgorithms(Primary: HTTPCompressionAlgorithm[], Secondary: HTTPCompressionAlgorithm[]): HTTPCompressionAlgorithm[] { + const Allowed = new Set(Secondary) + return Primary.filter((Item, Index) => Allowed.has(Item) && Primary.indexOf(Item) === Index) +} + +export function IsReadableStream(Value: unknown): Value is NodeJS.ReadableStream { + return typeof Value === 'object' && Value !== null && typeof (Value as NodeJS.ReadableStream).pipe === 'function' +} + +export function IsAbortSignal(Value: unknown): Value is AbortSignal { + return typeof Value === 'object' + && Value !== null + && typeof (Value as AbortSignal).aborted === 'boolean' + && typeof (Value as AbortSignal).addEventListener === 'function' + && typeof (Value as AbortSignal).removeEventListener === 'function' +} + +export function IsAsyncIterable(Value: unknown): Value is AsyncIterable { + return typeof Value === 'object' && Value !== null && Symbol.asyncIterator in Value +} + +export function IsStreamingPayload(Value: unknown): Value is NodeJS.ReadableStream | AsyncIterable { + return IsReadableStream(Value) || IsAsyncIterable(Value) +} + +export function PayloadChunkToUint8Array(Value: unknown): Uint8Array { + if (typeof Value === 'string') { + return Buffer.from(Value) + } + + if (Value instanceof ArrayBuffer) { + return new Uint8Array(Value) + } + + if (Value instanceof Uint8Array) { + return Value + } + + if (ArrayBuffer.isView(Value)) { + return new Uint8Array(Value.buffer, Value.byteOffset, Value.byteLength) + } + + throw new TypeError('Unsupported payload chunk type') +} + +export function GetPayloadByteLength(Value: Exclude>): number { + return PayloadChunkToUint8Array(Value).byteLength +} + +export function ToReadableStream(Value: NodeJS.ReadableStream | AsyncIterable): Readable { + if (IsReadableStream(Value)) { + return Value as Readable + } + + return Readable.from(Value) +} + +export function ResolveContentEncoding(Value?: string): HTTPCompressionAlgorithm | 'identity' { + const Encodings = ParseTokenList(Value).filter(Item => Item !== 'identity') + + if (Encodings.length === 0) { + return 'identity' + } + + if (Encodings.length > 1) { + throw new Error('Multiple content-encoding values are not supported') + } + + const [Encoding] = Encodings + if (Encoding === 'zstd' || Encoding === 'gzip' || Encoding === 'deflate') { + return Encoding + } + + throw new Error(`Unsupported response content-encoding: ${Encoding}`) +} + +export function CreateDecodedBodyStream(Stream: Readable, EncodingHeader?: string): { + Stream: Readable, + ContentEncoding: HTTPCompressionAlgorithm | 'identity', + DecodedBody: boolean +} { + const ContentEncoding = ResolveContentEncoding(EncodingHeader) + + switch (ContentEncoding) { + case 'identity': + return { + Stream, + ContentEncoding, + DecodedBody: false, + } + case 'zstd': + return { + Stream: Stream.pipe(ZLib.createZstdDecompress()), + ContentEncoding, + DecodedBody: true, + } + case 'gzip': + return { + Stream: Stream.pipe(ZLib.createGunzip()), + ContentEncoding, + DecodedBody: true, + } + case 'deflate': + return { + Stream: Stream.pipe(ZLib.createInflate()), + ContentEncoding, + DecodedBody: true, + } + } +} diff --git a/tests/cancellation.test.ts b/tests/cancellation.test.ts new file mode 100644 index 0000000..e1f7723 --- /dev/null +++ b/tests/cancellation.test.ts @@ -0,0 +1,72 @@ +import test from 'ava' +import { ReadStreamAsString, StartTestServer } from './support/server.js' +import { CreateTestClient } from './support/client.js' + +test('SecureReq enforces per-request timeouts while waiting for response headers', async T => { + const TestServer = await StartTestServer() + const Client = CreateTestClient() + + T.teardown(async () => { + Client.Close() + await TestServer.Close() + }) + + const Error = await T.throwsAsync(async () => { + await Client.Request(new URL('/slow-headers', TestServer.BaseUrl), { + ExpectedAs: 'String', + TimeoutMs: 75, + }) + }) + + T.is(Error?.name, 'TimeoutError') + T.is(Error?.message, 'Request timed out after 75ms') +}) + +test('SecureReq keeps request timeouts active for streaming responses', async T => { + const TestServer = await StartTestServer() + const Client = CreateTestClient() + + T.teardown(async () => { + Client.Close() + await TestServer.Close() + }) + + const Response = await Client.Request(new URL('/slow-stream', TestServer.BaseUrl), { + ExpectedAs: 'Stream', + TimeoutMs: 150, + }) + + const Error = await T.throwsAsync(async () => { + await ReadStreamAsString(Response.Body) + }) + + T.is(Error?.name, 'TimeoutError') + T.is(Error?.message, 'Request timed out after 150ms') +}) + +test('SecureReq supports AbortSignal cancellation', async T => { + const TestServer = await StartTestServer() + const Client = CreateTestClient() + + T.teardown(async () => { + Client.Close() + await TestServer.Close() + }) + + const Controller = new AbortController() + const PendingRequest = Client.Request(new URL('/slow-headers', TestServer.BaseUrl), { + ExpectedAs: 'String', + Signal: Controller.signal, + }) + + setTimeout(() => { + Controller.abort() + }, 75) + + const Error = await T.throwsAsync(async () => { + await PendingRequest + }) + + T.is(Error?.name, 'AbortError') + T.is(Error?.message, 'Request was aborted') +}) diff --git a/tests/capabilities.test.ts b/tests/capabilities.test.ts new file mode 100644 index 0000000..5718d37 --- /dev/null +++ b/tests/capabilities.test.ts @@ -0,0 +1,46 @@ +import test from 'ava' +import { StartTestServer } from './support/server.js' +import { CreateTestClient } from './support/client.js' + +test('SecureReq keeps origin capabilities conservative until evidence is observed', async T => { + const TestServer = await StartTestServer() + const Client = CreateTestClient() + + T.teardown(async () => { + Client.Close() + await TestServer.Close() + }) + + const Response = await Client.Request(new URL('/plain', TestServer.BaseUrl), { + ExpectedAs: 'String', + }) + const Capabilities = Client.GetOriginCapabilities(new URL(TestServer.BaseUrl)) + + T.is(Response.Protocol, 'http/1.1') + T.is(Capabilities?.PreferredProtocol, 'http/1.1') + T.deepEqual(Capabilities?.SupportedCompressions, []) + T.false(Capabilities?.HTTP3Advertised ?? true) +}) + +test('SecureReq evicts least-recently-used origin capability entries', async T => { + const TestServers = await Promise.all([StartTestServer(), StartTestServer()]) + const [FirstServer, SecondServer] = TestServers + const Client = CreateTestClient({ + OriginCapabilityCacheLimit: 1, + }) + + T.teardown(async () => { + Client.Close() + await Promise.all(TestServers.map(async TestServer => { + await TestServer.Close() + })) + }) + + await Client.Request(new URL('/negotiate', FirstServer.BaseUrl), { ExpectedAs: 'JSON' }) + T.truthy(Client.GetOriginCapabilities(new URL(FirstServer.BaseUrl))) + + await Client.Request(new URL('/negotiate', SecondServer.BaseUrl), { ExpectedAs: 'JSON' }) + + T.is(Client.GetOriginCapabilities(new URL(FirstServer.BaseUrl)), undefined) + T.truthy(Client.GetOriginCapabilities(new URL(SecondServer.BaseUrl))) +}) diff --git a/tests/compression.test.ts b/tests/compression.test.ts new file mode 100644 index 0000000..9fe51f0 --- /dev/null +++ b/tests/compression.test.ts @@ -0,0 +1,43 @@ +import test from 'ava' +import { StartTestServer } from './support/server.js' +import { CreateTestClient } from './support/client.js' + +for (const Encoding of ['gzip', 'deflate', 'zstd'] as const) { + test(`SecureReq decodes ${Encoding} responses`, async T => { + const TestServer = await StartTestServer() + const Client = CreateTestClient() + + T.teardown(async () => { + Client.Close() + await TestServer.Close() + }) + + const Response = await Client.Request(new URL(`/encoded/${Encoding}`, TestServer.BaseUrl), { + ExpectedAs: 'String', + HttpHeaders: { + 'accept-encoding': Encoding, + }, + }) + + T.is(Response.Protocol, 'http/1.1') + T.is(Response.ContentEncoding, Encoding) + T.true(Response.DecodedBody) + T.is(Response.Body, `compressed:${Encoding}`) + }) +} + +test('SecureReq normalizes duplicate supported compressions from constructor options', async T => { + const TestServer = await StartTestServer() + const Client = CreateTestClient({ + SupportedCompressions: ['gzip', 'gzip', 'deflate'], + }) + + T.teardown(async () => { + Client.Close() + await TestServer.Close() + }) + + const Response = await Client.Request(new URL('/negotiate', TestServer.BaseUrl), { ExpectedAs: 'JSON' }) + + T.is(Response.Headers['x-observed-accept-encoding'], 'gzip, deflate') +}) diff --git a/tests/index.test.ts b/tests/index.test.ts deleted file mode 100644 index 2f7060a..0000000 --- a/tests/index.test.ts +++ /dev/null @@ -1,30 +0,0 @@ -import test from 'ava' -import { HTTPSRequest, HTTPS2Request } from '@/index.js' - -test('www.example.com HTML request', async T => { - let Url = new URL('https://postman-echo.com/get?foo1=bar1&foo2=bar2') - let HTTPSRes = await HTTPSRequest(Url, { ExpectedAs: 'String' }) - T.is(HTTPSRes.StatusCode, 200) - T.true(typeof HTTPSRes.Body === 'string') -}) - -test('JSON request without ExpectedAs', async T => { - let Url = new URL('https://postman-echo.com/get?foo1=bar1&foo2=bar2') - let HTTPSRes = await HTTPSRequest(Url) - T.is(HTTPSRes.StatusCode, 200) - T.true(typeof HTTPSRes.Body === 'object' && HTTPSRes.Body instanceof ArrayBuffer) -}) - -test('HTTP/2 www.example.com HTML request', async T => { - let Url = new URL('https://postman-echo.com/get?foo1=bar1&foo2=bar2') - let HTTPSRes = await HTTPS2Request(Url, { ExpectedAs: 'String' }) - T.is(HTTPSRes.StatusCode, 200) - T.true(typeof HTTPSRes.Body === 'string') -}) - -test('HTTP/2 JSON request without ExpectedAs', async T => { - let Url = new URL('https://postman-echo.com/get?foo1=bar1&foo2=bar2') - let HTTPSRes = await HTTPS2Request(Url) - T.is(HTTPSRes.StatusCode, 200) - T.true(typeof HTTPSRes.Body === 'object' && HTTPSRes.Body instanceof ArrayBuffer) -}) \ No newline at end of file diff --git a/tests/options-and-parsing.test.ts b/tests/options-and-parsing.test.ts new file mode 100644 index 0000000..fc97ac1 --- /dev/null +++ b/tests/options-and-parsing.test.ts @@ -0,0 +1,52 @@ +import test from 'ava' +import { SecureReq } from '@/index.js' +import { StartTestServer } from './support/server.js' +import { CreateTestClient } from './support/client.js' + +test('SecureReq auto-detects response parsing when ExpectedAs is omitted', async T => { + const TestServer = await StartTestServer() + const Client = CreateTestClient() + + T.teardown(async () => { + Client.Close() + await TestServer.Close() + }) + + const JSONResponse = await Client.Request(new URL('/auto.json', TestServer.BaseUrl)) + const TextResponse = await Client.Request(new URL('/auto.txt', TestServer.BaseUrl)) + const BufferResponse = await Client.Request(new URL('/plain', TestServer.BaseUrl)) + + T.deepEqual(JSONResponse.Body, { ok: true }) + T.is(TextResponse.Body, 'auto-text') + T.true(BufferResponse.Body instanceof ArrayBuffer) +}) + +test('SecureReq validates constructor options at initialization time', async T => { + const InvalidCompression = 'brotli' as unknown as 'gzip' + + T.throws(() => { + return new SecureReq({ + HTTP2SessionIdleTimeout: 0, + }) + }) + + T.throws(() => { + return new SecureReq({ + OriginCapabilityCacheLimit: 0, + }) + }) + + T.throws(() => { + return new SecureReq({ + SupportedCompressions: [InvalidCompression], + }) + }) + + T.throws(() => { + return new SecureReq({ + DefaultOptions: { + HttpMethod: 'TRACE' as unknown as 'GET', + }, + }) + }) +}) diff --git a/tests/protocol-negotiation.test.ts b/tests/protocol-negotiation.test.ts new file mode 100644 index 0000000..e58ef9e --- /dev/null +++ b/tests/protocol-negotiation.test.ts @@ -0,0 +1,202 @@ +import { Readable } from 'node:stream' +import test from 'ava' +import { ReadStreamAsString, StartHTTP1OnlyTestServer, StartTestServer } from './support/server.js' +import { CreateTestClient } from './support/client.js' + +test('SecureReq probes with http/1.1 then upgrades to http/2 with negotiated compression state', async T => { + const TestServer = await StartTestServer() + const Client = CreateTestClient() + + T.teardown(async () => { + Client.Close() + await TestServer.Close() + }) + + const First = await Client.Request(new URL('/negotiate', TestServer.BaseUrl), { ExpectedAs: 'JSON' }) + const Second = await Client.Request(new URL('/negotiate', TestServer.BaseUrl), { ExpectedAs: 'JSON' }) + const Capabilities = Client.GetOriginCapabilities(new URL(TestServer.BaseUrl)) + + T.is(First.Protocol, 'http/1.1') + T.is(First.ContentEncoding, 'gzip') + T.true(First.DecodedBody) + T.is((First.Body as { Protocol: string }).Protocol, 'http/1.1') + T.is(First.Headers['x-observed-accept-encoding'], 'zstd, gzip, deflate') + + T.is(Second.Protocol, 'http/2') + T.is((Second.Body as { Protocol: string }).Protocol, 'http/2') + T.is(Second.Headers['x-observed-accept-encoding'], 'gzip') + + T.truthy(Capabilities) + T.deepEqual(Capabilities?.SupportedCompressions, ['gzip']) + T.true(Capabilities?.HTTP3Advertised ?? false) + T.is(Capabilities?.PreferredProtocol, 'http/2') +}) + +test('SecureReq supports streaming upload and streaming download after HTTP/2 upgrade', async T => { + const TestServer = await StartTestServer() + const Client = CreateTestClient() + + T.teardown(async () => { + Client.Close() + await TestServer.Close() + }) + + await Client.Request(new URL('/negotiate', TestServer.BaseUrl), { ExpectedAs: 'JSON' }) + await Client.Request(new URL('/negotiate', TestServer.BaseUrl), { ExpectedAs: 'JSON' }) + + const Response = await Client.Request(new URL('/stream-upload', TestServer.BaseUrl), { + Payload: Readable.from(['alpha-', 'beta-', 'gamma']), + ExpectedAs: 'Stream', + HttpMethod: 'POST', + }) + + T.is(Response.Protocol, 'http/2') + T.true(Response.DecodedBody) + T.is(await ReadStreamAsString(Response.Body), 'echo:alpha-beta-gamma') +}) + +test('SecureReq supports explicit protocol preferences without legacy wrappers', async T => { + const TestServer = await StartTestServer() + const Client = CreateTestClient() + + T.teardown(async () => { + Client.Close() + await TestServer.Close() + }) + + const HTTP1Response = await Client.Request(new URL('/stream-upload', TestServer.BaseUrl), { + HttpMethod: 'POST', + Payload: Readable.from(['explicit-', 'http1']), + ExpectedAs: 'Stream', + PreferredProtocol: 'http/1.1', + }) + + const HTTP2Response = await Client.Request(new URL('/plain', TestServer.BaseUrl), { + ExpectedAs: 'String', + PreferredProtocol: 'http/2', + }) + + T.is(HTTP1Response.Protocol, 'http/1.1') + T.is(await ReadStreamAsString(HTTP1Response.Body), 'echo:explicit-http1') + + T.is(HTTP2Response.Protocol, 'http/2') + T.is(HTTP2Response.Body, 'plain:http/2') +}) + +test('SecureReq safely falls back to http/1.1 after automatic HTTP/2 negotiation failure for GET', async T => { + const TestServer = await StartHTTP1OnlyTestServer() + const Client = CreateTestClient() + + T.teardown(async () => { + Client.Close() + await TestServer.Close() + }) + + const First = await Client.Request(new URL('/plain', TestServer.BaseUrl), { + ExpectedAs: 'String', + }) + const Second = await Client.Request(new URL('/plain', TestServer.BaseUrl), { + ExpectedAs: 'String', + }) + + T.is(First.Protocol, 'http/1.1') + T.is(Second.Protocol, 'http/1.1') + T.is(TestServer.GetRequestCount('/plain'), 2) + T.is(TestServer.GetSecureConnectionCount(), 2) +}) + +test('SecureReq does not auto-retry non-idempotent requests while HTTP/2 support is still unknown', async T => { + const TestServer = await StartHTTP1OnlyTestServer() + const Client = CreateTestClient() + + T.teardown(async () => { + Client.Close() + await TestServer.Close() + }) + + await Client.Request(new URL('/plain', TestServer.BaseUrl), { + ExpectedAs: 'String', + }) + + const Response = await Client.Request(new URL('/stream-upload', TestServer.BaseUrl), { + HttpMethod: 'POST', + Payload: Readable.from(['no-', 'retry']), + ExpectedAs: 'Stream', + }) + + T.is(Response.Protocol, 'http/1.1') + T.is(await ReadStreamAsString(Response.Body), 'echo:no-retry') + T.is(TestServer.GetRequestCount('/stream-upload'), 1) +}) + +test('SecureReq does not auto-retry HTTP/2 JSON parse failures', async T => { + const TestServer = await StartTestServer() + const Client = CreateTestClient() + + T.teardown(async () => { + Client.Close() + await TestServer.Close() + }) + + await Client.Request(new URL('/negotiate', TestServer.BaseUrl), { ExpectedAs: 'JSON' }) + + const Error = await T.throwsAsync(async () => { + await Client.Request(new URL('/invalid-json', TestServer.BaseUrl), { + ExpectedAs: 'JSON', + }) + }) + + T.truthy(Error) + T.is(TestServer.GetRequestCount('/invalid-json'), 1) +}) + +test('SecureReq does not auto-retry HTTP/2 timeouts', async T => { + const TestServer = await StartTestServer() + const Client = CreateTestClient() + + T.teardown(async () => { + Client.Close() + await TestServer.Close() + }) + + await Client.Request(new URL('/negotiate', TestServer.BaseUrl), { ExpectedAs: 'JSON' }) + + const Error = await T.throwsAsync(async () => { + await Client.Request(new URL('/slow-headers', TestServer.BaseUrl), { + ExpectedAs: 'String', + TimeoutMs: 75, + }) + }) + + T.is(Error?.name, 'TimeoutError') + T.is(TestServer.GetRequestCount('/slow-headers'), 1) +}) + +test('SecureReq does not auto-retry HTTP/2 aborts', async T => { + const TestServer = await StartTestServer() + const Client = CreateTestClient() + + T.teardown(async () => { + Client.Close() + await TestServer.Close() + }) + + await Client.Request(new URL('/negotiate', TestServer.BaseUrl), { ExpectedAs: 'JSON' }) + + const Controller = new AbortController() + const PendingRequest = Client.Request(new URL('/slow-headers', TestServer.BaseUrl), { + ExpectedAs: 'String', + Signal: Controller.signal, + }) + + setTimeout(() => { + Controller.abort() + }, 75) + + const Error = await T.throwsAsync(async () => { + await PendingRequest + }) + + T.is(Error?.name, 'AbortError') + T.is(TestServer.GetRequestCount('/slow-headers'), 1) +}) diff --git a/tests/redirects.test.ts b/tests/redirects.test.ts new file mode 100644 index 0000000..aabe00c --- /dev/null +++ b/tests/redirects.test.ts @@ -0,0 +1,129 @@ +import { Readable } from 'node:stream' +import test from 'ava' +import { StartTestServer } from './support/server.js' +import { CreateTestClient } from './support/client.js' + +test('SecureReq returns redirect responses as-is by default', async T => { + const TestServer = await StartTestServer() + const Client = CreateTestClient() + + T.teardown(async () => { + Client.Close() + await TestServer.Close() + }) + + const Response = await Client.Request(new URL('/redirect/plain', TestServer.BaseUrl), { + ExpectedAs: 'String', + }) + + T.is(Response.StatusCode, 302) + T.is(Response.Headers.location, '/plain') + T.is(TestServer.GetRequestCount('/redirect/plain'), 1) + T.is(TestServer.GetRequestCount('/plain'), 0) +}) + +test('SecureReq follows redirects when FollowRedirects is enabled', async T => { + const TestServer = await StartTestServer() + const Client = CreateTestClient() + + T.teardown(async () => { + Client.Close() + await TestServer.Close() + }) + + const Response = await Client.Request(new URL('/redirect/plain', TestServer.BaseUrl), { + ExpectedAs: 'String', + FollowRedirects: true, + }) + + T.is(Response.StatusCode, 200) + T.true(Response.Body === 'plain:http/1.1' || Response.Body === 'plain:http/2') + T.is(TestServer.GetRequestCount('/redirect/plain'), 1) + T.is(TestServer.GetRequestCount('/plain'), 1) +}) + +test('SecureReq enforces MaxRedirects while following redirects', async T => { + const TestServer = await StartTestServer() + const Client = CreateTestClient() + + T.teardown(async () => { + Client.Close() + await TestServer.Close() + }) + + const Error = await T.throwsAsync(async () => { + await Client.Request(new URL('/redirect/chain/1', TestServer.BaseUrl), { + ExpectedAs: 'String', + FollowRedirects: true, + MaxRedirects: 1, + }) + }) + + T.is(Error?.message, 'Maximum redirect limit exceeded (1)') +}) + +test('SecureReq converts POST to GET for 302 redirects', async T => { + const TestServer = await StartTestServer() + const Client = CreateTestClient() + + T.teardown(async () => { + Client.Close() + await TestServer.Close() + }) + + const Response = await Client.Request(new URL('/redirect/post-302', TestServer.BaseUrl), { + ExpectedAs: 'JSON', + FollowRedirects: true, + HttpMethod: 'POST', + Payload: 'alpha-beta', + }) + + T.is((Response.Body as { Method: string }).Method, 'GET') + T.is((Response.Body as { Body: string }).Body, '') + T.true(['http/1.1', 'http/2'].includes((Response.Body as { Protocol: string }).Protocol)) +}) + +test('SecureReq preserves method and payload for 307 redirects with replayable payloads', async T => { + const TestServer = await StartTestServer() + const Client = CreateTestClient() + + T.teardown(async () => { + Client.Close() + await TestServer.Close() + }) + + const Response = await Client.Request(new URL('/redirect/post-307', TestServer.BaseUrl), { + ExpectedAs: 'JSON', + FollowRedirects: true, + HttpMethod: 'POST', + Payload: 'alpha-beta', + }) + + T.deepEqual(Response.Body, { + Method: 'POST', + Body: 'alpha-beta', + Protocol: 'http/1.1', + }) +}) + +test('SecureReq rejects redirect replay for streaming payloads', async T => { + const TestServer = await StartTestServer() + const Client = CreateTestClient() + + T.teardown(async () => { + Client.Close() + await TestServer.Close() + }) + + const Error = await T.throwsAsync(async () => { + await Client.Request(new URL('/redirect/post-307', TestServer.BaseUrl), { + ExpectedAs: 'JSON', + FollowRedirects: true, + HttpMethod: 'POST', + Payload: Readable.from(['alpha-', 'beta']), + }) + }) + + T.is(Error?.message, 'Cannot automatically follow redirects that require replaying a streaming payload') + T.is(TestServer.GetRequestCount('/inspect-request'), 0) +}) diff --git a/tests/support/client.ts b/tests/support/client.ts new file mode 100644 index 0000000..6a970a3 --- /dev/null +++ b/tests/support/client.ts @@ -0,0 +1,15 @@ +import { SecureReq } from '@/index.js' +import type { SecureReqOptions } from '@/index.js' + +export function CreateTestClient(Options: SecureReqOptions = {}): SecureReq { + return new SecureReq({ + ...Options, + DefaultOptions: { + ...Options.DefaultOptions, + TLS: { + RejectUnauthorized: false, + ...(Options.DefaultOptions?.TLS ?? {}), + }, + }, + }) +} diff --git a/tests/support/server.ts b/tests/support/server.ts new file mode 100644 index 0000000..7e9bc7e --- /dev/null +++ b/tests/support/server.ts @@ -0,0 +1,395 @@ +import * as HTTP from 'node:http' +import * as HTTP2 from 'node:http2' +import * as HTTPS from 'node:https' +import * as ZLib from 'node:zlib' +import { CreateTestTLSCertificate } from './tls.js' +import type { HTTPCompressionAlgorithm } from '@/index.js' + +type TestRequest = HTTP.IncomingMessage | HTTP2.Http2ServerRequest +type TestResponse = HTTP.ServerResponse | HTTP2.Http2ServerResponse +type TestNodeServer = HTTPS.Server | HTTP2.Http2SecureServer + +interface TestServerTLSOptions { + MinVersion?: 'TLSv1.2' | 'TLSv1.3', + MaxVersion?: 'TLSv1.2' | 'TLSv1.3', + Ciphers?: string, + CertificateAlgorithm?: 'ed25519' | 'prime256v1' +} + +interface CreateTestServerOptions { + AdvertiseHTTP3: boolean, + HTTP2Enabled: boolean, + TLS?: TestServerTLSOptions +} + +export interface TestServer { + BaseUrl: string, + GetRequestCount: (Path: string) => number, + GetSecureConnectionCount: () => number, + Close: () => Promise +} + +function SelectCompression(HeaderValue: string): HTTPCompressionAlgorithm | undefined { + const Normalized = HeaderValue.toLowerCase() + + if (Normalized.includes('gzip')) { + return 'gzip' + } + + if (Normalized.includes('deflate')) { + return 'deflate' + } + + if (Normalized.includes('zstd')) { + return 'zstd' + } + + return undefined +} + +function CompressBody(Body: string | Uint8Array, Encoding?: HTTPCompressionAlgorithm): Buffer { + const Input = Buffer.from(Body) + + switch (Encoding) { + case 'gzip': + return ZLib.gzipSync(Input) + case 'deflate': + return ZLib.deflateSync(Input) + case 'zstd': + return ZLib.zstdCompressSync(Input) + default: + return Input + } +} + +function IncrementRequestCount(RequestCounts: Map, Path: string): void { + RequestCounts.set(Path, (RequestCounts.get(Path) ?? 0) + 1) +} + +function GetAcceptEncoding(Request: TestRequest): string { + return Array.isArray(Request.headers['accept-encoding']) + ? Request.headers['accept-encoding'].join(', ') + : Request.headers['accept-encoding'] ?? '' +} + +function WriteResponse(Response: TestResponse, Chunk: Uint8Array): void { + ;(Response as HTTP.ServerResponse).write(Chunk) +} + +function EndResponse(Response: TestResponse, Chunk: Uint8Array): void { + ;(Response as HTTP.ServerResponse).end(Chunk) +} + +export async function ReadStreamAsString(Stream: AsyncIterable): Promise { + let Result = '' + + for await (const Chunk of Stream) { + Result += typeof Chunk === 'string' ? Chunk : Buffer.from(Chunk).toString('utf-8') + } + + return Result +} + +async function ReadRequestBody(Request: AsyncIterable): Promise { + return await ReadStreamAsString(Request) +} + +function CreateRequestHandler(RequestCounts: Map, AdvertiseHTTP3: boolean): (Request: TestRequest, Response: TestResponse) => void { + return (Request, Response) => { + void (async () => { + const RequestUrl = new URL(Request.url ?? '/', 'https://localhost') + const AcceptEncoding = GetAcceptEncoding(Request) + const Protocol = Request.httpVersion === '2.0' ? 'http/2' : 'http/1.1' + + IncrementRequestCount(RequestCounts, RequestUrl.pathname) + + switch (RequestUrl.pathname) { + case '/negotiate': { + const ChosenEncoding = SelectCompression(AcceptEncoding) + const Payload = CompressBody(JSON.stringify({ + Protocol, + AcceptEncoding, + }), ChosenEncoding) + + Response.statusCode = 200 + Response.setHeader('content-type', 'application/json') + Response.setHeader('x-observed-accept-encoding', AcceptEncoding) + if (AdvertiseHTTP3) { + Response.setHeader('alt-svc', 'h3=":443"; ma=60') + } + if (ChosenEncoding) { + Response.setHeader('content-encoding', ChosenEncoding) + } + Response.end(Payload) + break + } + + case '/invalid-json': { + Response.statusCode = 200 + Response.setHeader('content-type', 'application/json') + Response.end(Buffer.from('{invalid-json')) + break + } + + case '/auto.json': { + Response.statusCode = 200 + Response.setHeader('content-type', 'application/json') + Response.end(Buffer.from(JSON.stringify({ ok: true }))) + break + } + + case '/auto.txt': { + Response.statusCode = 200 + Response.setHeader('content-type', 'text/plain; charset=utf-8') + Response.end(Buffer.from('auto-text')) + break + } + + case '/redirect/plain': { + Response.statusCode = 302 + Response.setHeader('location', '/plain') + Response.end(Buffer.from('redirecting')) + break + } + + case '/redirect/chain/1': { + Response.statusCode = 302 + Response.setHeader('location', '/redirect/chain/2') + Response.end(Buffer.from('redirect-chain-1')) + break + } + + case '/redirect/chain/2': { + Response.statusCode = 302 + Response.setHeader('location', '/plain') + Response.end(Buffer.from('redirect-chain-2')) + break + } + + case '/redirect/post-302': { + Response.statusCode = 302 + Response.setHeader('location', '/inspect-request') + Response.end(Buffer.from('redirect-post-302')) + break + } + + case '/redirect/post-307': { + Response.statusCode = 307 + Response.setHeader('location', '/inspect-request') + Response.end(Buffer.from('redirect-post-307')) + break + } + + case '/inspect-request': { + const RequestBody = await ReadRequestBody(Request) + + Response.statusCode = 200 + Response.setHeader('content-type', 'application/json') + Response.end(Buffer.from(JSON.stringify({ + Method: Request.method ?? 'GET', + Body: RequestBody, + Protocol, + }))) + break + } + + case '/encoded/gzip': + case '/encoded/deflate': + case '/encoded/zstd': { + const Encoding = RequestUrl.pathname.split('/').at(-1) as HTTPCompressionAlgorithm + const Payload = CompressBody(`compressed:${Encoding}`, Encoding) + + Response.statusCode = 200 + Response.setHeader('content-type', 'text/plain; charset=utf-8') + Response.setHeader('content-encoding', Encoding) + Response.end(Payload) + break + } + + case '/stream-upload': { + const RequestBody = await ReadRequestBody(Request) + const ResponseBody = CompressBody(`echo:${RequestBody}`, 'gzip') + const Half = Math.ceil(ResponseBody.length / 2) + + Response.statusCode = 200 + Response.setHeader('content-type', 'text/plain; charset=utf-8') + Response.setHeader('content-encoding', 'gzip') + + WriteResponse(Response, ResponseBody.subarray(0, Half)) + setTimeout(() => { + EndResponse(Response, ResponseBody.subarray(Half)) + }, 10) + break + } + + case '/plain': { + Response.statusCode = 200 + Response.setHeader('content-type', 'text/plain; charset=utf-8') + Response.end(Buffer.from(`plain:${Protocol}`)) + break + } + + case '/slow-headers': { + setTimeout(() => { + if (Response.writableEnded) { + return + } + + Response.statusCode = 200 + Response.setHeader('content-type', 'text/plain; charset=utf-8') + Response.end(Buffer.from('slow-headers')) + }, 250) + break + } + + case '/slow-stream': { + Response.statusCode = 200 + Response.setHeader('content-type', 'text/plain; charset=utf-8') + ;(Response as TestResponse & { flushHeaders?: () => void }).flushHeaders?.() + WriteResponse(Response, Buffer.from('slow-')) + + setTimeout(() => { + if (Response.writableEnded) { + return + } + + EndResponse(Response, Buffer.from('stream')) + }, 400) + break + } + + default: { + Response.statusCode = 404 + Response.setHeader('content-type', 'text/plain; charset=utf-8') + Response.end(Buffer.from('not-found')) + } + } + })().catch(Cause => { + Response.statusCode = 500 + Response.setHeader('content-type', 'text/plain; charset=utf-8') + Response.end(Buffer.from(String(Cause))) + }) + } +} + +async function StartServer( + Server: TestNodeServer, + RequestCounts: Map, + TLSCleanup: () => Promise, +): Promise { + let IsClosed = false + let SecureConnectionCount = 0 + + Server.on('secureConnection', () => { + SecureConnectionCount += 1 + }) + + try { + await new Promise((Resolve, Reject) => { + const HandleError = (Error: Error) => { + Server.off('listening', HandleListening) + Reject(Error) + } + + const HandleListening = () => { + Server.off('error', HandleError) + Resolve() + } + + Server.once('error', HandleError) + Server.once('listening', HandleListening) + Server.listen(0, '127.0.0.1') + }) + } catch (Cause) { + await TLSCleanup() + throw Cause + } + + const Address = Server.address() + if (Address === null || typeof Address === 'string') { + await TLSCleanup() + throw new Error('Failed to resolve test server address') + } + + return { + BaseUrl: `https://localhost:${Address.port}`, + GetRequestCount: Path => RequestCounts.get(Path) ?? 0, + GetSecureConnectionCount: () => SecureConnectionCount, + Close: async () => { + if (IsClosed) { + return + } + + IsClosed = true + + await new Promise((Resolve, Reject) => { + Server.close(Error => { + if (Error) { + Reject(Error) + return + } + + Resolve() + }) + }) + + await TLSCleanup() + }, + } +} + +async function CreateTLSServer(Options: CreateTestServerOptions): Promise { + const TLSCertificate = await CreateTestTLSCertificate({ + Algorithm: Options.TLS?.CertificateAlgorithm, + }) + const RequestCounts = new Map() + const Handler = CreateRequestHandler(RequestCounts, Options.AdvertiseHTTP3) + + const ServerOptions = { + key: TLSCertificate.Key, + cert: TLSCertificate.Cert, + minVersion: Options.TLS?.MinVersion, + maxVersion: Options.TLS?.MaxVersion, + ciphers: Options.TLS?.Ciphers, + } + + const Server = Options.HTTP2Enabled + ? HTTP2.createSecureServer({ + allowHTTP1: true, + ...ServerOptions, + }) + : HTTPS.createServer({ + ...ServerOptions, + }) + + Server.on('request', Handler) + + return await StartServer(Server, RequestCounts, TLSCertificate.Cleanup) +} + +export async function StartTestServer(): Promise { + return await CreateTLSServer({ + AdvertiseHTTP3: true, + HTTP2Enabled: true, + }) +} + +export async function StartHTTP1OnlyTestServer(): Promise { + return await CreateTLSServer({ + AdvertiseHTTP3: false, + HTTP2Enabled: false, + }) +} + +export async function StartTLS12ECDSATestServer(): Promise { + return await CreateTLSServer({ + AdvertiseHTTP3: false, + HTTP2Enabled: false, + TLS: { + MinVersion: 'TLSv1.2', + MaxVersion: 'TLSv1.2', + Ciphers: 'ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305', + CertificateAlgorithm: 'prime256v1', + }, + }) +} diff --git a/tests/support/tls.ts b/tests/support/tls.ts new file mode 100644 index 0000000..2775fba --- /dev/null +++ b/tests/support/tls.ts @@ -0,0 +1,103 @@ +import { execFile } from 'node:child_process' +import * as Fs from 'node:fs/promises' +import * as OS from 'node:os' +import * as Path from 'node:path' +import * as Process from 'node:process' +import { promisify } from 'node:util' + +const ExecFileAsync = promisify(execFile) + +export interface TestTLSCertificate { + Cert: string, + Key: string, + Cleanup: () => Promise +} + +export interface TestTLSCertificateOptions { + Algorithm?: 'ed25519' | 'prime256v1' +} + +async function RunOpenSSL(Arguments: string[]): Promise { + const Command = Process.platform === 'win32' ? 'openssl.exe' : 'openssl' + + try { + await ExecFileAsync(Command, Arguments, { + env: Process.env, + }) + } catch (Cause) { + throw new Error('OpenSSL is required to generate a test TLS certificate', { cause: Cause }) + } +} + +export async function CreateTestTLSCertificate(Options: TestTLSCertificateOptions = {}): Promise { + const TemporaryDirectory = await Fs.mkdtemp(Path.join(OS.tmpdir(), 'securereq-test-tls-')) + const KeyPath = Path.join(TemporaryDirectory, 'key.pem') + const CertificatePath = Path.join(TemporaryDirectory, 'cert.pem') + let IsCleanedUp = false + const Algorithm = Options.Algorithm ?? 'ed25519' + + try { + await RunOpenSSL( + Algorithm === 'prime256v1' + ? [ + 'req', + '-x509', + '-newkey', + 'ec', + '-pkeyopt', + 'ec_paramgen_curve:prime256v1', + '-pkeyopt', + 'ec_param_enc:named_curve', + '-nodes', + '-keyout', + KeyPath, + '-out', + CertificatePath, + '-subj', + '/CN=localhost', + '-days', + '1', + '-addext', + 'subjectAltName=DNS:localhost,IP:127.0.0.1', + ] + : [ + 'req', + '-x509', + '-newkey', + 'ed25519', + '-nodes', + '-keyout', + KeyPath, + '-out', + CertificatePath, + '-subj', + '/CN=localhost', + '-days', + '1', + '-addext', + 'subjectAltName=DNS:localhost,IP:127.0.0.1', + ], + ) + + const [Key, Cert] = await Promise.all([ + Fs.readFile(KeyPath, 'utf8'), + Fs.readFile(CertificatePath, 'utf8'), + ]) + + return { + Key, + Cert, + Cleanup: async () => { + if (IsCleanedUp) { + return + } + + IsCleanedUp = true + await Fs.rm(TemporaryDirectory, { recursive: true, force: true }) + }, + } + } catch (Cause) { + await Fs.rm(TemporaryDirectory, { recursive: true, force: true }) + throw Cause + } +} diff --git a/tests/tls-options.test.ts b/tests/tls-options.test.ts new file mode 100644 index 0000000..42f24cb --- /dev/null +++ b/tests/tls-options.test.ts @@ -0,0 +1,53 @@ +import test from 'ava' +import { CreateTestClient } from './support/client.js' +import { StartTLS12ECDSATestServer } from './support/server.js' + +test('SecureReq explains TLSv1.2 curve mismatches when KeyExchanges is too restrictive', async T => { + const TestServer = await StartTLS12ECDSATestServer() + const Client = CreateTestClient() + + T.teardown(async () => { + Client.Close() + await TestServer.Close() + }) + + const Error = await T.throwsAsync(async () => { + await Client.Request(new URL('/plain', TestServer.BaseUrl), { + ExpectedAs: 'String', + TLS: { + RejectUnauthorized: false, + MinTLSVersion: 'TLSv1.2', + MaxTLSVersion: 'TLSv1.2', + Ciphers: ['ECDHE-ECDSA-AES256-GCM-SHA384', 'ECDHE-ECDSA-CHACHA20-POLY1305'], + KeyExchanges: ['X25519'], + }, + }) + }) + + T.true(Error?.message.includes('KeyExchanges (X25519)')) + T.true(Error?.message.includes('P-256')) +}) + +test('SecureReq can connect to TLSv1.2 ECDSA servers when KeyExchanges includes a compatible certificate curve', async T => { + const TestServer = await StartTLS12ECDSATestServer() + const Client = CreateTestClient() + + T.teardown(async () => { + Client.Close() + await TestServer.Close() + }) + + const Response = await Client.Request(new URL('/plain', TestServer.BaseUrl), { + ExpectedAs: 'String', + TLS: { + RejectUnauthorized: false, + MinTLSVersion: 'TLSv1.2', + MaxTLSVersion: 'TLSv1.2', + Ciphers: ['ECDHE-ECDSA-AES256-GCM-SHA384', 'ECDHE-ECDSA-CHACHA20-POLY1305'], + KeyExchanges: ['X25519', 'P-256'], + }, + }) + + T.is(Response.Body, 'plain:http/1.1') + T.is(Response.Protocol, 'http/1.1') +}) diff --git a/tests/tsconfig.json b/tests/tsconfig.json index 72b5af2..105efec 100644 --- a/tests/tsconfig.json +++ b/tests/tsconfig.json @@ -1,6 +1,6 @@ { "extends": "../tsconfig.json", "include": [ - "**/*.test.ts" + "**/*.ts" ] -} \ No newline at end of file +} diff --git a/tests/types/request-types.ts b/tests/types/request-types.ts new file mode 100644 index 0000000..b9d5955 --- /dev/null +++ b/tests/types/request-types.ts @@ -0,0 +1,29 @@ +import type { Readable } from 'node:stream' +import { SecureReq } from '@/index.js' + +type IsEqual = ( + (() => Value extends Left ? 1 : 2) extends + (() => Value extends Right ? 1 : 2) + ? true + : false +) + +type Assert = Condition + +const Client = new SecureReq() + +const AutoDetectedRequest = Client.Request(new URL('https://example.com/auto.json')) +const ExplicitStringRequest = Client.Request(new URL('https://example.com/value.txt'), { + ExpectedAs: 'String', +}) +const ExplicitStreamRequest = Client.Request(new URL('https://example.com/value.bin'), { + ExpectedAs: 'Stream', +}) + +type AutoDetectedBody = Awaited['Body'] +type ExplicitStringBody = Awaited['Body'] +type ExplicitStreamBody = Awaited['Body'] + +type _AutoDetectedShouldBeUnknown = Assert> +type _ExplicitStringShouldBeString = Assert> +type _ExplicitStreamShouldBeStream = Assert>