Skip to content

feat(github): classify GitHub rate-limit, abuse-detection, and auth responses #221

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
John-David Dalton (jdalton) wants to merge 1 commit into
base: main
Choose a base branch
from
Open

feat(github): classify GitHub rate-limit, abuse-detection, and auth responses #221

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 15 additions & 14 deletions docs/api.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -465,20 +465,21 @@ Each entry links to the source module and shows the first sentence of its `@file

## github/

| Subpath | Description |
| ----------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [`@socketsecurity/lib-stable/github/commit`](../src/github/commit.ts) | Create a SIGNED commit on a branch via the GitHub git-objects API (blob -> tree -> commit -> ref PATCH). |
| [`@socketsecurity/lib-stable/github/constants`](../src/github/constants.ts) | GitHub API URL + cache-TTL constants. |
| [`@socketsecurity/lib-stable/github/errors`](../src/github/errors.ts) | Named errors thrown by `github/*` helpers. |
| [`@socketsecurity/lib-stable/github/ghsa`](../src/github/ghsa.ts) | GitHub Security Advisory (GHSA) lookups. |
| [`@socketsecurity/lib-stable/github/refs`](../src/github/refs.ts) | Resolve GitHub git refs (tag / branch / commit) to full commit SHAs. |
| [`@socketsecurity/lib-stable/github/refs-cache`](../src/github/refs-cache.ts) | TtlCache singleton for github/refs. |
| [`@socketsecurity/lib-stable/github/refs-graphql`](../src/github/refs-graphql.ts) | Resolve a GitHub git ref via GraphQL. |
| [`@socketsecurity/lib-stable/github/refs-rest`](../src/github/refs-rest.ts) | Resolve a GitHub git ref via REST tier-cascade. |
| [`@socketsecurity/lib-stable/github/request`](../src/github/request.ts) | Authenticated GitHub REST fetch. |
| [`@socketsecurity/lib-stable/github/token`](../src/github/token.ts) | GitHub token resolution. |
| [`@socketsecurity/lib-stable/github/types`](../src/github/types.ts) | Public type surface for `github/*` modules — pure interfaces. |
| [`@socketsecurity/lib-stable/github/workflow-runs`](../src/github/workflow-runs.ts) | Look up a GitHub Actions workflow run for a commit and classify its verdict — the "is CI green on this SHA?" primitive behind gated releases (bump/tag/publish only after the pre-release commit's CI passes). |
| Subpath | Description |
| ------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [`@socketsecurity/lib-stable/github/commit`](../src/github/commit.ts) | Create a SIGNED commit on a branch via the GitHub git-objects API (blob -> tree -> commit -> ref PATCH). |
| [`@socketsecurity/lib-stable/github/constants`](../src/github/constants.ts) | GitHub API URL + cache-TTL constants. |
| [`@socketsecurity/lib-stable/github/error-classification`](../src/github/error-classification.ts) | Classify a GitHub API error response from its status, headers, and body. |
| [`@socketsecurity/lib-stable/github/errors`](../src/github/errors.ts) | Named errors thrown by `github/*` helpers. |
| [`@socketsecurity/lib-stable/github/ghsa`](../src/github/ghsa.ts) | GitHub Security Advisory (GHSA) lookups. |
| [`@socketsecurity/lib-stable/github/refs`](../src/github/refs.ts) | Resolve GitHub git refs (tag / branch / commit) to full commit SHAs. |
| [`@socketsecurity/lib-stable/github/refs-cache`](../src/github/refs-cache.ts) | TtlCache singleton for github/refs. |
| [`@socketsecurity/lib-stable/github/refs-graphql`](../src/github/refs-graphql.ts) | Resolve a GitHub git ref via GraphQL. |
| [`@socketsecurity/lib-stable/github/refs-rest`](../src/github/refs-rest.ts) | Resolve a GitHub git ref via REST tier-cascade. |
| [`@socketsecurity/lib-stable/github/request`](../src/github/request.ts) | Authenticated GitHub REST fetch. |
| [`@socketsecurity/lib-stable/github/token`](../src/github/token.ts) | GitHub token resolution. |
| [`@socketsecurity/lib-stable/github/types`](../src/github/types.ts) | Public type surface for `github/*` modules — pure interfaces. |
| [`@socketsecurity/lib-stable/github/workflow-runs`](../src/github/workflow-runs.ts) | Look up a GitHub Actions workflow run for a commit and classify its verdict — the "is CI green on this SHA?" primitive behind gated releases (bump/tag/publish only after the pre-release commit's CI passes). |

## globs/

Expand Down
3 changes: 2 additions & 1 deletion llms.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# @socketsecurity/lib

> Core utilities and infrastructure for Socket.dev security tools. 586 subpath exports, grouped by namespace.
> Core utilities and infrastructure for Socket.dev security tools. 587 subpath exports, grouped by namespace.

Import any namespace by its subpath, e.g. `import '@socketsecurity/lib/abort/signal'`. Each link below points at the TypeScript declarations shipped in the package, where the full signature for that subpath lives.

Expand Down Expand Up @@ -406,6 +406,7 @@ Import any namespace by its subpath, e.g. `import '@socketsecurity/lib/abort/sig

- [@socketsecurity/lib/github/commit](./dist/github/commit.d.ts): Create a SIGNED commit on a branch via the GitHub git-objects API (blob -> tree -> commit -> ref PATCH).
- [@socketsecurity/lib/github/constants](./dist/github/constants.d.ts): GitHub API URL + cache-TTL constants.
- [@socketsecurity/lib/github/error-classification](./dist/github/error-classification.d.ts): Classify a GitHub API error response from its status, headers, and body.
- [@socketsecurity/lib/github/errors](./dist/github/errors.d.ts): Named errors thrown by `github/*` helpers.
- [@socketsecurity/lib/github/ghsa](./dist/github/ghsa.d.ts): GitHub Security Advisory (GHSA) lookups.
- [@socketsecurity/lib/github/refs](./dist/github/refs.d.ts): Resolve GitHub git refs (tag / branch / commit) to full commit SHAs.
Expand Down
5 changes: 5 additions & 0 deletions package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1766,6 +1766,11 @@
"types": "./dist/github/constants.d.ts",
"default": "./dist/github/constants.js"
},
"./github/error-classification": {
"source": "./src/github/error-classification.ts",
"types": "./dist/github/error-classification.d.ts",
"default": "./dist/github/error-classification.js"
},
"./github/errors": {
"source": "./src/github/errors.ts",
"types": "./dist/github/errors.d.ts",
Expand Down
231 changes: 231 additions & 0 deletions src/github/error-classification.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,231 @@
/**
* @file Classify a GitHub API error response from its status, headers, and
* body. Pure data in, pure data out — no fetch, no logging, no result type —
* so any caller can run it against whatever HTTP client it already uses.
* GitHub reports three conditions in ways that are easy to misread as
* ordinary failures, and misreading one turns a throttled run into a silent
* success. A rate limit arrives as HTTP 429, or as HTTP 403 carrying
* `x-ratelimit-remaining: 0` — that second form has no distinguishing status
* code, so code that reads the body without checking the status sees "this
* repo has nothing to return". Abuse detection, GitHub's secondary rate
* limit, arrives as HTTP 403 with a body saying so, sharing its status with
* both the rate-limit form and a plain permission denial. An auth failure
* arrives as HTTP 401, meaning the token is invalid, expired, or missing a
* scope, so no amount of waiting helps.
* All three are BLOCKING: they turn on the credential and the clock, not on
* the resource being requested, so every later request in a loop over repos
* fails the same way. A caller iterating resources should stop on the first
* one instead of retrying it or moving to the next. A plain permission denial
* is deliberately NOT one of them — that one IS about the resource, so
* skipping it and continuing is right.
* Retry policy for what comes back lives in `releases/github-retry-config`;
* this module only decides WHAT a response is.
*/

import { parseRetryAfterHeader } from '../http-request/headers'

import { ArrayIsArray } from '../primordials/array'
import { DateNow } from '../primordials/date'
import { MathFloor, MathMax } from '../primordials/math'
import { NumberIsFinite, NumberParseInt } from '../primordials/number'
import { ObjectFreeze, ObjectKeys } from '../primordials/object'
import {
StringPrototypeIncludes,
StringPrototypeToLowerCase,
} from '../primordials/string'

/**
* Which blocking condition a GitHub error response represents.
*
* - `abuse-detection` — the secondary rate limit, tripped by bursty traffic.
* - `auth-failure` — the credential itself is rejected.
* - `rate-limit` — the primary hourly quota is spent.
*/
export type GitHubErrorKind = 'abuse-detection' | 'auth-failure' | 'rate-limit'

/**
* Every kind {@link classifyGitHubErrorResponse} can return, in sorted order.
*
* Exported so a caller can derive its own blocking-condition table from this
* list rather than hard-coding one. A caller that does so picks up a future
* kind for free instead of silently treating it as an ordinary error.
*/
export const GITHUB_BLOCKING_ERROR_KINDS: readonly GitHubErrorKind[] =
ObjectFreeze(['abuse-detection', 'auth-failure', 'rate-limit'])

/**
* What {@link classifyGitHubErrorResponse} decided about a response. Getting one
* back at all means the condition is blocking.
*/
export interface GitHubErrorClassification {
/**
* Which condition this is.
*/
kind: GitHubErrorKind
/**
* Whether waiting can clear the condition. `true` for the two rate limits,
* `false` for an auth failure, which the same token never recovers from.
*/
retryable: boolean
/**
* Seconds until the limit resets, read from `Retry-After` or
* `x-ratelimit-reset`. `undefined` when the response did not say, which is
* the common case for the primary hourly limit.
*/
waitSeconds: number | undefined
}

/**
* Response headers in either shape a caller is likely to hold: a Fetch
* `Headers` object, or the plain record that Node's HTTP layer produces.
*/
export type GitHubResponseHeaders =
| { get(name: string): string | null }
| Record<string, string | string[] | undefined>

/**
* Classify a GitHub API response as one of the blocking conditions.
*
* Order matters: abuse detection is checked before the primary rate limit
* because both arrive as HTTP 403 and the abuse form is the more specific of
* the two.
*
* @example
* ;```ts
* const bodyText = await response.text()
* const blocked = classifyGitHubErrorResponse({
* body: bodyText,
* headers: response.headers,
* status: response.status,
* })
* if (blocked) {
* // Stop the loop; every later request fails the same way.
* }
* ```
*
* @param response - The status, headers, and body text to classify.
*
* @returns The classification, or `undefined` when the response is not one of
* the blocking conditions. `undefined` covers healthy responses and ordinary
* errors alike, so the caller keeps its own handling of 404s, empty repos,
* and permission denials.
*/
export function classifyGitHubErrorResponse(response: {
body?: string | undefined
headers?: GitHubResponseHeaders | undefined
status: number
}): GitHubErrorClassification | undefined {
const { body, headers, status } = response
const lowerBody = body ? StringPrototypeToLowerCase(body) : ''

if (
status === 403 &&
(StringPrototypeIncludes(lowerBody, 'secondary rate limit') ||
StringPrototypeIncludes(lowerBody, 'abuse detection'))
) {
return {
kind: 'abuse-detection',
retryable: true,
waitSeconds: getGitHubRateLimitWaitSeconds(headers),
}
}

const remaining = getGitHubResponseHeader(headers, 'x-ratelimit-remaining')
if (
status === 429 ||
(status === 403 &&
(remaining === '0' || StringPrototypeIncludes(lowerBody, 'rate limit')))
) {
return {
kind: 'rate-limit',
retryable: true,
waitSeconds: getGitHubRateLimitWaitSeconds(headers),
}
}

if (status === 401) {
return {
kind: 'auth-failure',
retryable: false,
waitSeconds: undefined,
}
}

return undefined
}

/**
* Seconds to wait before a throttled GitHub request could succeed.
*
* Prefers `Retry-After`, which GitHub sends on secondary limits and which RFC
* 7231 allows to be either a delay in seconds or an absolute HTTP date. Falls
* back to `x-ratelimit-reset`, an absolute epoch-seconds timestamp, converted
* to a relative wait against the current clock and floored at zero so a reset
* already in the past reads as "no wait" instead of a negative number.
*
* @example
* ;```ts
* const seconds = getGitHubRateLimitWaitSeconds(response.headers)
* if (seconds !== undefined && seconds <= 30) {
* // Short enough to wait out.
* }
* ```
*
* @param headers - The response headers.
*
* @returns Whole seconds to wait, or `undefined` when neither header is usable.
*/
export function getGitHubRateLimitWaitSeconds(
headers: GitHubResponseHeaders | undefined,
): number | undefined {
const retryAfterMs = parseRetryAfterHeader(
getGitHubResponseHeader(headers, 'retry-after'),
)
if (retryAfterMs !== undefined) {
return MathFloor(retryAfterMs / 1000)
}
const reset = getGitHubResponseHeader(headers, 'x-ratelimit-reset')
if (reset) {
const resetEpochSeconds = NumberParseInt(reset, 10)
if (NumberIsFinite(resetEpochSeconds)) {
return MathMax(0, resetEpochSeconds - MathFloor(DateNow() / 1000))
}
}
return undefined
}

/**
* Read one header out of either header shape.
*
* A Fetch `Headers` already matches names case-insensitively. A plain record
* does not, so the record branch compares lowercased keys rather than trusting
* the caller to have normalized them. An array-valued header yields its first
* entry, matching how Node exposes repeated headers.
*
* @param headers - The response headers.
* @param name - The header name, lowercase.
*
* @returns The header value, or `undefined` when absent.
*/
export function getGitHubResponseHeader(
headers: GitHubResponseHeaders | undefined,
name: string,
): string | undefined {
if (!headers) {
return undefined
}
const getter = (headers as { get?: unknown | undefined }).get
if (typeof getter === 'function') {
const value = (headers as { get(name: string): string | null }).get(name)
return value === null ? undefined : value
}
const record = headers as Record<string, string | string[] | undefined>
for (const key of ObjectKeys(record)) {
if (StringPrototypeToLowerCase(key) !== name) {
continue
}
const value = record[key]
return ArrayIsArray(value) ? value[0] : value
}
return undefined
}
Loading
Loading