Skip to content

Releases: brekkylab/backlot

v0.0.5

Choose a tag to compare

@khj809 khj809 released this 03 Oct 12:13
b381268

🙌 New contributors

🔧 Fixes

Each closes a measured divergence from the vendor's real API. Those that change what a source answers are marked in bold.

Amazon S3

  • Every answer carries real's x-amz-request-id and x-amz-id-2, and an error body repeats them. A method the router does not serve answers what real answers for that method and sub-resource selector, the 405 naming BUCKET, OBJECT, SERVICE or the selector's type, the CORS 400/403 for an OPTIONS, rather than one shared JSON 405. A write is NotImplemented (501). (#319)
  • A refusal names what real names it with, in real's order, and a key in a bucket that does not exist or that the caller cannot see is NoSuchBucket. (#361)
  • Signature Version 2, SigV4 and SigV4a are verified, in the header and in the query, and each fault is refused with real's code, message and members. (#361)
  • A credential scope has to name us-east-1, the region Backlot presents; a client signing for another region gets real's AuthorizationHeaderMalformed, where Backlot used to read the region back out of the scope and serve any. An unsigned request is the anonymous caller's and sees no bucket. (#361)
  • A bucket's and an object's sub-resources answer as real answers an unconfigured one, ?versions is ListObjectVersions, ListBuckets pages and refuses its four parameters as real does, and GetObject reads partNumber, versionId=null and x-amz-checksum-mode. (#361)
  • Four writes are checked before the 501, so a malformed PutObject, DeleteObjects, POST ?restore or PUT ?encryption is refused for what real refuses it for, with all ten checksum algorithms computed. A HEAD is framed the way real frames it. (#361)

Google

  • Gmail's message tree is real's: a message with an attachment is multipart/mixed over a multipart/alternative, every part carries headers, format=raw is built from the same tree, and format=metadata serves mimeType and headers alone. attachments.get serves {size, data}, labels.list serves fifteen labels in the measured order without counts, an empty list is {"resultSizeEstimate": 0}, threads.get has no snippet, and a snippet escapes < and >. (#400)
  • Gmail's threads.get refuses a reply's message id with the 404 a well-formed unknown id gets, rather than serving a one-message thread. (#395)
  • A Gmail attachment read on a missing message or attachment id is real's 400 Invalid attachment token, not a 404. (#394)
  • Drive and Sheets parse every repeat of a typed query parameter and refuse all the failures in one 400 with a details entry each, after the credential and before the lookup. Drive's pageSize and pageToken are checked as real checks them, a blank fields answers {}, and files.export refuses a format the file's type does not export to and serves one it does under the mimeType exactly as sent. (#338)
  • A repeated query parameter is read from the end real reads it from: the first for Drive's fields, q, pageSize, pageToken, orderBy and export mimeType and for Sheets' fields and prettyPrint, the last for the Sheets typed parameters. prettyPrint compacts a Sheets answer at false and 0 only. (#336)
  • An anonymous POST is 401 UNAUTHENTICATED on every family, where a GET on Drive or Sheets stays the 403 unregistered caller. (#316)
  • commentsViewMode on the Docs, Slides and Sheets get routes is an acknowledged gap. (#359)

Notion

  • GET /v1/comments refuses a block_id it cannot use the way real does: absent, empty or not a uuid is a 400 validation_error, a uuid naming nothing visible a 404, a database's id a 403. A visible block's id answers the empty list, and backlot mcp's list_comments now requires block_id. (#398)
  • A refusal carries request_id and x-notion-request-id, and the 401 says which credential failed: a header that is not exactly Bearer <token> is refused for its format, a token that does not resolve as API token is invalid.. (#328)
  • A URL Notion does not publish is real's 400, and an operation it publishes that Backlot does not serve answers the credential first. One trailing slash is dropped and a HEAD is the GET without its body. (#328)

Atlassian

  • A HEAD is the GET with the body left off, and an OPTIONS is answered per product and per caller: Jira's 200 with its measured Allow, Confluence's 404, its search answering by Accept. (#330)
  • A path neither product serves gets each product's own answer: Jira's RFC 7807 No endpoint …, Confluence's JAX-RS 404, and the 401 for an operation the vendor publishes that Backlot does not serve. Outside the API mounts the site answers as its web app does, and a trailing slash or a run of slashes is ignored as the gateway ignores it. (#330)
  • Every answer carries the headers real sends beside the body, atl-request-id, atl-traceid, x-arequestid, x-confluence-request-time, the deprecation trio and Jira's burst quota among them. (#330)
  • Confluence's listings read limit and start with each route's own bounds, child/page, child/comment and label page rather than serving the collection, and every page answers real's _links, the CQL search's cursor included. (#320)
  • Jira's search/jql refuses a maxResults outside 1–5000, a body field it cannot coerce and an unknown body field. A negative maxResults used to serve the whole visible collection. (#286)
  • setContextDefaultValues is an acknowledged gap, with the early-access 403 the site answers recorded beside it. (#368, #399)

GitHub

  • A request is refused once the caller's rate-limit window is spent, real's 403 with used pinned at limit, for a caller with no credential and for a token alike, where Backlot used to let every request through. A suite sending more than ten code searches a minute now meets that refusal; BACKLOT_GITHUB_ENFORCE_RATE_LIMITS=false turns it off. (#317)
  • A trailing slash means what each route makes of it: a ref keeps it and is refused, contents redirects one slash at a time with real's Location encoding, and readme/{dir} serves that directory's README. (#326)
  • /rate_limit answers resources before rate, the two search windows measure a minute, and a caller with no credential has no code_search window of its own. A credential that does not resolve gets its 401 with no rate-limit headers and no count. (#289)
  • A path no route matches carries no version echo, and the x-ratelimit-* headers only for a caller that sent no credential. (#288)
  • relates_to, external installation properties and step logs are acknowledged gaps, and eighteen operations GitHub no longer publishes leave the baseline. (#360)

Slack

  • A file's user is served as a Slack id, minted from the address the corpus writes, as reactions[].users and edited.user already are. The corpus schema now types files[].user as an address, so a corpus carrying "user": "U01" on a file stops at import. (#291)
  • A deactivated member drops the ten fields real never carries for one, real_name, is_admin and tz among them, rather than serving defaults. (#293)
  • has_2fa is served to an admin caller on every active person and to a caller on their own member, and never on a bot. (#293, #344)

Linear

  • AgentSession.codingHarness and Query.dependencyPackageMetadata are acknowledged gaps. (#358)

Tooling, packaging and CI

  • xxhash and cryptography are base dependencies, for S3's checksums and SigV4a. (#361)
  • CONTRIBUTING's pull request checklist asks for ruff check . && ruff format --check . beside pytest. (#396)
  • The linear example tracks @linear/sdk 96.0.0; ruff 0.16.9, fastmcp 4.0.9, google-auth 2.58.1 and the actions group. (#298, #299, #300, #355, #356, #357)

Full changelog: v0.0.4...v0.0.5

v0.0.4

Choose a tag to compare

@khj809 khj809 released this 18 Sep 08:47
f1e5055

🤖 The agent loop

We're applying an agent-based automation loop to Backlot's own maintenance: a Claude Code routine takes an issue labelled agent, measures the real vendor API, fixes the divergence and opens a pull request that two reviewer agents have passed, leaving the merge to a person. It is still at an early stage, and we're looking forward to stabilising it in the near future. How to drive it is in docs/loop.md.

⚠️ Breaking changes

  • A corpus imported before this release has to be re-imported. Drive files gained a folded title and Slack a table for deactivated members; the server names both at startup. A record needs no change unless it is one below. (#201, #259)
  • A Slack reaction or edit names its people by address. reactions[].users is a non-empty list of addresses with no count, and edited is {"user": <the author's address>, "ts": <a later second>}; the router mints the Slack ids. A corpus carrying "users": ["U01"] stops at import. (#164, #257)
  • Notion requires Notion-Version. A missing header or an unpublished version is refused with real's message; the seven published versions pass. 2025-09-03 is no longer assumed for a caller that sent nothing. (#226)

🔧 Fixes

Each closes a measured divergence from the vendor's real API. Those that change what a source answers are marked in bold.

Amazon S3

  • A bare bucket GET is ListObjects and list-type=2 is ListObjectsV2, each reading its own parameters and refusing the other's. Backlot answered the V2 shape regardless. A delimited V1 walk now terminates. (#204)
  • encoding-type=url is applied, which boto3 sends on every listing; a repeated parameter reads its first value; max-keys=-1 is real's 400 rather than a bare 500. (#204)
  • An undecodable continuation-token, or an empty one, is refused instead of answered with page one. (#281)
  • A HEAD on a bucket sub-resource carries Allow: GET where the same path serves a GET. (#241)

Notion

  • Notion-Version selects the database query path: databases/{id}/query before 2025-09-03, data_sources/{id}/query and GET data_sources/{id} from it, each invalid_request_url outside its range. Every route declares the header, so backlot mcp sends it. (#226)
  • The AI-plugins and AI-skills directory is an acknowledged gap. (#252)

Google

  • Drive's files.list parses q as the reference's grammar — and, or, not, parentheses, every operator per field — and refuses a clause it cannot evaluate rather than dropping it and answering the whole listing. Six of the issue's seven queries answered every file before. (#201)
  • A Sheets values read accepts R1C1, which LlamaIndex's GoogleSheetsReader sends; point_sheets_at redirects that reader, with an example. (#201)
  • Every Google error body is rendered the way real writes one: indented, charset=UTF-8, real's 209 escaped characters. callback= on a GET answers the error as JSONP at 200; alt is read case-insensitively. (#220)
  • $.xgafv is honoured on every route: 1 adds the legacy errors[] array where each family's own rule says so, and any other value is refused ahead of everything else. (#202)
  • backlot mcp --source gdrive offers Docs, Sheets and Slides beside Drive — thirteen tools where there were six. (#199)
  • backlot diff compares Google's batch endpoint against the batchPath its documents declare. (#283)

Atlassian

  • A query parameter is read the way each product's own binder reads it: an empty value is the default, whitespace is removed, a repeated integer takes its first value, startAt is a long, Confluence refuses a negative where Jira clamps. ?limit=-1 no longer reaches SQLite as "no limit". A conversion failure carries Jira's RFC 7807 body or Confluence's Spring pair. (#206)
  • Jira's search/jql reads the query string on GET and the body on POST, nothing else. A POST with no body, a wrong media type, an undecodable nextPageToken or a missing jql is refused the way real refuses it. (#213)
  • Confluence's space listing answers a page: limit and start are read and _links carries next/prev spelled as real spells them, cut from the caller's own reachable spaces. (#271)
  • The space permission roster is ?expand=permissions, one entry per grant, and GET space/{key}/permission is real's 405. A wrong method carries each product's own body, Jira's with Allow. (#245)
  • A Jira JSON body is application/json;charset=UTF-8; Confluence stays bare. (#284)
  • The Forge panel pin status and workflow copy are acknowledged gaps. (#228)

Slack

  • A deactivated member is deleted: true, absent from conversations.members, and account_inactive on their own token, while their messages stay. A roster entry states deactivated: true. (#259)
  • A reaction's users and an edit's user are rendered from addresses, so no invented Slack id is left in the bundled corpora. (#164, #257)

GitHub

  • A trailing slash is a 404, not a 307, real having no slash redirect; /contents/ stays the root listing's 200. The 404 wins over a bad bearer. (#250)
  • A bad bearer is answered ahead of an unsupported X-GitHub-Api-Version; a caller with no credential still meets the version check first. (#243)
  • The actions-policies family is an acknowledged gap. (#253)

Linear

  • IntegrationService carries datadog; IssueLabel.groupType is served, which @linear/sdk 95.1 selects; four other new fields are acknowledged gaps. (#227, #275)

Tooling, packaging and CI

  • scripts/gen_docs.py renders from the checkout it runs in, not whichever backlot is installed. (#282)
  • google-auth>=2.55 in the official-sdk extra; the linear example tracks @linear/sdk 95.1. (#211, #209, #275)
  • ruff 0.16.7, mcp 2.2.0, the actions group. (#210, #211)

Full changelog: v0.0.3...v0.0.4

v0.0.3

Choose a tag to compare

@khj809 khj809 released this 11 Sep 07:50
bef7320

⚠️ Breaking changes

  • A corpus imported before this release has to be re-imported. A spreadsheet's cells now live in their own table, which a 0.0.2 data dir does not have; the server names it at startup instead of failing every Sheets read. A record itself needs no change. (#161)
  • The examples extra is now official-sdk. pip install backlot[examples] does not fail — pip and uv install the base package at exit 0 — it dies later, on the import the extra was carrying. (#186)
  • The mirage extra is mirage-ai[fuse,s3], floored at >=0.0.6. 0.0.5 has no mirage.core.google.constants for the patchers to rebind, and mirage's S3 backend imports aioboto3, which only its own s3 extra brings. (#183)

🔧 Fixes

Each of these closes a measured divergence from the vendor's real API. Several change what a source answers, so a client written against Backlot's old behaviour rather than the vendor's may need updating — those are marked.

Google Sheets

  • A spreadsheet is a workbook, not one sheet called Sheet1. A record states named sheets over a 2D grid whose cells keep the type the corpus gave them, so UNFORMATTED_VALUE answers a number where FORMATTED_VALUE answers a string. A record that states only content keeps the older reading, one line per cell. (#161)
  • The read surface is complete — one entry per sheet with its own sheetId, index, title and gridProperties, ranges filtering the sheets array, and spreadsheets.getByDataFilter and values:batchGetByDataFilter, which were absent. A key-by-key diff against real finds nothing missing and nothing invented. (#161)
  • A prose spreadsheet longer than 1000 lines reports rowCount at its line count, and a range inside it answers rather than 400ing. (#161)
  • values.get takes any sheet, matched case-insensitively; a fields mask naming a field Backlot cannot emit is refused rather than answered empty. (#161)

GitHub

  • Every response carries real's five x-ratelimit-* headers, with GET /rate_limit behind them — real's limits, counted per credential and per resource. Nothing is refused when a window runs out, so a test suite is never failed for its own volume. (#175)
  • The four listings order by sort and direction, and the two repository listings filter by type and visibility, as the wire serves them rather than as GitHub's description states. (#175)
  • A HEAD is the GET without its body — same status, headers, content-length and Link. (#166)
  • An unsent per_page serves 30 and a sent one is capped at 100, real's two numbers. (#152)
  • Every search stops at the first 1000 results, and /search/code refuses an unparseable page in text/plain as its own backend does. (#144)
  • A JSON body carries charset=utf-8 on every route but /search/code, whose backend sends the bare type. (#147)
  • The spec states real's "(max 100)" on per_page and declares state on the issue and pull listings. The AI Scan settings are acknowledged gaps. (#166, #185)

Atlassian

  • A Jira comment read serves the page its envelope describes. All three parameters were read from nowhere, so every response was the whole collection labelled as page one. maxResults caps at 100, orderBy takes created in either direction, and total stays the whole collection's count. (#165)
  • A Confluence space is listed and readable exactly when the caller can read a page in it; one the caller reaches no page in answers Confluence's own 404, on the space and on its permission roster. (#140)

Amazon S3

  • ListMultipartUploads answers the empty page real serves, where it was a 501. The four markers come back present and empty, and what was sent is echoed in real's fixed order. (#176)

Linear

  • or reads one branch's keys as alternatives on CommentFilter and IssueLabelFilter, and by precedence on the label collection, as Linear does. (#143)
  • Team.initiativesEnabled and Project.resourceCount are served, so @linear/sdk 93's own Team, Teams and Project documents validate. (#157)
  • The inbox-notification, SSH-address and view-preference roots, and a project update's shortSummary, are acknowledged gaps. (#142, #182, #195)

backlot diff

  • Docs, Sheets, Slides, Jira v2 and HubSpot's Associations v4 were compared against nothing — one source type can be served through several vendor APIs, and a comparison held one document. It now holds one per API: thirteen where there were eight, reaching 11 paths and 12 operations no document reached. The Jira comment paging above is what the first run turned up. (#162)
  • Jira v2 is compared against Atlassian's own v2 document, which is what both SDKs call by default. (#162)
  • Previously invisible gaps, acknowledged in one pass: google_drive 146 → 224, hubspot 14 → 23, jira 640 → 1243, with zero extra_operation. Every served path must now be compared or carry the reason none covers it. (#162)

Packaging and CI

  • ruff's selection includes isort, and every import block is sorted. (#158, #160)
  • CI installs with uv sync --all-extras --locked, which resolves the aioboto3 pin pip abandons as resolution-too-deep. (#183)
  • Every install line in a tracked file is held to an extra pyproject.toml defines — 52 occurrences, previously unheld. (#186)
  • The repository scans itself with HOL's plugin-scanner, and the scheduled fidelity check runs again. (#149, #181)
  • ruff 0.15.22 → 0.16.6, the actions group, and @linear/sdk to 93.0.1. (#154, #155, #156)

Full changelog: v0.0.2...v0.0.3

v0.0.2

Choose a tag to compare

@khj809 khj809 released this 04 Sep 09:42
bab49cb

✨ New features

backlot mcp (#108, #109)

Every source as MCP tools, over stdio — the command an MCP client runs (claude mcp add backlot -- backlot mcp):

backlot mcp                                 # every source, as the admin
backlot mcp --user ava.chen@acme.com        # answer as one person, so the ACL applies
backlot mcp --source slack                  # one source, under its plain tool names

It bridges the server at --url; without one, the server already listening on 127.0.0.1:8000 if a Backlot server answers there, and otherwise one it starts itself on a free port over the data dir's corpus, stopping it when the client disconnects. --user resolves that person's credential for every source at once — bearer, Atlassian's Basic spelling, S3's signing keys — so the per-document ACL decides each answer. --depth sets the GraphQL selection depth for Linear and Fireflies.

backlot diff (#106)

Compares what Backlot serves against the vendor's own contract — live introspection for the GraphQL sources, the published OpenAPI or Discovery document for the REST ones, a running server's own answers for S3:

backlot diff --source github

Both directions, and arguments as well as fields. Divergences already accepted live in backlot/fidelity/baseline/<source>.json and ship with the package, so a run reports only what is new. It exits 1 when the contracts disagree, 2 when the vendor's contract could not be read at all, and 3 when a declared credential is missing. This release's Linear, S3 and GitHub fixes are what it found on its first runs.

Agent skill and plugins (#94)

skills/backlot/SKILL.md gives an agent the loop from a request to a served corpus: check the install, pick a corpus, write records against the source's schema, --dry-run, load, serve, wire a client. The repository is its own plugin marketplace, one manifest per harness over a single copy of the skill:

claude plugin marketplace add brekkylab/backlot && claude plugin install backlot@brekkylab
codex plugin marketplace add brekkylab/backlot && codex plugin add backlot@brekkylab

⚠️ Breaking changes

  • BACKLOT_EXPOSE_TOKENS is removed. /_meta/users and /_meta/credentials always answer — the setting gated the only credential input backlot mcp --user has. Drop it from your .env.
  • The mcp extra is on generation 2 — mcp>=2, fastmcp>=4, httpx2. The floors have to move together, so a mcp<2 pin alongside Backlot no longer resolves. (#98)

🔧 Fixes

Each of these closes a measured divergence from the vendor's real API. Several change what a source answers, so a client written against Backlot's old behaviour rather than the vendor's may need updating — those are marked.

Linear

  • The served schema declares Linear's own types on 47 fields — date comparators, ten enums, ExternalEntityInfo.id. Regenerate any client built from it. (#111)
  • An unset priority, an updatedAt with no recorded edit, and null rows under neq / nin now filter and sort on the value that was served. (#111)
  • labels: {some: …} / {every: …} answer a label-less issue the way Linear does. (#116)
  • null: true on a relation filter reads nothing else in the object, and IntegrationService carries origin. (#129)
  • or inside a relation filter follows Linear's three rules rather than being the union of its branches. (#133)

GitHub

  • Every listing pages. Ten routes ignored page/per_page entirely, so a walk that increments page until the answer is empty never terminated. A page url now echoes only the parameters the caller actually sent. (#121, #131)
  • Every error answers GitHub's {"message", "documentation_url", "status"} envelope, and the 401 says which credential failed. (#104)
  • A branch object carries all six members real answers, protection and _links included. (#122)
  • /branches, /branches/{branch} and /tags are served, ACL-scoped like every other repo route. A subtype: "repo" record states its own default_branch, branches and tags; without one the listing holds the refs the repo's pulls advertise, and a name it omits is not a ref. (#95, #110)
  • A repo: qualifier that names no searchable repository is refused instead of being dropped and answered with the whole corpus. (#131)
  • stargazers/history is recorded in the fidelity baseline as an acknowledged gap. (#132)

Atlassian

  • Basic authenticates the pair. Knowing an address was enough to read as that person; an empty, wrong or another account's token is now 401. (#114)
  • A credential Backlot cannot resolve is no longer 401 — Jira serves the request as anonymous, Confluence refuses with 403, and an unresolvable bearer is a third 403 shape. None of the three real answers was a 401. (#134)

Amazon S3

  • The sub-resources Backlot does not implement are refused — 26 at a bucket's path and 8 at an object's, 501 NotImplemented instead of falling through to the listing or the object's bytes. The listing, ?list-type=2, ?location and GetObject are untouched. (#118)
  • The canonical query for a signature is the wire query string, so a key containing ? verifies. (#115)
  • bucket_get declares the query parameters it already honoured, so a truncated bucket can be paged through a tool.

Slack

  • auth.test answers user_id with the caller's own derived id; the admin token keeps USERVICE0. (#99)
  • The bundled corpus states a channel name real Slack could hold, and the schema holds channel to Slack's charset. (#97)

Packaging and CI

  • The awslabs aws-api MCP server the S3 test drives is pinned to the mcp it imports. (#98)
  • An optional test gate must be reachable from a documented extra, so a gate no extra carries can no longer skip silently everywhere. (#102)
  • docs/configuration.md and .env.example are held to Settings' field list; the fidelity baselines ship as package data.

Full changelog: v0.0.1...v0.0.2

v0.0.1

Choose a tag to compare

@khj809 khj809 released this 28 Aug 07:45
6a2d16f

Backlot serves enterprise SaaS APIs over a corpus you supply, so a connector built on the vendors' own SDKs runs end to end with no accounts, no OAuth apps and no network.

Install

pip install backlot
backlot import --bundled    # the corpus bundled in the package
backlot serve               # http://127.0.0.1:8000

Features

Sources — Slack, Gmail, Google Drive, GitHub, Jira, Confluence, Notion, Amazon S3, HubSpot, Linear, Fireflies. The real response shapes, status codes, pagination and auth scheme of each, including S3 SigV4 signing and GraphQL for Linear and Fireflies.

Per-document ACLs — every record states its own readers, and a token sees only theirs, so "does this leak?" is a test rather than an audit. An admin token bypasses filtering.

Corpora — backlot import --bundled serves the bundled example corpus inside the wheel with nothing to download; backlot import mycorpus.jsonl loads your own as BYO JSONL; backlot import --type enterpriserag-bench downloads and imports EnterpriseRAG-Bench.

Deterministic — every served id is derived from the corpus rather than random, so two runs answer identically.

Four ways to run it — backlot serve, backlot.serve() as a context manager on a free port, docker run, docker compose up. backlot status reports what a data dir holds, and backlot export dumps the imported corpus out as JSONL.

MCP — each source's OpenAPI and each GraphQL source's introspection are served as MCP tools, so an agent can drive the mock with no bridge written for it.

Examples — one runnable script per service for official vendor SDKs, MCP servers with agents, LlamaIndex readers, and Mirage's virtual filesystem. backlot.integrations rebinds the clients that hardcode a vendor host and offer no base-URL option.

Known limitations

Read surfaces only — write, update and delete are not served yet.