Repository navigation
Releases: brekkylab/backlot
Release list
v0.0.5
🙌 New contributors
- @shrikargs7-cloud made their first contribution in #394
- @wanjinhao1 made their first contribution in #395
- @emiribrahimm made their first contribution in #398
- @Roshan1299 made their first contribution in #400
🔧 Fixes
Each closes a measured divergence from the vendor's real API. Those that change what a source answers are marked in bold.
Amazon S3
- Every answer carries real's
x-amz-request-idandx-amz-id-2, and an error body repeats them. A method the router does not serve answers what real answers for that method and sub-resource selector, the 405 namingBUCKET,OBJECT,SERVICEor the selector's type, the CORS 400/403 for anOPTIONS, rather than one shared JSON 405. A write isNotImplemented(501). (#319) - A refusal names what real names it with, in real's order, and a key in a bucket that does not exist or that the caller cannot see is
NoSuchBucket. (#361) - Signature Version 2, SigV4 and SigV4a are verified, in the header and in the query, and each fault is refused with real's code, message and members. (#361)
- A credential scope has to name
us-east-1, the region Backlot presents; a client signing for another region gets real'sAuthorizationHeaderMalformed, where Backlot used to read the region back out of the scope and serve any. An unsigned request is the anonymous caller's and sees no bucket. (#361) - A bucket's and an object's sub-resources answer as real answers an unconfigured one,
?versionsis ListObjectVersions, ListBuckets pages and refuses its four parameters as real does, and GetObject readspartNumber,versionId=nullandx-amz-checksum-mode. (#361) - Four writes are checked before the 501, so a malformed PutObject, DeleteObjects,
POST ?restoreorPUT ?encryptionis refused for what real refuses it for, with all ten checksum algorithms computed. AHEADis framed the way real frames it. (#361)
- Gmail's message tree is real's: a message with an attachment is
multipart/mixedover amultipart/alternative, every part carriesheaders,format=rawis built from the same tree, andformat=metadataservesmimeTypeandheadersalone.attachments.getserves{size, data},labels.listserves fifteen labels in the measured order without counts, an empty list is{"resultSizeEstimate": 0},threads.gethas nosnippet, and a snippet escapes<and>. (#400) - Gmail's
threads.getrefuses a reply's message id with the 404 a well-formed unknown id gets, rather than serving a one-message thread. (#395) - A Gmail attachment read on a missing message or attachment id is real's 400
Invalid attachment token, not a 404. (#394) - Drive and Sheets parse every repeat of a typed query parameter and refuse all the failures in one 400 with a
detailsentry each, after the credential and before the lookup. Drive'spageSizeandpageTokenare checked as real checks them, a blankfieldsanswers{}, andfiles.exportrefuses a format the file's type does not export to and serves one it does under themimeTypeexactly as sent. (#338) - A repeated query parameter is read from the end real reads it from: the first for Drive's
fields,q,pageSize,pageToken,orderByand exportmimeTypeand for Sheets'fieldsandprettyPrint, the last for the Sheets typed parameters.prettyPrintcompacts a Sheets answer atfalseand0only. (#336) - An anonymous
POSTis 401UNAUTHENTICATEDon every family, where aGETon Drive or Sheets stays the 403 unregistered caller. (#316) commentsViewModeon the Docs, Slides and Sheets get routes is an acknowledged gap. (#359)
Notion
GET /v1/commentsrefuses ablock_idit cannot use the way real does: absent, empty or not a uuid is a 400validation_error, a uuid naming nothing visible a 404, a database's id a 403. A visible block's id answers the empty list, andbacklot mcp'slist_commentsnow requiresblock_id. (#398)- A refusal carries
request_idandx-notion-request-id, and the 401 says which credential failed: a header that is not exactlyBearer <token>is refused for its format, a token that does not resolve asAPI token is invalid.. (#328) - A URL Notion does not publish is real's 400, and an operation it publishes that Backlot does not serve answers the credential first. One trailing slash is dropped and a
HEADis theGETwithout its body. (#328)
Atlassian
- A
HEADis theGETwith the body left off, and anOPTIONSis answered per product and per caller: Jira's 200 with its measuredAllow, Confluence's 404, itssearchanswering byAccept. (#330) - A path neither product serves gets each product's own answer: Jira's RFC 7807
No endpoint …, Confluence's JAX-RS 404, and the 401 for an operation the vendor publishes that Backlot does not serve. Outside the API mounts the site answers as its web app does, and a trailing slash or a run of slashes is ignored as the gateway ignores it. (#330) - Every answer carries the headers real sends beside the body,
atl-request-id,atl-traceid,x-arequestid,x-confluence-request-time, the deprecation trio and Jira's burst quota among them. (#330) - Confluence's listings read
limitandstartwith each route's own bounds,child/page,child/commentandlabelpage rather than serving the collection, and every page answers real's_links, the CQL search'scursorincluded. (#320) - Jira's
search/jqlrefuses amaxResultsoutside 1–5000, a body field it cannot coerce and an unknown body field. A negativemaxResultsused to serve the whole visible collection. (#286) setContextDefaultValuesis an acknowledged gap, with the early-access 403 the site answers recorded beside it. (#368, #399)
GitHub
- A request is refused once the caller's rate-limit window is spent, real's 403 with
usedpinned atlimit, for a caller with no credential and for a token alike, where Backlot used to let every request through. A suite sending more than ten code searches a minute now meets that refusal;BACKLOT_GITHUB_ENFORCE_RATE_LIMITS=falseturns it off. (#317) - A trailing slash means what each route makes of it: a ref keeps it and is refused,
contentsredirects one slash at a time with real'sLocationencoding, andreadme/{dir}serves that directory's README. (#326) /rate_limitanswersresourcesbeforerate, the two search windows measure a minute, and a caller with no credential has nocode_searchwindow of its own. A credential that does not resolve gets its 401 with no rate-limit headers and no count. (#289)- A path no route matches carries no version echo, and the
x-ratelimit-*headers only for a caller that sent no credential. (#288) relates_to, external installation properties and step logs are acknowledged gaps, and eighteen operations GitHub no longer publishes leave the baseline. (#360)
Slack
- A file's
useris served as a Slack id, minted from the address the corpus writes, asreactions[].usersandedited.useralready are. The corpus schema now typesfiles[].useras an address, so a corpus carrying"user": "U01"on a file stops at import. (#291) - A deactivated member drops the ten fields real never carries for one,
real_name,is_adminandtzamong them, rather than serving defaults. (#293) has_2fais served to an admin caller on every active person and to a caller on their own member, and never on a bot. (#293, #344)
Linear
AgentSession.codingHarnessandQuery.dependencyPackageMetadataare acknowledged gaps. (#358)
Tooling, packaging and CI
xxhashandcryptographyare base dependencies, for S3's checksums and SigV4a. (#361)- CONTRIBUTING's pull request checklist asks for
ruff check . && ruff format --check .besidepytest. (#396) - The
linearexample tracks@linear/sdk96.0.0; ruff 0.16.9, fastmcp 4.0.9, google-auth 2.58.1 and the actions group. (#298, #299, #300, #355, #356, #357)
Full changelog: v0.0.4...v0.0.5
v0.0.4
🤖 The agent loop
We're applying an agent-based automation loop to Backlot's own maintenance: a Claude Code routine takes an issue labelled agent, measures the real vendor API, fixes the divergence and opens a pull request that two reviewer agents have passed, leaving the merge to a person. It is still at an early stage, and we're looking forward to stabilising it in the near future. How to drive it is in docs/loop.md.
⚠️ Breaking changes
- A corpus imported before this release has to be re-imported. Drive files gained a folded title and Slack a table for deactivated members; the server names both at startup. A record needs no change unless it is one below. (#201, #259)
- A Slack reaction or edit names its people by address.
reactions[].usersis a non-empty list of addresses with nocount, andeditedis{"user": <the author's address>, "ts": <a later second>}; the router mints the Slack ids. A corpus carrying"users": ["U01"]stops at import. (#164, #257) - Notion requires
Notion-Version. A missing header or an unpublished version is refused with real's message; the seven published versions pass.2025-09-03is no longer assumed for a caller that sent nothing. (#226)
🔧 Fixes
Each closes a measured divergence from the vendor's real API. Those that change what a source answers are marked in bold.
Amazon S3
- A bare bucket
GETis ListObjects andlist-type=2is ListObjectsV2, each reading its own parameters and refusing the other's. Backlot answered the V2 shape regardless. A delimited V1 walk now terminates. (#204) encoding-type=urlis applied, which boto3 sends on every listing; a repeated parameter reads its first value;max-keys=-1is real's 400 rather than a bare 500. (#204)- An undecodable
continuation-token, or an empty one, is refused instead of answered with page one. (#281) - A
HEADon a bucket sub-resource carriesAllow: GETwhere the same path serves aGET. (#241)
Notion
Notion-Versionselects the database query path:databases/{id}/querybefore2025-09-03,data_sources/{id}/queryandGET data_sources/{id}from it, eachinvalid_request_urloutside its range. Every route declares the header, sobacklot mcpsends it. (#226)- The AI-plugins and AI-skills directory is an acknowledged gap. (#252)
- Drive's
files.listparsesqas the reference's grammar —and,or,not, parentheses, every operator per field — and refuses a clause it cannot evaluate rather than dropping it and answering the whole listing. Six of the issue's seven queries answered every file before. (#201) - A Sheets values read accepts R1C1, which LlamaIndex's
GoogleSheetsReadersends;point_sheets_atredirects that reader, with an example. (#201) - Every Google error body is rendered the way real writes one: indented,
charset=UTF-8, real's 209 escaped characters.callback=on a GET answers the error as JSONP at 200;altis read case-insensitively. (#220) $.xgafvis honoured on every route:1adds the legacyerrors[]array where each family's own rule says so, and any other value is refused ahead of everything else. (#202)backlot mcp --source gdriveoffers Docs, Sheets and Slides beside Drive — thirteen tools where there were six. (#199)backlot diffcompares Google's batch endpoint against thebatchPathits documents declare. (#283)
Atlassian
- A query parameter is read the way each product's own binder reads it: an empty value is the default, whitespace is removed, a repeated integer takes its first value,
startAtis along, Confluence refuses a negative where Jira clamps.?limit=-1no longer reaches SQLite as "no limit". A conversion failure carries Jira's RFC 7807 body or Confluence's Spring pair. (#206) - Jira's
search/jqlreads the query string on GET and the body on POST, nothing else. A POST with no body, a wrong media type, an undecodablenextPageTokenor a missingjqlis refused the way real refuses it. (#213) - Confluence's space listing answers a page:
limitandstartare read and_linkscarriesnext/prevspelled as real spells them, cut from the caller's own reachable spaces. (#271) - The space permission roster is
?expand=permissions, one entry per grant, andGET space/{key}/permissionis real's 405. A wrong method carries each product's own body, Jira's withAllow. (#245) - A Jira JSON body is
application/json;charset=UTF-8; Confluence stays bare. (#284) - The Forge panel pin status and workflow copy are acknowledged gaps. (#228)
Slack
- A deactivated member is
deleted: true, absent fromconversations.members, andaccount_inactiveon their own token, while their messages stay. A roster entry statesdeactivated: true. (#259) - A reaction's
usersand an edit'suserare rendered from addresses, so no invented Slack id is left in the bundled corpora. (#164, #257)
GitHub
- A trailing slash is a 404, not a 307, real having no slash redirect;
/contents/stays the root listing's 200. The 404 wins over a bad bearer. (#250) - A bad bearer is answered ahead of an unsupported
X-GitHub-Api-Version; a caller with no credential still meets the version check first. (#243) - The actions-policies family is an acknowledged gap. (#253)
Linear
IntegrationServicecarriesdatadog;IssueLabel.groupTypeis served, which@linear/sdk95.1 selects; four other new fields are acknowledged gaps. (#227, #275)
Tooling, packaging and CI
scripts/gen_docs.pyrenders from the checkout it runs in, not whicheverbacklotis installed. (#282)google-auth>=2.55in theofficial-sdkextra; thelinearexample tracks@linear/sdk95.1. (#211, #209, #275)- ruff 0.16.7, mcp 2.2.0, the actions group. (#210, #211)
Full changelog: v0.0.3...v0.0.4
v0.0.3
⚠️ Breaking changes
- A corpus imported before this release has to be re-imported. A spreadsheet's cells now live in their own table, which a 0.0.2 data dir does not have; the server names it at startup instead of failing every Sheets read. A record itself needs no change. (#161)
- The
examplesextra is nowofficial-sdk.pip install backlot[examples]does not fail — pip and uv install the base package at exit 0 — it dies later, on the import the extra was carrying. (#186) - The
mirageextra ismirage-ai[fuse,s3], floored at>=0.0.6. 0.0.5 has nomirage.core.google.constantsfor the patchers to rebind, and mirage's S3 backend importsaioboto3, which only its owns3extra brings. (#183)
🔧 Fixes
Each of these closes a measured divergence from the vendor's real API. Several change what a source answers, so a client written against Backlot's old behaviour rather than the vendor's may need updating — those are marked.
Google Sheets
- A spreadsheet is a workbook, not one sheet called
Sheet1. A record states named sheets over a 2D grid whose cells keep the type the corpus gave them, soUNFORMATTED_VALUEanswers a number whereFORMATTED_VALUEanswers a string. A record that states onlycontentkeeps the older reading, one line per cell. (#161) - The read surface is complete — one entry per sheet with its own
sheetId,index,titleandgridProperties,rangesfiltering thesheetsarray, andspreadsheets.getByDataFilterandvalues:batchGetByDataFilter, which were absent. A key-by-key diff against real finds nothing missing and nothing invented. (#161) - A prose spreadsheet longer than 1000 lines reports
rowCountat its line count, and a range inside it answers rather than400ing. (#161) values.gettakes any sheet, matched case-insensitively; afieldsmask naming a field Backlot cannot emit is refused rather than answered empty. (#161)
GitHub
- Every response carries real's five
x-ratelimit-*headers, withGET /rate_limitbehind them — real's limits, counted per credential and per resource. Nothing is refused when a window runs out, so a test suite is never failed for its own volume. (#175) - The four listings order by
sortanddirection, and the two repository listings filter bytypeandvisibility, as the wire serves them rather than as GitHub's description states. (#175) - A
HEADis theGETwithout its body — same status, headers,content-lengthandLink. (#166) - An unsent
per_pageserves 30 and a sent one is capped at 100, real's two numbers. (#152) - Every search stops at the first 1000 results, and
/search/coderefuses an unparseablepageintext/plainas its own backend does. (#144) - A JSON body carries
charset=utf-8on every route but/search/code, whose backend sends the bare type. (#147) - The spec states real's "(max 100)" on
per_pageand declaresstateon the issue and pull listings. The AI Scan settings are acknowledged gaps. (#166, #185)
Atlassian
- A Jira comment read serves the page its envelope describes. All three parameters were read from nowhere, so every response was the whole collection labelled as page one.
maxResultscaps at 100,orderBytakescreatedin either direction, andtotalstays the whole collection's count. (#165) - A Confluence space is listed and readable exactly when the caller can read a page in it; one the caller reaches no page in answers Confluence's own
404, on the space and on its permission roster. (#140)
Amazon S3
ListMultipartUploadsanswers the empty page real serves, where it was a501. The four markers come back present and empty, and what was sent is echoed in real's fixed order. (#176)
Linear
orreads one branch's keys as alternatives onCommentFilterandIssueLabelFilter, and by precedence on the label collection, as Linear does. (#143)Team.initiativesEnabledandProject.resourceCountare served, so@linear/sdk93's own Team, Teams and Project documents validate. (#157)- The inbox-notification, SSH-address and view-preference roots, and a project update's
shortSummary, are acknowledged gaps. (#142, #182, #195)
backlot diff
- Docs, Sheets, Slides, Jira v2 and HubSpot's Associations v4 were compared against nothing — one source type can be served through several vendor APIs, and a comparison held one document. It now holds one per API: thirteen where there were eight, reaching 11 paths and 12 operations no document reached. The Jira comment paging above is what the first run turned up. (#162)
- Jira v2 is compared against Atlassian's own v2 document, which is what both SDKs call by default. (#162)
- Previously invisible gaps, acknowledged in one pass:
google_drive146 → 224,hubspot14 → 23,jira640 → 1243, with zeroextra_operation. Every served path must now be compared or carry the reason none covers it. (#162)
Packaging and CI
- ruff's selection includes isort, and every import block is sorted. (#158, #160)
- CI installs with
uv sync --all-extras --locked, which resolves theaioboto3pin pip abandons as resolution-too-deep. (#183) - Every install line in a tracked file is held to an extra
pyproject.tomldefines — 52 occurrences, previously unheld. (#186) - The repository scans itself with HOL's plugin-scanner, and the scheduled fidelity check runs again. (#149, #181)
- ruff 0.15.22 → 0.16.6, the actions group, and
@linear/sdkto 93.0.1. (#154, #155, #156)
Full changelog: v0.0.2...v0.0.3
v0.0.2
✨ New features
backlot mcp (#108, #109)
Every source as MCP tools, over stdio — the command an MCP client runs (claude mcp add backlot -- backlot mcp):
backlot mcp # every source, as the admin
backlot mcp --user ava.chen@acme.com # answer as one person, so the ACL applies
backlot mcp --source slack # one source, under its plain tool namesIt bridges the server at --url; without one, the server already listening on 127.0.0.1:8000 if a Backlot server answers there, and otherwise one it starts itself on a free port over the data dir's corpus, stopping it when the client disconnects. --user resolves that person's credential for every source at once — bearer, Atlassian's Basic spelling, S3's signing keys — so the per-document ACL decides each answer. --depth sets the GraphQL selection depth for Linear and Fireflies.
backlot diff (#106)
Compares what Backlot serves against the vendor's own contract — live introspection for the GraphQL sources, the published OpenAPI or Discovery document for the REST ones, a running server's own answers for S3:
backlot diff --source githubBoth directions, and arguments as well as fields. Divergences already accepted live in backlot/fidelity/baseline/<source>.json and ship with the package, so a run reports only what is new. It exits 1 when the contracts disagree, 2 when the vendor's contract could not be read at all, and 3 when a declared credential is missing. This release's Linear, S3 and GitHub fixes are what it found on its first runs.
Agent skill and plugins (#94)
skills/backlot/SKILL.md gives an agent the loop from a request to a served corpus: check the install, pick a corpus, write records against the source's schema, --dry-run, load, serve, wire a client. The repository is its own plugin marketplace, one manifest per harness over a single copy of the skill:
claude plugin marketplace add brekkylab/backlot && claude plugin install backlot@brekkylab
codex plugin marketplace add brekkylab/backlot && codex plugin add backlot@brekkylab⚠️ Breaking changes
BACKLOT_EXPOSE_TOKENSis removed./_meta/usersand/_meta/credentialsalways answer — the setting gated the only credential inputbacklot mcp --userhas. Drop it from your.env.- The
mcpextra is on generation 2 —mcp>=2,fastmcp>=4,httpx2. The floors have to move together, so amcp<2pin alongside Backlot no longer resolves. (#98)
🔧 Fixes
Each of these closes a measured divergence from the vendor's real API. Several change what a source answers, so a client written against Backlot's old behaviour rather than the vendor's may need updating — those are marked.
Linear
- The served schema declares Linear's own types on 47 fields — date comparators, ten enums,
ExternalEntityInfo.id. Regenerate any client built from it. (#111) - An unset
priority, anupdatedAtwith no recorded edit, and null rows underneq/ninnow filter and sort on the value that was served. (#111) labels: {some: …}/{every: …}answer a label-less issue the way Linear does. (#116)null: trueon a relation filter reads nothing else in the object, andIntegrationServicecarriesorigin. (#129)orinside a relation filter follows Linear's three rules rather than being the union of its branches. (#133)
GitHub
- Every listing pages. Ten routes ignored
page/per_pageentirely, so a walk that incrementspageuntil the answer is empty never terminated. A page url now echoes only the parameters the caller actually sent. (#121, #131) - Every error answers GitHub's
{"message", "documentation_url", "status"}envelope, and the401says which credential failed. (#104) - A branch object carries all six members real answers,
protectionand_linksincluded. (#122) /branches,/branches/{branch}and/tagsare served, ACL-scoped like every other repo route. Asubtype: "repo"record states its owndefault_branch,branchesandtags; without one the listing holds the refs the repo's pulls advertise, and a name it omits is not a ref. (#95, #110)- A
repo:qualifier that names no searchable repository is refused instead of being dropped and answered with the whole corpus. (#131) stargazers/historyis recorded in the fidelity baseline as an acknowledged gap. (#132)
Atlassian
- Basic authenticates the pair. Knowing an address was enough to read as that person; an empty, wrong or another account's token is now
401. (#114) - A credential Backlot cannot resolve is no longer
401— Jira serves the request as anonymous, Confluence refuses with403, and an unresolvable bearer is a third403shape. None of the three real answers was a401. (#134)
Amazon S3
- The sub-resources Backlot does not implement are refused — 26 at a bucket's path and 8 at an object's,
501 NotImplementedinstead of falling through to the listing or the object's bytes. The listing,?list-type=2,?locationandGetObjectare untouched. (#118) - The canonical query for a signature is the wire query string, so a key containing
?verifies. (#115) bucket_getdeclares the query parameters it already honoured, so a truncated bucket can be paged through a tool.
Slack
auth.testanswersuser_idwith the caller's own derived id; the admin token keepsUSERVICE0. (#99)- The bundled corpus states a channel name real Slack could hold, and the schema holds
channelto Slack's charset. (#97)
Packaging and CI
- The awslabs aws-api MCP server the S3 test drives is pinned to the
mcpit imports. (#98) - An optional test gate must be reachable from a documented extra, so a gate no extra carries can no longer skip silently everywhere. (#102)
docs/configuration.mdand.env.exampleare held toSettings' field list; the fidelity baselines ship as package data.
Full changelog: v0.0.1...v0.0.2
v0.0.1
Backlot serves enterprise SaaS APIs over a corpus you supply, so a connector built on the vendors' own SDKs runs end to end with no accounts, no OAuth apps and no network.
Install
pip install backlot
backlot import --bundled # the corpus bundled in the package
backlot serve # http://127.0.0.1:8000Features
Sources — Slack, Gmail, Google Drive, GitHub, Jira, Confluence, Notion, Amazon S3, HubSpot, Linear, Fireflies. The real response shapes, status codes, pagination and auth scheme of each, including S3 SigV4 signing and GraphQL for Linear and Fireflies.
Per-document ACLs — every record states its own readers, and a token sees only theirs, so "does this leak?" is a test rather than an audit. An admin token bypasses filtering.
Corpora — backlot import --bundled serves the bundled example corpus inside the wheel with nothing to download; backlot import mycorpus.jsonl loads your own as BYO JSONL; backlot import --type enterpriserag-bench downloads and imports EnterpriseRAG-Bench.
Deterministic — every served id is derived from the corpus rather than random, so two runs answer identically.
Four ways to run it — backlot serve, backlot.serve() as a context manager on a free port, docker run, docker compose up. backlot status reports what a data dir holds, and backlot export dumps the imported corpus out as JSONL.
MCP — each source's OpenAPI and each GraphQL source's introspection are served as MCP tools, so an agent can drive the mock with no bridge written for it.
Examples — one runnable script per service for official vendor SDKs, MCP servers with agents, LlamaIndex readers, and Mirage's virtual filesystem. backlot.integrations rebinds the clients that hardcode a vendor host and offer no base-URL option.
Known limitations
Read surfaces only — write, update and delete are not served yet.