Skip to content

feat(gog): add batch begin / batch end primitive with --batch flag on edit commands #698

@sebsnyk

Description

@sebsnyk

Motivation

The Google Docs documents.batchUpdate endpoint accepts up to 500 requests in a single call. Sheets spreadsheets.batchUpdate and Slides presentations.batchUpdate have the same shape with per-service request schemas.

gog already uses BatchUpdate internally on every edit command (docs_table_column_width.go, docs_cell_style.go, docs_insert_image.go, docs_smart_chips.go, docs_write_*.go) — each command builds []*docs.Request and submits as one Documents.BatchUpdate(...). But the unit of batching today is one CLI invocation, so consumer scripts that orchestrate dozens of edits per doc burn one API call per command.

Concrete profile of a typical markdown-import workflow that pushes one doc:

Edit kind Calls / push Request type
docs table-column-width (N cols x M tables) 12 - 25 UpdateTableColumnProperties
docs cell-style (background + bold + col-span headers) 8 - 14 UpdateTableCellStyle + UpdateTextStyle
docs insert-person (raw walk + delete + insert-person, reverse-index) 3 x N DeleteContentRange + InsertPerson
docs insert-image --at <placeholder> 1 - 6 InsertInlineImage / ReplaceImage
docs find-replace 0 - 4 ReplaceAllText
Total ~30 mixed
  • Today: ~30 sequential calls = ~30 write-quota items + ~15 s wall-clock.
  • With batch primitive: 1 batchUpdate = 1 quota item + ~1.5 s wall-clock. 30x quota reduction, 10x wall-clock reduction.

Direct evidence: the same consumer is hitting quotaExceeded (429) on back-to-back pushes when several docs are pushed inside a minute. Per-doc quota cost goes from 30 to 1.

Proposed CLI surface

gog batch begin --doc=<id> [--service=docs|sheets|slides] [--name=<label>]
gog batch list
gog batch show <batch-id> [-j|--verbose]
gog batch end <batch-id> [--dry-run] [--continue-on-error] [--auto-split]
gog batch abort <batch-id>
gog batch prune --older-than=72h

And a --batch=<batch-id> flag on every edit command (docs write, insert, delete, update, find-replace, format, cell-style, table-column-width, insert-image, insert-person, insert-page-break). When set, the command resolves its arguments and anchors as it does today, builds the []*docs.Request payload it would have submitted, and appends to the batch state file instead of calling the API.

begin returns the batch id (UUID v7, sortable). Plain-mode output is the bare id so BID=$(gog batch begin --doc=$DOC) works without jq. --service defaults to docs; required to be set upfront (state-file shape depends on it).

State file

Use the existing config.StateDir() helper — siblings the existing gmail-watch/ directory. Honours GOG_HOME and GOG_STATE_DIR overrides.

<StateDir>/batches/<batch-id>.json

Schema:

{
  "batch_id": "01HXYZ...",
  "name": "docs:1ISHDn3...",
  "service": "docs",
  "doc_id": "1ISHDn3...",
  "account": "user@example.com",
  "client": "default",
  "created_at": "2026-06-05T14:23:11Z",
  "updated_at": "2026-06-05T14:23:47Z",
  "requests": [
    {
      "appended_at": "...",
      "command": "docs table-column-width",
      "argv": ["--table-index=1", "--col=1", "--width=200"],
      "request": { /* docs.Request JSON */ }
    }
  ]
}

Concurrent appends from multiple shells use the gmail-watch pattern: flock(LOCK_EX) + read + append + write-temp + rename. Same code shape as internal/cmd/gmail_watch_state.go. No auto-expiry; batch list shows age, batch prune is explicit.

Atomicity

documents.batchUpdate is strictly atomic — if any request fails, none are applied. This is a behavioural change from today's per-call independence.

Recommendation: strict-atomic by default. It matches the API guarantee and the principle of least surprise (batch end either applies all or none).

Add an opt-in --continue-on-error flag on batch end: on a 400 from the batched submit, gog re-submits each accumulated request as its own batchUpdate and reports per-request status. Lossier semantics, but recovers the "today" recovery behaviour for callers who explicitly want it. Stderr warning is unambiguous: "atomic batch failed; falling back to per-request submission (request N of M)".

Anchor resolution (--at <text>) happens at append time against the live doc, not at submit time. Resolving at submit time would require gog to rebuild index-shift tracking inside the batch executor — significant scope escalation. Keep append-time resolution; document the moving-target caveat.

write --replace --markdown distinction

This command today does delete-all + insert-converted-body + apply-tables-styles as one big internal batchUpdate. In --batch mode it should append every constituent request to the batch, not submit. But because its first request is a DeleteContentRange over the full body, appending it to a non-empty batch makes prior appends moot. Recommendation: refuse with a clear error when appending write --replace --markdown to a non-empty batch; future --allow-replace-after if a legit case emerges.

Multi-doc + cross-service

A single batchUpdate is scoped to one doc and one service. CLI rejects mismatched appends with batch <id> targets <service>:<doc-id>; this command targets <other>. For the parallel multi-doc case the pattern is "open N batches and end them in parallel" — already possible with the proposed surface.

>500 request limit

The Docs API caps at 500 requests per call. Recommendation: refuse by default with a clear error; offer --auto-split for callers who explicitly want non-atomic chunked submission (chunk size 500, stderr warning on each chunk).

Migration path

Before (~30 sequential API calls):

gog docs table-column-width $DOC --table-index=1 --col=1 --width=200
gog docs table-column-width $DOC --table-index=1 --col=2 --width=200
gog docs cell-style $DOC --table-index=2 --row=2 --col=0 --col-span=6 \
  --background-color=#C8E6C9 --bold
# ... 27 more sequential calls ...

After (1 API call):

BID=$(gog batch begin --doc=$DOC --service=docs)
gog docs table-column-width $DOC --table-index=1 --col=1 --width=200 --batch=$BID
gog docs table-column-width $DOC --table-index=1 --col=2 --width=200 --batch=$BID
gog docs cell-style $DOC --table-index=2 --row=2 --col=0 --col-span=6 \
  --background-color=#C8E6C9 --bold --batch=$BID
# ... 27 more --batch=$BID appends ...
gog batch end $BID

Optional pre-submit review:

gog batch end $BID --dry-run   # inspect payload
gog batch end $BID             # submit

Tiny diff to consumer scripts: one begin, one end, one extra flag.

Relationship to other open issues

All of these add edit commands or extend existing ones; each should support --batch from day one:

Once --batch lands, the per-command increment for every future edit command is: build []*docs.Request -> if --batch set, append; else submit. Trivial.

Acceptance criteria

  1. gog batch begin --doc=<id> creates <StateDir>/batches/<id>.json and prints the batch id (plain) or {"batch_id": "..."} in -j.
  2. Any gog docs edit command with --batch=<id> appends its []*docs.Request payload to the state file under a file lock and does not call the API.
  3. gog batch end <id> submits accumulated requests as one documents.batchUpdate and deletes the state file on success.
  4. gog batch end <id> --dry-run prints the would-be payload, does not call the API, keeps the state file.
  5. gog batch end <id> --continue-on-error falls back to per-request submission on atomic-batch failure and reports per-request status on stderr.
  6. --batch=<id> rejects mismatched (service, doc_id) appends with a clear error before any state mutation.
  7. Batches > 500 requests refuse batch end unless --auto-split is passed; --auto-split chunks at 500 with stderr warning on each chunk.
  8. gog batch list / show / abort / prune behave as documented; batch show -j emits the exact wire payload.

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Normal priority bug or improvement with limited blast radius.clawsweeper:needs-maintainer-reviewClawSweeper marked this issue as needing maintainer review before automation.clawsweeper:needs-product-decisionClawSweeper marked this issue as needing a product or behavior decision.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.impact:otherThis issue has meaningful maintainer-visible impact outside the owned taxonomy.issue-rating: 🌊 off-meta tidepoolIssue quality rating does not apply to this item.

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions