Skip to content

Seeding large upstreams fails: single push 408s (chunk the seed push) #44

@cabljac

Description

@cabljac

Problem

venfork setup seeds a new private mirror with a single git push <default>:<default> of full upstream history. For large upstreams this fails:

error: RPC failed; HTTP 408 curl 22 The requested URL returned error: 408
send-pack: unexpected disconnect while reading sideband packet
fatal: the remote end hung up unexpectedly

Reproduced seeding genkit-ai/genkit (~203 MB pack) into a private mirror over gh-authenticated HTTPS (0.9.0, with the SSH-hang fix). HTTP 408 = GitHub server-side request-duration timeout on one giant monolithic push POST. Not a rate limit, not the auth hang — a distinct large-repo seeding limitation.

Why not just use GitHub import

The Source Imports REST API (PUT /repos/{owner}/{repo}/import) was deprecated and returns an error since 2024-04-12. Only the manual web Importer remains. So venfork cannot offload seeding to a server-side import API; it must transfer the pack itself.

Proposed fix

Chunk the seed push: walk git rev-list --reverse <default> and push in commit batches (force, --no-thin) so each POST stays under GitHub's HTTP timeout, then a final push of the real tip (+ tags). Add retry on transient 408. Consider http.postBuffer bump and http.version=HTTP/1.1. Keep full history (shared SHAs with upstream) — required for sync/stage/--pr to work.

Notes

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions