Skip to content

[TEST] feat/pullfactory — PullFactory extension point + widened LLM payload overrides#47

Merged
Dead-Bytes merged 8 commits into
pre-releasefrom
feat/pullfactory
May 14, 2026
Merged

[TEST] feat/pullfactory — PullFactory extension point + widened LLM payload overrides#47
Dead-Bytes merged 8 commits into
pre-releasefrom
feat/pullfactory

Conversation

@Dead-Bytes
Copy link
Copy Markdown
Collaborator

What changed

  • New PullFactory extension point in @bytebell/ingest-github, mirroring the existing SourceFactory for index jobs. registerGithubWorkers({ sourceFactory, pullFactory }) now accepts both; OSS leaves both undefined and falls back to the default disk-backed clone + git diff path.
  • runPull (pipeline/pull.ts) refactored:
    • When a pullFactory is provided, it returns { source, diff, targetCommit, archiveSink? } and the worker skips the local ensureReposRoot / syncRepository / materialiseEndpoints / assertReachableFromBranch / computePullDiff / checkoutCommit / createDiskSourceReader chain entirely.
    • When omitted, behaviour is byte-for-byte the previous OSS flow.
    • analyseChangedFiles now takes a SourceReader (and optional ArchiveSink) instead of a raw repoDir, so the pull path no longer assumes a local checkout exists.
  • strategies/flat-folder/analyse-changed.ts updated to read through SourceReader instead of the filesystem directly.
  • Public exports widened on @bytebell/ingest-github's index.ts so downstream wrappers can compose their own runner/handlers:
    • createPipelineRunner + CreatePipelineRunnerDeps
    • createGithubIngestHandler, createLocalIngestHandler, IngestJobHandlerDeps
    • runPull, reposRoot
    • IngestRunnerDeps, IngestRunnerInput
    • PullFactory, PullFactoryInput, PullFactoryResult
    • DiffResult, RenamedFile
  • @bytebell/types PayloadLlmOverrides:
    • llmProvider widened from the closed union "openrouter" | "ollama" to string so downstream taxonomies ("anthropic", "gemini", "mistral", …) round-trip through the payload. OSS still only routes on openrouter/ollama; other values are ignored by the OSS client.
    • New optional llmKeyId — opaque to OSS, used by downstream resolvers as an audit pointer back to the source of truth.
  • GithubPullPayload gains an optional orgId (parallel to GithubIndexPayload.orgId). OSS leaves it unset and reads Config.OrgId ("local") as before; multi-tenant wrappers stamp it from the request.
  • context.md updated everywhere touched: ingest-github/, ingest-github/src/, pipeline/, strategies/flat-folder/, types/, plus @bytebell/types package + src.

Why

The per-call LLM credentials work landed the data path; this PR lands the matching control path for pulls. Downstream multi-tenant deployments need to:

  1. Inject their own source acquisition strategy for pulls (e.g. cached archive fetch via api-streaming instead of a local git clone per worker, or a pre-materialised mount).
  2. Carry richer provider/credential metadata through the job payload without forcing OSS to recognise every provider name.
  3. Bind a pull to an org without re-engineering the OSS single-tenant default.

Without PullFactory, every downstream had to fork runPull. Without llmKeyId / widened llmProvider / orgId, downstream payloads couldn't round-trip cleanly through the OSS type. OSS standalone behaviour is unchanged — all new fields are optional and the default branch in runPull is the old code path verbatim.

Builds on top of feat/per-call-llm-creds, hence the base.

@Dead-Bytes Dead-Bytes added enhancement New feature or request testing changes are being tested labels May 14, 2026
@Dead-Bytes Dead-Bytes changed the base branch from feat/per-call-llm-creds to pre-release May 14, 2026 13:20
@Dead-Bytes Dead-Bytes merged commit fc746c8 into pre-release May 14, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request testing changes are being tested

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant