fix(pipeline): hash vector IDs to stay under Vectorize 64 byte cap - #84
Merged
liplus-lin-lay merged 1 commit intoApr 19, 2026
Conversation
Vectorize enforces a 64-byte limit on vector IDs. The previous scheme embedded repo + path/tag/sha as plain text and overflowed for long paths and for commit-diff IDs (e.g. docs/b.-... 74 bytes, diff IDs routinely >100 bytes), causing every affected embed to fail with VECTOR_UPSERT_ERROR code 40008. すべての vector surface (issue / pr / release / doc / diff) の ID を SHA-256 digest を base64url で 43 char に収めたハッシュ方式に統一し、 type prefix 1 文字 + ":" を付けて常に 46 byte に揃える。同じ入力から 同じ ID が出るので polling の change-detection には影響しない。呼出元 は async 化に追従して await を追加。mcp.ts は既に metadata ベースで URL を組んでいるため変更なし。既存 index の vector は古い ID のまま 残り、新 push / poll で新 ID が upsert される。完全 wipe が必要な場合 は従来どおり /admin/reset-hashes 経由で再 embed する。 Closes #83 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ✅ Deployment successful! View logs |
github-rag-mcp | 8fdb906 | Apr 19 2026, 09:06 AM |
liplus-lin-lay
deleted the
83-bugpipeline-vectorize-vector-id-exceeds-64-byte-limit-for-long-paths-and-commit-diffs
branch
April 19, 2026 09:07
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #83
Vectorize の Vector ID 64 byte 上限を超えて
VECTOR_UPSERT_ERROR (code=40008)で embed が失敗する問題を修正する。doc 長パス (74 byte) と、PR #81 で導入された diff ID scheme (100 byte 以上、原理的に動いていなかった) の両方が対象。変更方針 (issue #83 Option A)
{prefix}:{base64url(sha256(parts.join("\0")))}形式に統一iissue/pr,ddoc,rrelease,cdiff) の 1 文字 +:区切り変更ファイル
src/pipeline.ts—stableVectorId導入、4 surface の ID builder を async 化、processAndUpsertCommitDiffの chunk 内 ID 生成をPromise.all化src/webhook.ts—deleteByIds呼出の await 追従 (issue/release/doc 3 箇所)src/poller.ts— doc deletion の await 追従src/mcp.tsは search_issues の URL builder が既に metadata (doc_path/file_path/commit_sha/number) ベースで組まれており、ID からの reverse-parse は無かったので変更なし。既存 index 互換性
/admin/reset-hashes経由で再 embed影響範囲と互換性