docs(blog): update embedding-cap description and add full serialized example#63
Merged
Conversation
…example The "Building the embedding input" section described a flat ~6,000-char source cap. Update it to match the current behavior: a configurable EMBEDDING_MAX_CHARS budget applied to the whole string, with truncation logged. Also add a concrete example of the full text that becomes one dense vector, and fix the list_users CodeSymbol's symbol_type to the value the parser actually assigns to a FastAPI route (api_route). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Two improvements to
blog.mdin the "Building the embedding input" section:Updated the embedding-cap description. It described a flat ~6,000-char cap on the source body. That now reflects the current behavior (Dense embedding silently truncates large symbols at 6000 chars #61 / merged PR fix: make dense-embedding truncation observable and configurable #62): a configurable
EMBEDDING_MAX_CHARSbudget applied to the whole serialized string (preamble + signature counted first, source fills the rest), with truncation logged so the loss is visible.Added a concrete full-string example showing exactly what the
list_userssymbol serializes into — the single text that becomes one dense vector — plus a short note on the metadata/docstring block, the signature, and why the signature appears twice.Also fixed the earlier
list_usersCodeSymbolexample:symbol_typeisapi_route(what_classify_functionactually assigns to a FastAPI route), notfunction— keeping the two examples consistent.Why
The blog predated the embedding-truncation fix, and readers had no concrete picture of the final serialized embedding input. Doc-only changes.