feat: Add support for topics to Search Contexts#1028
Merged
brendan-kellam merged 4 commits intosourcebot-dev:mainfrom Mar 23, 2026
Merged
feat: Add support for topics to Search Contexts#1028brendan-kellam merged 4 commits intosourcebot-dev:mainfrom
topics to Search Contexts#1028brendan-kellam merged 4 commits intosourcebot-dev:mainfrom
Conversation
This commit updates Sourcebot to include support for using `topics` as part of the Search Context definition. As part of this: * Updated `repoMetadataSchema` to store `topics` for `github` and `gitlab` host types * Populate the topic list when compiling GitHub and Gitlab repos * Updated schemas to support `includeTopics/excludeTopics` * Expanded test coverage * Updated Docs
Contributor
WalkthroughAdds topic-based filtering to Search Contexts via new Changes
Sequence Diagram(s)mermaid Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Suggested reviewers
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 4
🧹 Nitpick comments (3)
packages/backend/src/ee/syncSearchContexts.test.ts (1)
93-95: Consider extracting a smallgetConnectedIdshelper to reduce repetition.The
upsertCall.create.repos.connect.map(...)pattern is repeated many times, which adds noise and makes future assertion changes tedious.🧹 Optional cleanup
+const getConnectedIds = (db: PrismaClient): number[] => { + const upsertCall = vi.mocked(db.searchContext.upsert).mock.calls[0][0]; + return upsertCall.create.repos.connect.map((r: { id: number }) => r.id); +}; ... -const upsertCall = vi.mocked(db.searchContext.upsert).mock.calls[0][0]; -const connectedIds = upsertCall.create.repos.connect.map((r: { id: number }) => r.id); +const connectedIds = getConnectedIds(db);🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/backend/src/ee/syncSearchContexts.test.ts` around lines 93 - 95, Extract a small helper function (e.g., getConnectedIds) in the test to read connected repo ids from the mocked upsert call instead of repeating upsertCall.create.repos.connect.map(...); implement getConnectedIds(upsertCallOrCreate) to accept either the full upsertCall (from vi.mocked(db.searchContext.upsert).mock.calls[0][0]) or the create object and return the mapped id array, then replace occurrences where you currently do upsertCall.create.repos.connect.map((r: { id: number }) => r.id) with calls to getConnectedIds(upsertCall) to reduce repetition and simplify assertions in tests referencing db.searchContext.upsert and upsertCall.packages/backend/src/ee/syncSearchContexts.ts (2)
69-71: Normalize repo IDs once before upsert.Dedup currently happens only in the includeTopics path. Consolidating dedup right before upsert also covers connection-based duplicates and lets Line 173 use a
Setinstead of repeated array rebuilds.⚙️ Suggested refactor
+// Canonicalize once before read/write operations +const uniqueReposById = new Map<number, { id: number; name: string; metadata: unknown }>(); +for (const repo of newReposInContext) { + uniqueReposById.set(repo.id, repo); +} +newReposInContext = [...uniqueReposById.values()]; +const newRepoIds = new Set(newReposInContext.map(repo => repo.id)); ... await db.searchContext.upsert({ ... update: { repos: { connect: newReposInContext.map(repo => ({ id: repo.id })), disconnect: currentReposInContext - .filter(repo => !newReposInContext.map(r => r.id).includes(repo.id)) + .filter(repo => !newRepoIds.has(repo.id)) .map(repo => ({ id: repo.id })), },Also applies to: 87-93, 169-173
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/backend/src/ee/syncSearchContexts.ts` around lines 69 - 71, The loop that builds newReposInContext by concatenating connection.repos currently defers deduplication until includeTopics path; instead, normalize and deduplicate repo IDs once immediately before the upsert to cover duplicates from connections and topics: collect all repo IDs into newReposInContext (from connection.repos.map(repo => repo.repo) and any includeTopics additions), then replace the repeated array concatenations with a single Set-based dedupe (e.g., const uniqueRepos = Array.from(new Set(newReposInContext))) and use uniqueRepos for the upsert call and anywhere Line 173 reconstructs arrays so you can remove repeated rebuilds and switch to Set lookups.
74-86: Extract topic parsing/matching into a shared helper to avoid drift.The same parse + topic extraction + lowercase matching logic appears twice (include and exclude paths). A helper would keep behavior consistent and easier to evolve.
♻️ Suggested refactor
+const getNormalizedRepoTopics = (metadata: unknown): string[] | null => { + const parsed = repoMetadataSchema.safeParse(metadata); + if (!parsed.success) { + return null; + } + return [ + ...(parsed.data.codeHostMetadata?.gitlab?.topics ?? []), + ...(parsed.data.codeHostMetadata?.github?.topics ?? []), + ].map(topic => topic.toLowerCase()); +}; ... if (newContextConfig.includeTopics) { const topicPatterns = newContextConfig.includeTopics.map(t => t.toLowerCase()); const matching = allRepos.filter(repo => { - const parsed = repoMetadataSchema.safeParse(repo.metadata); - if (!parsed.success) { + const repoTopics = getNormalizedRepoTopics(repo.metadata); + if (!repoTopics) { return false; } - const repoTopics = [ - ...(parsed.data.codeHostMetadata?.gitlab?.topics ?? []), - ...(parsed.data.codeHostMetadata?.github?.topics ?? []), - ]; - return repoTopics.some(t => micromatch.isMatch(t.toLowerCase(), topicPatterns)); + return repoTopics.some(t => micromatch.isMatch(t, topicPatterns)); }); ... if (newContextConfig.excludeTopics) { const topicPatterns = newContextConfig.excludeTopics.map(t => t.toLowerCase()); newReposInContext = newReposInContext.filter(repo => { - const parsed = repoMetadataSchema.safeParse(repo.metadata); - if (!parsed.success) { + const repoTopics = getNormalizedRepoTopics(repo.metadata); + if (!repoTopics) { return true; } - const repoTopics = [ - ...(parsed.data.codeHostMetadata?.gitlab?.topics ?? []), - ...(parsed.data.codeHostMetadata?.github?.topics ?? []), - ]; - return !repoTopics.some(t => micromatch.isMatch(t.toLowerCase(), topicPatterns)); + return !repoTopics.some(t => micromatch.isMatch(t, topicPatterns)); }); }Also applies to: 133-145
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/backend/src/ee/syncSearchContexts.ts` around lines 74 - 86, Extract the repeated parse+topic-extraction+lowercase+matching logic into a shared helper used by both include and exclude paths: create a function (e.g., extractRepoTopics or getNormalizedRepoTopics) that takes a repo object, uses repoMetadataSchema.safeParse to validate, returns a normalized array of lowercase topic strings (combining parsed.data.codeHostMetadata.gitlab.topics and github.topics or empty array on parse failure), and then replace the inline logic in the includeTopics block (which currently builds topicPatterns and calls micromatch.isMatch) and the excludeTopics block to use this helper for matching with micromatch.isMatch; ensure the helper is imported/defined near syncSearchContexts so both places call the same code to avoid drift.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@docs/snippets/schemas/v3/index.schema.mdx`:
- Around line 154-179: The schema descriptions for includeTopics and
excludeTopics are GitLab-specific; update their "description" values to be
host-agnostic (referencing repositories or repository topics rather than
"GitLab") so they accurately reflect repository-level topic filtering across
hosts; modify the "includeTopics" and "excludeTopics" entries in the schema to
use neutral wording like "List of repository topics to include/exclude from the
search context. Repositories matching..." and make the same wording change for
the other occurrences of these fields (the similar block around lines 342-367)
to keep descriptions consistent.
In `@packages/backend/src/gitlab.test.ts`:
- Around line 144-156: The test currently asserts exclusion due to a casing
mismatch; update the test in gitlab.test.ts to expect the project is NOT
excluded (i.e., matching should be case-insensitive) and then modify the
implementation in packages/backend/src/gitlab.ts so shouldExcludeProject
normalizes project topics the same way it normalizes config topics before
matching: when evaluating include.topics and exclude.topics in
shouldExcludeProject, lowercase (or otherwise normalize) the entries from
project.topics as well as the config topics so comparisons are case-insensitive;
locate the matching logic inside shouldExcludeProject in gitlab.ts and apply the
same normalization helper used for config topics (or add one) to project.topics
before performing includes/excludes.
In `@packages/schemas/src/v3/index.schema.ts`:
- Around line 341-366: Update the descriptions for the includeTopics and
excludeTopics schema entries to use host-agnostic wording (e.g., "repository
topics") instead of "GitLab topics" so they match the base SearchContext
phrasing; locate the includeTopics and excludeTopics properties in
index.schema.ts and replace "List of GitLab topics..." with something like "List
of repository topics..." for both description fields to avoid implying
GitLab-only support.
In `@schemas/v3/searchContext.json`:
- Around line 47-72: Update the description strings for the includeTopics and
excludeTopics schema properties so they no longer say "GitLab topics" but
instead refer generically to "repository topics" (or state "GitHub and GitLab
repository topics") and keep the rest of the text intact; locate the
includeTopics and excludeTopics properties in the searchContext.json schema and
replace their description values accordingly so they match the generated type
files' wording.
---
Nitpick comments:
In `@packages/backend/src/ee/syncSearchContexts.test.ts`:
- Around line 93-95: Extract a small helper function (e.g., getConnectedIds) in
the test to read connected repo ids from the mocked upsert call instead of
repeating upsertCall.create.repos.connect.map(...); implement
getConnectedIds(upsertCallOrCreate) to accept either the full upsertCall (from
vi.mocked(db.searchContext.upsert).mock.calls[0][0]) or the create object and
return the mapped id array, then replace occurrences where you currently do
upsertCall.create.repos.connect.map((r: { id: number }) => r.id) with calls to
getConnectedIds(upsertCall) to reduce repetition and simplify assertions in
tests referencing db.searchContext.upsert and upsertCall.
In `@packages/backend/src/ee/syncSearchContexts.ts`:
- Around line 69-71: The loop that builds newReposInContext by concatenating
connection.repos currently defers deduplication until includeTopics path;
instead, normalize and deduplicate repo IDs once immediately before the upsert
to cover duplicates from connections and topics: collect all repo IDs into
newReposInContext (from connection.repos.map(repo => repo.repo) and any
includeTopics additions), then replace the repeated array concatenations with a
single Set-based dedupe (e.g., const uniqueRepos = Array.from(new
Set(newReposInContext))) and use uniqueRepos for the upsert call and anywhere
Line 173 reconstructs arrays so you can remove repeated rebuilds and switch to
Set lookups.
- Around line 74-86: Extract the repeated
parse+topic-extraction+lowercase+matching logic into a shared helper used by
both include and exclude paths: create a function (e.g., extractRepoTopics or
getNormalizedRepoTopics) that takes a repo object, uses
repoMetadataSchema.safeParse to validate, returns a normalized array of
lowercase topic strings (combining parsed.data.codeHostMetadata.gitlab.topics
and github.topics or empty array on parse failure), and then replace the inline
logic in the includeTopics block (which currently builds topicPatterns and calls
micromatch.isMatch) and the excludeTopics block to use this helper for matching
with micromatch.isMatch; ensure the helper is imported/defined near
syncSearchContexts so both places call the same code to avoid drift.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: b3cf0aef-13cb-47f7-a01c-21e0e13d56dd
📒 Files selected for processing (13)
docs/docs/features/search/search-contexts.mdxdocs/snippets/schemas/v3/index.schema.mdxdocs/snippets/schemas/v3/searchContext.schema.mdxpackages/backend/src/ee/syncSearchContexts.test.tspackages/backend/src/ee/syncSearchContexts.tspackages/backend/src/gitlab.test.tspackages/backend/src/repoCompileUtils.tspackages/schemas/src/v3/index.schema.tspackages/schemas/src/v3/index.type.tspackages/schemas/src/v3/searchContext.schema.tspackages/schemas/src/v3/searchContext.type.tspackages/shared/src/types.tsschemas/v3/searchContext.json
added 2 commits
March 23, 2026 10:47
brendan-kellam
approved these changes
Mar 23, 2026
Contributor
brendan-kellam
left a comment
brendan-kellam
left a comment
There was a problem hiding this comment.
Thanks for the high quality PR, this looks great!
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This commit updates Sourcebot to include support for using
topicsaspart of the Search Context definition.
As part of this:
repoMetadataSchemato storetopicsforgithubandgitlabhost typesincludeTopics/excludeTopicsFixes: #1027
N.B Code largely generated using Claude.
Summary by CodeRabbit
New Features
Documentation
Tests