Skip to content

Fail fast on mixed id types for mongo connector. - #382

Merged
adiom-mark merged 1 commit into
mainfrom
mixedid-failfast
Apr 29, 2026
Merged

Fail fast on mixed id types for mongo connector.#382
adiom-mark merged 1 commit into
mainfrom
mixedid-failfast

Conversation

@adiom-mark

@adiom-mark adiom-mark commented Apr 28, 2026

Copy link
Copy Markdown
Collaborator

Summary by CodeRabbit

  • Bug Fixes

    • Enhanced MongoDB synchronization plan generation with improved ID type validation across collections.
  • Performance

    • Optimized initial sync by pre-calculating ID boundaries upfront, eliminating redundant computations.

@coderabbitai

coderabbitai Bot commented Apr 28, 2026

Copy link
Copy Markdown
📝 Walkthrough

Walkthrough

During initial sync plan generation, the code now computes both the smallest and largest _id values upfront per collection using a new findSmallestID helper function. It validates that if both bounds exist, they share the same bson.RawValue type, returning an error on mismatch. Previous logic that recomputed the largest _id inside a low-count branch is removed since high is now computed earlier.

Changes

Cohort / File(s) Summary
MongoDB Bounds Computation
connectors/mongo/conn.go
Added findSmallestID helper function and refactored plan generation to determine both smallest and largest _id values upfront. Validates type consistency between bounds and removes redundant recomputation logic from the low-count branch.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

  • Add upper bound to mongo queries #356: Also modifies mongo connector bounds logic in connectors/mongo/conn.go—adds findLargestID and uses upper bound for partition cursors, complementing this PR's findSmallestID addition for bidirectional bound handling.

Poem

🐰 Hop, hop, the bounds align!
Smallest and largest, now by design,
Types must match, or errors ring true,
Efficient syncing through and through!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'Fail fast on mixed id types for mongo connector' directly and concisely describes the main change: adding logic to detect and fail on mixed _id types in the MongoDB connector.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch mixedid-failfast

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@connectors/mongo/conn.go`:
- Around line 260-270: The _id type validation currently checks the entire
collection but the sync read path uses the filtered query (c.query), so change
the validation to use the same filter scope: update findSmallestID and
findLargestID to accept a filter parameter (e.g., bson.D) and pass c.query (the
same filter used in the read path) when calling them from the validation block
that currently references partition.Namespace; ensure the implementations use
the provided filter in their Mongo queries so mixed _id types outside the filter
won't cause a false-positive error.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 061ccffb-2870-47b9-8fe9-e95cbe3f11b0

📥 Commits

Reviewing files that changed from the base of the PR and between 0dab9f2 and b98b345.

📒 Files selected for processing (1)
  • connectors/mongo/conn.go

Comment thread connectors/mongo/conn.go
Comment on lines +260 to +270
lowest, err := findSmallestID(ctx, col)
if err != nil {
return fmt.Errorf("err finding smallest id: %w", err)
}
high, err := findLargestID(ctx, col)
if err != nil {
return fmt.Errorf("err finding largest id: %w", err)
}
if !lowest.IsZero() && !high.IsZero() && lowest.Type != high.Type {
return fmt.Errorf("mixed _id types not supported in %v: found %v and %v", partition.Namespace, lowest.Type, high.Type)
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Fail-fast mixed _id validation should use the same filter scope as sync reads.

At Line 260-Line 270, _id type validation runs on the full collection, while read path filtering uses c.query (Line 460-Line 463). This can fail planning for a valid filtered sync if mixed types exist outside the filter.

Proposed fix
-			lowest, err := findSmallestID(ctx, col)
+			planFilter := bson.D{}
+			if len(c.query) > 0 {
+				planFilter = append(planFilter, c.query...)
+			}
+			lowest, err := findSmallestID(ctx, col, planFilter)
 			if err != nil {
 				return fmt.Errorf("err finding smallest id: %w", err)
 			}
-			high, err := findLargestID(ctx, col)
+			high, err := findLargestID(ctx, col, planFilter)
 			if err != nil {
 				return fmt.Errorf("err finding largest id: %w", err)
 			}
// Update helpers to accept a filter:
func findSmallestID(ctx context.Context, col *mongo.Collection, filter bson.D) (bson.RawValue, error)
func findLargestID(ctx context.Context, col *mongo.Collection, filter bson.D) (bson.RawValue, error)
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@connectors/mongo/conn.go` around lines 260 - 270, The _id type validation
currently checks the entire collection but the sync read path uses the filtered
query (c.query), so change the validation to use the same filter scope: update
findSmallestID and findLargestID to accept a filter parameter (e.g., bson.D) and
pass c.query (the same filter used in the read path) when calling them from the
validation block that currently references partition.Namespace; ensure the
implementations use the provided filter in their Mongo queries so mixed _id
types outside the filter won't cause a false-positive error.

@adiom-mark
adiom-mark enabled auto-merge (squash) April 29, 2026 16:00
@adiom-mark
adiom-mark disabled auto-merge April 29, 2026 16:00
@adiom-mark
adiom-mark merged commit ce716e6 into main Apr 29, 2026
2 checks passed
@adiom-mark
adiom-mark deleted the mixedid-failfast branch April 29, 2026 16:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant