chg(opensearch): drop unknown fields instead of auto-creating#2690
Merged
Conversation
Set dynamic=false on all index mappings (channels, documents, chats, emails, emails_v2) so writes with fields not in the schema are stored in _source but not indexed. Adds a set_dynamic_false.ts migration script to apply this setting to existing indices.
Contributor
📝 WalkthroughWalkthroughThese changes disable dynamic field creation across OpenSearch indices by adding Changes
🚥 Pre-merge checks | ✅ 2✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@infra/stacks/opensearch/helpers/scripts/set_dynamic_false.ts`:
- Around line 12-18: Extract the literal 'emails_v2' into a shared constant
named EMAIL_INDEX_V2 in the project's helpers/constants module and replace the
hard-coded string in the INDICES array with an import of EMAIL_INDEX_V2; then
update the other location that currently defines EMAIL_INDEX_V2 (the
create_emails_v2 module) to import that same EMAIL_INDEX_V2 constant instead of
redeclaring it so both the INDICES constant and create_emails_v2 reference a
single source of truth.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: c5930536-afbf-4244-bff7-68ed815cfc3c
📒 Files selected for processing (3)
infra/stacks/opensearch/helpers/scripts/create_emails_v2.tsinfra/stacks/opensearch/helpers/scripts/create_indices.tsinfra/stacks/opensearch/helpers/scripts/set_dynamic_false.ts
Comment on lines
+12
to
+18
| const INDICES = [ | ||
| CHAT_INDEX, | ||
| DOCUMENT_INDEX, | ||
| EMAIL_INDEX, | ||
| CHANNEL_INDEX, | ||
| 'emails_v2', | ||
| ]; |
Contributor
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
Extract 'emails_v2' to a shared constant to avoid string duplication.
The literal 'emails_v2' is also defined as EMAIL_INDEX_V2 in create_emails_v2.ts (line 6). Promote it to helpers/constants.ts alongside the other *_INDEX names so both scripts reference a single source of truth — this prevents drift if the index name ever changes.
♻️ Suggested change
In infra/stacks/opensearch/helpers/constants.ts:
const CHAT_INDEX = 'chats';
const DOCUMENT_INDEX = 'documents';
const EMAIL_INDEX = 'emails';
+const EMAIL_INDEX_V2 = 'emails_v2';
const CHANNEL_INDEX = 'channels';
-export { CHAT_INDEX, DOCUMENT_INDEX, EMAIL_INDEX, CHANNEL_INDEX };
+export { CHAT_INDEX, DOCUMENT_INDEX, EMAIL_INDEX, EMAIL_INDEX_V2, CHANNEL_INDEX };Then in this file:
import {
CHANNEL_INDEX,
CHAT_INDEX,
DOCUMENT_INDEX,
EMAIL_INDEX,
+ EMAIL_INDEX_V2,
IS_DRY_RUN,
} from '../constants';
const INDICES = [
CHAT_INDEX,
DOCUMENT_INDEX,
EMAIL_INDEX,
CHANNEL_INDEX,
- 'emails_v2',
+ EMAIL_INDEX_V2,
];And update create_emails_v2.ts to import EMAIL_INDEX_V2 from constants rather than redeclaring it locally.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@infra/stacks/opensearch/helpers/scripts/set_dynamic_false.ts` around lines 12
- 18, Extract the literal 'emails_v2' into a shared constant named
EMAIL_INDEX_V2 in the project's helpers/constants module and replace the
hard-coded string in the INDICES array with an import of EMAIL_INDEX_V2; then
update the other location that currently defines EMAIL_INDEX_V2 (the
create_emails_v2 module) to import that same EMAIL_INDEX_V2 constant instead of
redeclaring it so both the INDICES constant and create_emails_v2 reference a
single source of truth.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
dynamic: 'false'to all OpenSearch index mappings (channels, documents, chats, emails, emails_v2) so writes with fields not in the schema are stored in_sourcebut not indexed — no more accidental auto-mapping of unknown fields.Note:
falsedrops unknown fields silently. If we want writes to fail loudly on unknown fields instead, swap tostrict.Im running the update manually