Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support versioned index and improve name search #1378

Merged
merged 34 commits into from
Aug 25, 2023

Conversation

yeganathan18
Copy link
Member

@yeganathan18 yeganathan18 commented Aug 24, 2023

Changes proposed ✍️

What

🤖 Generated by Copilot at e5a06fb

This pull request improves the OpenSearch indexing process for the search_sync_worker service. It adds a function to trim long strings to avoid indexing errors, updates the types and settings for the indexes, and uses versioned indexes and aliases for each document type. It also adds new scripts to the package.json file for testing and debugging the indexing process. The changes affect the files activity.sync.service.ts, member.sync.service.ts, opensearch.service.ts, types.ts, package.json, and byteLength.ts.

🤖 Generated by Copilot at e5a06fb

To index data in OpenSearch
We added scripts and functions to fetch
We trimmed the strings
And added some things
To package.json and types.ts

Why

How

🤖 Generated by Copilot at e5a06fb

  • Add new function trimUtf8ToMaxByteLength to common library to truncate strings that exceed OpenSearch byte limit (link)
  • Use trimUtf8ToMaxByteLength to trim string attributes of activity and member documents before indexing them (link, link, link, link, link, link)
  • Add new scripts to package.json for running OpenSearch indexing tasks (link)
  • Modify OpenSearch service to use versioned indexes and aliases for managing documents (link, link, link, link, link, link, link, link, link)
  • Add new constant IndexVersions to types file to store version numbers for each index name (link)
  • Add new property analyzer to prefixedMapping object to specify custom analyzer for lowercasing keywords (link)
  • Add new object OPENSEARCH_INDEX_SETTINGS to types file to define settings for each index name (link)

Checklist ✅

  • Label appropriately with Feature, Improvement, or Bug.
  • Add screehshots to the PR description for relevant FE changes
  • New backend functionality has been unit-tested.
  • API documentation has been updated (if necessary) (see docs on API documentation).
  • Quality standards are met.

@yeganathan18 yeganathan18 added Bug Created by Linear-GitHub Sync Improvement Created by Linear-GitHub Sync labels Aug 24, 2023
@yeganathan18 yeganathan18 self-assigned this Aug 24, 2023
@yeganathan18 yeganathan18 removed the Bug Created by Linear-GitHub Sync label Aug 24, 2023
@yeganathan18 yeganathan18 merged commit 9748d10 into main Aug 25, 2023
9 checks passed
@yeganathan18 yeganathan18 deleted the bugfix/searchFullName branch August 25, 2023 01:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Improvement Created by Linear-GitHub Sync
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants