Enhancement/organizations in opensearch c 1978

[themarolt]

Changes proposed ✍️

What

🤖 Generated by Copilot at 64ea737

This pull request adds the functionality to sync and index the organization data from the database to the opensearch service, using the search sync worker and the nodejs worker services. It also fixes a bug in the frontend filter for the organization data, and adds some scripts and tests for the sync and cleanup operations. It modifies several files in the backend, frontend, and services modules, such as organizationRepository.ts, organizationService.ts, member.service.ts, organization.sync.service.ts, organization.data.ts, main.ts, and package.json. It also adds some new files, such as organization.repo.ts, segment.repo.ts, segment.data.ts, and U1691485202__organization-sync-timestamp.sql. It refactors some existing files, such as member.repo.ts, member.sync.service.ts, member.data.ts, and fieldTranslator.ts.
​

🤖 Generated by Copilot at 64ea737

Sing, O Muse, of the mighty deeds of the developers
Who labored hard to sync the data of organizations
With the opensearch service, the swift and powerful engine
That searches and filters the data with skill and precision

Why

How

🤖 Generated by Copilot at 64ea737

• Add and implement logic for syncing organization data to opensearch service (link, link, link, link, link, link, link, link, link, link, link, link, link, link, link, link, link, link, link, link, link)
• Create OrganizationSyncService class to provide methods for syncing, cleaning, and removing organization data from opensearch service (link, link, link)
• Add organization-related message types and cases to WorkerQueueReceiver class to handle organization sync events from SQS queue (link, link, link)
• Add OrganizationRepository class to query and manipulate organization data from database (link, link)
• Add searchSyncedAt column to organizations table and corresponding migration scripts (link, link)
• Add scripts to package.json file of search sync worker service to run sync and cleanup operations for organization data (link)
• Add OpenSearchIndex.ORGANIZATIONS enum value and corresponding mapping to opensearch service types and classes (link, link, link)
• Modify OrganizationService class to trigger organization sync events using SearchSyncWorkerEmitter class after creating, updating, or deleting organization records (link, link, link, link, link)
• Modify MemberService class to trigger organization sync events using SearchSyncWorkerEmitter class after updating member organizations (link, link, link)
• Modify OrganizationService class to use OrganizationRepository.findAndCountAllOpensearch method to query and paginate organization data from opensearch service instead of database (link)
• Modify InitService class to create and index a fake organization record for testing purposes (link, link)
• Refactor segment data logic and types from MemberRepository and MemberSyncService classes to separate SegmentRepository and SegmentData classes (link, link, link, link, link, link, link, link, link, link)
• Create SegmentRepository class to query and manipulate segment data from database (link)
• Create IDbSegmentInfo interface to define data type for segment data in database and opensearch service (link)
• Move getParentSegmentIds method from MemberRepository class to SegmentRepository class (link, link)
• Modify MemberSyncService class to use SegmentRepository class instead of MemberRepository class to get parent segment ids (link, link, link)
• Remove IDbSegmentInfo interface import from member.data.ts and member.repo.ts files (link, link)
• Add IDbSegmentInfo interface import to member.sync.service.ts file (link)
• Fix bug in organization search filter by replacing name field with displayName field in organizationSearchFilter function (link)
• Add imports of FieldTranslatorFactory and OpensearchQueryParser classes to organizationRepository.ts file to translate and parse queries for opensearch service (link)
• Add imports of isFeatureEnabled and FeatureFlag functions to organizationRepository.ts file to check if segments feature is enabled (link)
• Add import of getSearchSyncWorkerEmitter function to organizationService.ts file to get instance of SearchSyncWorkerEmitter class (link)
• Add import of IMemberAttribute interface to member.repo.ts file to define data type for member attributes (link)
• Add preventNestedFieldTranslation property to modelField interface in fieldTranslator.ts file to prevent translation of some fields that are already in crowd format (link)

Checklist ✅

• Label appropriately with Feature, Improvement, or Bug.
• Add screehshots to the PR description for relevant FE changes
• New backend functionality has been unit-tested.
• API documentation has been updated (if necessary) (see docs on API documentation).
• Quality standards are met.