Implement Registry Multi-tenancy with Cognito in the loop

[jordanpadams]

💡 Description

Initial design and Serverless, schema setup:

• Design authorization handshake with Cognito, API Gateway, and Lambda for multi-tenancy approach #179
• Initial OpenSearch Serverless Setup and Deployment in MCP #223
• Search for a way to have OpenSearch aliases so that registry-dd can be accessed through <node>-registry-dd name #268

OpenSearch Serverless Authentication

• Implement Cloud Resources for OpenSearch Serverless Authentication #258

Tools adaptations

Loader Tools

• Update to utilize new multi-tenancy approach registry-mgr#66
• Update to utilize new multi-tenancy approach harvest#118
• Add schema reference to all examples to take advantage of new schema harvest#156

API

• Update to utilize new multi-tenancy approach registry-api#304
• As a web developer, I want to access read-only any feature provided by opensearch registry-api#433

Sweeper

• Registry-sweeper upgrade for multitenant registry registry-sweepers#120
• Registry-Sweeper ECS Enchancements (Post Multi-Tenancy) registry-sweepers#60
• Implement ECS task concurrency prevention for registry-sweepers planetary-data-cloud#105

Other

• As a data manager, I want to query OpenSearch Serverless from the command-line #273

Additional Testing

• Respond to User Questions Regarding Supporting Migration to OpenSearch Serverless #275
• Re-test validate-refs using populated multi-tenant registry validate#621
• TBD Deep Archive smoke test
• Upgrade docker compose as needed for the multitenant setup #269
• Smoke tests to ensure the migration is successful #279

Deployment and Data Migration

• Deploy Multi-tenant Registry to production and sync data #271
• Deploy Multi-tenant Registry to test #272
• Create snapshots of production OpenSearch clusters prior to shutdown #278
• Copy data necessary for web mod efforts to search-en registry prior to shutdown #277

Cutover planning

• Write a migration plan to multitenant registry #217

[jordanpadams]

📆 April Status: Completion date expected to be delayed 1 week due to technology comparison needed. No impact on other tasks or deliveries

[jordanpadams]

📆 May status: Delayed 1 sprint due to increase in scope and complexity. Solution has been identified and is underway. Slack built into schedule should alleviate delays. No impact on delivery expected.

[jordanpadams]

📆 June status: Delayed an additional sprint due to increasing scope and learning-on-the-fly with OpenSearch authorization. No impact on build anticipated.

[jordanpadams]

📆 July status: Long delay due to Registry / API issues causing block in #179. Full deployment with authorization layer to be delayed, but partial deployment expected by delivery date. This delivery will actually cause less impact on users, so the partial deployment will have no effect on success of delivery.

[jordanpadams]

📆 August status: Working this architecture and deployment with MCP SAs. Expected completion by end of month.

[jordanpadams]

📆 September status: tightly coupled with NASA-PDS/planetary-data-cloud#35, this is in work. Continuing to work issues with serverless approach and AWS team.

[jordanpadams]

📆 October Status: tightly coupled with NASA-PDS/planetary-data-cloud#35, this is in work, but blocked by AWS technical team response times and issues being encountered with the service

[jordanpadams]

📆 November status: Finally unblocked by NASA-PDS/planetary-data-cloud#35, and migration is underway. Working on user management right now. On schedule.

[jordanpadams]

📆 December status: In work. On schedule.

[jordanpadams]

📆 01/2024 status: In work. Delayed 2 sprints. Will impact deferral of NASA-PDS/registry-api#285

[jordanpadams]

📆 02/2024 status: In work. Should be completed be end of sprint (3/7)

[jordanpadams]

📆 03/2024 status: Delayed 2 additional sprints as this is a blocker for the build delivery. I&T will begin on time, but the team will test this last.