Sprint Reviews
- DUM upgrades (Scott)
- Nucleus automated deployment (Ramesh)
- Multitenant registry
- Plan/schedule/communication with the nodes (Jordan, Thomas)
- Data migration (Sagar)
- harvest/registry-mgr (Al)
- sweeper (Alex)
- api (Thomas)
- NASA SMD Software workshop, API lightning talk and overall report (Thomas)
- PDS4 Information Model upgrades (Steve)
- Web analytics pipelines improvments (Kai)
- Web modernization implementation (Anil, Eddie)
- Baseline IAM policies (Sagar)
- Data Upload Manager upgrades (Scott)
- Loader tools (harvest/registry-mgr) for multi-tenant registry (Al)
- Nucleus updates (Ramesh)
- S3 browser (Eddie)
- Web modernization implementation updates (Anil, Eddie)
- NASA A&A (Jordan)
- Keyword search on legacy registry (Jordan)
- Continuous Integration for nodejs, on dummy repository (Sean)
- LDD bug fixes (Steve, Jordan)
- Nucleus updates (Ramesh)
- Multitenant updates (Sagar, Al)
- Registry-api, integration of new OpenSearchClient, first step (Thomas)
- Web modernization implementation updates (Anil, Eddie)
- Upload CSS data on the cloud with DUM (Vivian, Ramesh)
- add deep-archive client to registry-api branch CICD test (Jordan)
- CodeQL static analysis (Jordan)
- Summer Internship on GIS (Thomas)
- Web modernization implementation status (Anil, Eddie)
- Registry-Sweepers improvements, memory, ... (Alex)
- Deep-archive bug fix and api dependency upgrade (Sean)
- RDD automated generation fix (Thomas, Jordan)
- 500 error on registry-api fix (Alex)
- MI-label bug fix (Jordan)
- Nucleus (Ramesh, all) 20'
- sprint objectives:
- DUM load to S3 staging bucket and synchronize to ODR s3 bucket
- (out of scope for now) at bundle level: pds-deep-archive, registry-mgr release.
- DAG at product level:
- sprint objectives:
For CSS:
- validate
- validate-ref (on-hold)
- harvest
- glacier
- copy data to ODR
- remove data from EFS
- remove staging s3 bucket
- Multitenant Registry (Sagar, Al, all) 20'
-
Open Data Registry, what is it ? How is it useful for PDS/CSS (Vivian)
-
Incident Response Plan (Jordan)
-
Build 14.1 plan update submitted to Software Working Group (Jordan)
-
Validate on Windows (Jordan)
-
Multitenants Registry updates (Sagar)
-
Documentation of archive status for registry (Thomas)
-
Nucleus udpdates (Ramesh)
-
property list in pds-api-client (Thomas)
-
Web modernization updates (Anil, Eddy)
-
What are your takes on the Software Symposium (all)
-
Other
Recording: TBD
-
PDS Engineering Node web site migrated and 508 compliance (Catherine)
-
Archive Browser with AWS SDK, progress so far (Eddie)
-
New web design implementation, progress so far (Anil)
-
Nodejs repo template and roundup integration (Sean)
-
CORS issues on feedback widget (Thomas)
-
Harvest updated job configuration for multi-tenant opensearch (Al, Jordan, Thomas)
-
Nucleus documentation on gh-pages (Ramesh):
Web implementation sprint to be detailed
- Press It Up: Web modernization development initial setup with Wordpress (Anil)
- Buckets for Days: Latest greatest web directory page for S3 buckets (Eddie)
- Paging 1995: New pagination interface and implementation for the registry-api (Alex)
- Sweepers Creepers: Consistent properties as list vs string in registry with sweepers (Alex)
- Checking In: Review the latest updates to Validate (Jordan)
- State of Nucleus Address: Latest Nucleus Updates and "end-to-end" execution (Ramesh)
- Fresh Rolls: Best Practices in creating IAM Roles and Security Groups (Sagar)
- Registry Design-a-thon: Multi-tenant Registry Architecture Updates for supporting signed URLs (Sagar)
Meeting held on 2023-11-30 due to holiday Recording: https://jpl365prod-my.sharepoint.com/:v:/g/personal/jordan_h_padams_jpl_nasa_gov/EYw3imfWNp9IlGgVIQqJk4MBv79cyNb5NvQJp39JQDi7Xg?referrer=Teams.TEAMS-ELECTRON&referrerScenario=MeetingChicletGetLink.view.view
- Web modernization design update (Mike/Edward)
- API wrapper PoC (Thomas)
- NAIF node missing in legacy dashboards (Thomas)
- Nucleus with fitz files (compressed), Nucleus cost issue and alternate design proposal (Ramesh)
- Re-try behavior on harvest (Alex)
- Build 14.0 I&T, TRR, DDR (Vivian, Miguel, Gary, Thomas)
- Don't leak your secrets (Sean)
- Updates on MCP (Sagar)
- Encryption of EC2 instances
- OIDC service
- Data Upload Manager status and documentation (Scott)
- Nucleus integration with Data Upload Manager (Ramesh), successful workflow with Messenger as planned for the CCS meeting
- First iteration of the provenance model in PDS4 (Steve)
- Release note with known bugs, for example https://github.com/NASA-PDS/registry-api/releases/tag/v1.3.1 (Thomas)
- Deep archive, installation instructions for Windows (Sean)
Recording: TBD
- Cutover to using new Registry API for generating context products json (Al, Jordan): cancelled, not working yet because API is slow over a few pages of results.
- Avoid race condition in the product checker (Ramesh)
- DPS presentations (Jordan: slides, Thomas: poster)
- Sinequa useability analysis (Joe)
- Navigation app bar in new web (Anil)
- Create a stable release for the registry, what does that mean ? (Thomas)
- ISA status (Jordan)
- Web Prototype (Mike)
- Web analytics (Kai)
- Web portal architecture with WordPress (Anil)
- Vulnerability fix from dependabot alerts on web development (Anil)
- Request the Sinequa API for search (Joe)
- Registry Automated Tests maintenance (Thomas)
- Registry-sweepers/RepairKit tuning for production (Alex, Sagar)
- DUM: Infrastructure as Code (Scott)
Recording: TBD
- Product Completion Checker lambda for Nucleus (Ramesh)
- New LDD (LRO, mgn) (Catherine, Steve)
- RepairKit optimization for production (Alex)
- Registry loading status dasboard in Kibana (Thomas)
- CloudWatch log monitoring for Registry-Sweepers & Registry-API using AWS Lambda / SNS (Sagar)
- Messenger data visualization with the PDS Web API and LeafMap (Joaquin)
- IPDA new web site published, the steps between after development to publication (Sean, video link (JPL only), PDF of slides)
- DUM: Parallelization of upload (Scott)
Recording: TBD
- The balkanization of pds-github-utils (Sean)
- Add TermMaps to default output JSON (Steve)
- Harvest tags the loaded products with its version number (Al, Thomas)
- Performance improvement on registry-sweepers (Alex)
- New alerts on registry-api and sweepers (Sagar)
- Nucleus, workflow triggers (Ramesh)
- Registry-api, JSON for everyone (Al, Thomas)
- Read images with the registry-api (Joaquin)
- Published results on the LLM tests for PDS Search (Amanda)
Zenhub says:
Demos:
- Updated npm package architecture for web UIs (Eddie, Anil)
- Updates and plans for Magnetometers analysis from the registry-api (Joaquin)
- Semantic Search/LLM, similarities results (Amanda)
- Analysis for product-triggered workflow for nucleus (Ramesh)
- Standards for describing AirFlow DAGs, options (Ramesh)
- Alerts on almost full storage on AWS (Sagar)
- Registry Docker compose for multiple architectures (Sean); presentation available as video (JPL-only), or as PDF
- Continuous Deployment with terraform on NGAP (Sagar) 5′
- Registry-api postman tests reports in TestRail (Sean) 5′
- Updated Sitemap for PDS web site (Catherine, Sean) 5′
- Use signed-url/cognito to connect to opensearch (Alex) 5′
- Registry-sweeper, what it does, how it is deployed (Alex) 5′
- Registry-api OVIRS notebook, OVIRS data loaded and new pds.api-client (Thomas) 5′
- Registry-api notebook: plot Galileo magnetometer observations extrcted from the registry-api with pyWWT (Joaquin) 5′
- Data Upload Manager: monitor the client commands launched by the Discipline Nodes (Scott) 5′
- Validate validates video labels and files (Al, Jordan) 5′
- Registry loader procedures documented , see https://github.com/NASA-PDS/operations/issues/359 (Vivian) 5′
- Semantic Search: Compute embedding vectors on PDS4 labels and user's search term, status and plan for next sprint (Amanda) 5′
- Sinequa Search (Joe) 5′
- Other:
- release build 14.0 notification
- PDS Phone book
- Registry architecture diagram https://lucid.app/lucidchart/d2d7eb2a-352b-4105-81d6-5694f3d50f8a/edit?viewport_loc=-238%2C-107%2C2440%2C1269%2C66uWzmvPQEeN&invitationId=inv_0ca74609-afa7-44b5-ae44-2a1c095f8f16
- jenkins vs harness (vs github actions) ?
- Messenger data loaded by Nucleus/Ramesh (to be used by Joaquin)
- DOI-Editor, soon available in production, what are the next steps ?
- Schedule a point build and deployment for registry-api (after bug fixes)
- Search architecture, how to move forward, sinequa/opensearch/semantic search/graph database (https://github.com/NASA-PDS/portal-tasks/issues/72) ...
- Start archive viewer development
List to be completed the morning of the Sprint review.
The demos are 5 minutes max but less is also ok, with short word on something which was completed, a screenshot, an explanation of what it is useful for:
-
Latest development of the Search Prototype (Eddie, presented by Thomas) 5'
-
LLMs for PDS, internship plan (Amanda) 5'
-
Nucleus (Ramesh): 5' [] share data between steps [] logging organization and presentation to users/operators
-
New test dataset for Registry integration tests (Thomas) 5'
-
Registry-api notebooks reaching our user's community (Joaquin) 5'
-
New health check end-point (Jimmie) 5'
-
Miscellaneous, 1 screenshot each: 5'
- B13.1 software release (Thomas)
- JPL logo on PDS ENG wordpress (Catherine)
- Organize a happy hour and hurt oneself so not to be sure to attend (Jordan)
Pushed to next sprint review because of training or sickness:
- Registry-sweeper, what it does, how it is deployed (Alex) 5'
- Continuous Deployment with terraform on NGAP (Sagar) 5'
- Registry-api postman tests reports in TestRail (Sean) 5'
Detailed review of the remaining open ticket by Thomas and Jordan. Next sprint only blockers should be raised by the team. Thomas and Jordan will review the remaining tickets offline or in a different meeting.