Skip to content

Update production-solr-collection-swap-process.md to reflect Dag name… #1706

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 30, 2025
Merged

Update production-solr-collection-swap-process.md to reflect Dag name… #1706

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions doc/production-solr-collection-swap-process.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,13 +12,13 @@ These steps are broken down into three sections as these parts can be run sequen
1. Skip this step if you want to reuse the s3 alma_sftp files from the last run.
1. Trigger the `boundwith_move_alma_sftp_to_s3` Dag and wait for completion.
1. Skip this step if you want to reuse the s3 alma_sftp files from the last run.
1. Turn off the catalog_preproduction_oai_harvest
1. Turn off the `catalog_pre_production_oai_harvest_v2` Dag.
1. Set the Airflow variables required for the full reindex:
1. Ensure the variable `CATALOG_PRE_PRODUCTION_SOLR_CONFIG` is using the correct version of solr configs. See the [tul_cob-catalog-solr releases](https://github.com/tulibraries/tul_cob-catalog-solr/releases) to confirm latest version, if needed.
1. Ensure the variable `PRE_PRODUCTION_COB_INDEX_VERSION` is the correct version of cob_index. See the [cob_index releases](https://github.com/tulibraries/cob_index/releases) to confirm latest version, if needed.
1. Update the `CATALOG_PRE_PRODUCTION_SOLR_COLLECTION` variable to be None.
1. Trigger `catalog_full_reindex` Dag and wait for completion.
1. Once a new preproduction Solr collection is created via `catalog_full_reindex`, turn on the `catalog_preproduction_oai_harvest` Dag.
1. Once a new preproduction Solr collection is created via `catalog_full_reindex`, turn on the `catalog_pre_production_oai_harvest_v2` Dag.

### Caveats and Incidentals
* `cob_datapipeline` deployments to airflow-prod only happen on new releases for that repo (plus verified CI deployment step).
Expand All @@ -29,15 +29,15 @@ If something goes wrong with the full reindex and you need to rerun it from the

## Testing

1. When testing a preproduction Solr collection, ensure the `catalog_preproduction_oai_harvest` Dag is on and indexing updates to that collection.
1. When testing a preproduction Solr collection, ensure the `catalog_pre_production_oai_harvest_v2` Dag is on and indexing updates to that collection.
1. Create a PR to `tul_cob` changing CATALOG_COLLECTION in .env to use the updated Airflow variable `CATALOG_PRE_PRODUCTION_SOLR_COLLECTION` and merge/deploy to Library Search QA.
1. Test changes on Library Search QA and get approval from any relevant stakeholders/managers for a release.

## Deploy to Production

1. Before proceeding, turn off the Pre Production and Production OAI Harvest Dags
1. Before proceeding, turn off the `catalog_pre_production_oai_harvest_v2` and `catalog_production_oai_harvest_v2` Dags.
1. Check that the gitlab HELM_VERSION_PROD variable is up-to-date (consult with colleagues if needed).
1. Create release of `tul_cob` and wait for deployment to complete to Library Search production.
1. The deployment process will automatically update the Airflow variable `CATALOG_PRODUCTION_SOLR_COLLECTION` to the same value as `CATALOG_PRE_PRODUCTION_SOLR_COLLECTION`
1. After the deployment is complete, update the Airflow variable `CATALOG_PROD_HARVEST_FROM_DATE` to the same value as `CATALOG_PRE_PRODUCTION_HARVEST_FROM_DATE` and update the Airflow variable `PROD_COB_INDEX_VERSION` to the same value as `PRE_PRODUCTION_COB_INDEX_VERSION`.
1. Turn the Production OAI Harvest back on.
1. Turn the `catalog_production_oai_harvest_v2` Dag back on.