Jk/cumulus 3135 fix integration tests #3403
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jennyhliu
approved these changes
Jun 5, 2023
etcart
approved these changes
Jun 5, 2023
Jkovarik
added a commit
that referenced
this pull request
Jun 9, 2023
* Update integration test scripts to fail on test timeout * Fixup * Fixup * Update script interpreter for test runs * Fix script * Fixup * Fixup * Update api/client and integration test usage of it to fix test failures * Fix formatting/lint/etc * Update test/minor fix --------- Co-authored-by: etcart <37375117+etcart@users.noreply.github.com>
npauzenga
pushed a commit
that referenced
this pull request
Jun 12, 2023
* CUMULUS-3135 - Update integration test scripts to fail on test timeout (#3401) * Update integration test scripts to fail on test timeout * Fixup * Fixup * Update script interpreter for test runs * Fix script * Fixup * Fixup * Update timeout pass/fail conditional * Jk/cumulus 3135 fix integration tests (#3403) * Update integration test scripts to fail on test timeout * Fixup * Fixup * Update script interpreter for test runs * Fix script * Fixup * Fixup * Update api/client and integration test usage of it to fix test failures * Fix formatting/lint/etc * Update test/minor fix --------- Co-authored-by: etcart <37375117+etcart@users.noreply.github.com> --------- Co-authored-by: etcart <37375117+etcart@users.noreply.github.com>
npauzenga
added a commit
that referenced
this pull request
Jul 14, 2023
* [CUMULUS-3226] Remove refs to async operations dynamo table (#3331) * Remove refs to async operations dynamo table * add changelog entry * Remove additional async operation table refs * Update process_dead_letter_archive role * CUMULUS-3053: removing references to dynamodb from docs (#3322) * CUMULUS-3053: removing references to dynamodb from docs * resolving lint errors * CUMULUS-3285: Updated isAuthBearTokenRequest to handle non-Bearer authorization header (merge to master) (#3350) (#3352) * CUMULUS-3285: Updated isAuthBearTokenRequest to handle non-Bearer authorization header (#3341) * Jk rel16/cumulus 3079 (#3357) * Update endpoints/granules :granuleName parameter to :granuleId (#3333) * Move PUT logic to PATCH. Implement error for PUT endpoint * Add put units/refactor * Update API client to use PATCH protocol cumulus's api is moving all existing PUT requests to PATCH. This commit updates the api-client to use those, adds 'PATCH' to typings and updates the test fixtures for the thin unit tests. * Fix broken API unit * Update CHANGELOG * Fix naming/minor refactor * Fix Dynamodb granule.files default bug * Fix elasticsearch default array write value * Update/add PUT endpoint and relevant tests * Update CHANGELOG * Update unit test to take advantage of fixed unit helper * Update API version * Add api-client method to use updated PUT endpoint * Add fixed annotations to CHANGELOG * Update API version unit * Minor comment refactor * Add PUT API spec test * Fix incorrect test label * Add missing unit tests * Add removed unit tests * Jk/cumulus 3072 add header check (#3257) * Add API version compliance checks to app middleware * Update middleware/add test defaults/fix typing * Fix broken unit tests * cleanup * Add middleware units * Add endpoint tests to validate middleware * Fix unit test * Remove unneeded type exports * Remove unneeded test * Remove async * Minor formatting fix/changeset reduction * Add missing ts-check * Update CHANGELOG * Fix bad import * Update 'version' header to 'Cumulus-API-Version' * Respond to PR feedback * validateApiCompliance -> requireApiVersion * Update packages/api/lib/request.js Co-authored-by: Marc <marc@element84.com> * Fix isMinVersionApi expression * Update granule units/fix prior test fixtures * Update packages/api/app/middleware.js Co-authored-by: Marc <marc@element84.com> * Fix spacing * Add api-client required headers * Update header restriction with new name, fix api-package header inclusions * Fix broken unit * Bring in fix from #3270 * Update packages/api/app/middleware.js Co-authored-by: Marc <marc@element84.com> * isNumber -> Number.isFinite --------- Co-authored-by: Marc <marc@element84.com> * Update PUT to use collectionId endpoint *only* * Update CHANGELOG * Fix api-client to call right PUT endpoint * Remove unneeded log output * Update tests, add granuleId parameter updates * Fix bad logic assertion * Update endpoints/granules :granuleName parameter to :granuleId * Fix lint 🔔 * Fix lint * Fix merged unit test * Fix merged comments --------- Co-authored-by: Marc <marc@element84.com> * Reorder/updated CHANGELOG * Minor P3 release edits --------- Co-authored-by: Marc <marc@element84.com> * Cumulus 3120 bugfix (#3364) * [CUMULUS-3226] Remove refs to async operations dynamo table (#3331) * Remove refs to async operations dynamo table * add changelog entry * Remove additional async operation table refs * Update process_dead_letter_archive role * Update endpoints/granules :granuleName parameter to :granuleId (#3333) * Move PUT logic to PATCH. Implement error for PUT endpoint * Add put units/refactor * Update API client to use PATCH protocol cumulus's api is moving all existing PUT requests to PATCH. This commit updates the api-client to use those, adds 'PATCH' to typings and updates the test fixtures for the thin unit tests. * Fix broken API unit * Update CHANGELOG * Fix naming/minor refactor * Fix Dynamodb granule.files default bug * Fix elasticsearch default array write value * Update/add PUT endpoint and relevant tests * Update CHANGELOG * Update unit test to take advantage of fixed unit helper * Update API version * Add api-client method to use updated PUT endpoint * Add fixed annotations to CHANGELOG * Update API version unit * Minor comment refactor * Add PUT API spec test * Fix incorrect test label * Add missing unit tests * Add removed unit tests * Jk/cumulus 3072 add header check (#3257) * Add API version compliance checks to app middleware * Update middleware/add test defaults/fix typing * Fix broken unit tests * cleanup * Add middleware units * Add endpoint tests to validate middleware * Fix unit test * Remove unneeded type exports * Remove unneeded test * Remove async * Minor formatting fix/changeset reduction * Add missing ts-check * Update CHANGELOG * Fix bad import * Update 'version' header to 'Cumulus-API-Version' * Respond to PR feedback * validateApiCompliance -> requireApiVersion * Update packages/api/lib/request.js Co-authored-by: Marc <marc@element84.com> * Fix isMinVersionApi expression * Update granule units/fix prior test fixtures * Update packages/api/app/middleware.js Co-authored-by: Marc <marc@element84.com> * Fix spacing * Add api-client required headers * Update header restriction with new name, fix api-package header inclusions * Fix broken unit * Bring in fix from #3270 * Update packages/api/app/middleware.js Co-authored-by: Marc <marc@element84.com> * isNumber -> Number.isFinite --------- Co-authored-by: Marc <marc@element84.com> * Update PUT to use collectionId endpoint *only* * Update CHANGELOG * Fix api-client to call right PUT endpoint * Remove unneeded log output * Update tests, add granuleId parameter updates * Fix bad logic assertion * Update endpoints/granules :granuleName parameter to :granuleId * Fix lint 🔔 * Fix lint * Fix merged unit test * Fix merged comments --------- Co-authored-by: Marc <marc@element84.com> * CUMULUS-3053: removing references to dynamodb from docs (#3322) * CUMULUS-3053: removing references to dynamodb from docs * resolving lint errors * CUMULUS-3285: Updated isAuthBearTokenRequest to handle non-Bearer authorization header (merge to master) (#3350) * CUMULUS-3285: Updated isAuthBearTokenRequest to handle non-Bearer authorization header (#3341) * CUMULUS-3121/3120 (#3360) * CUMULUS-3121/3120 v15.1.0 backport (#3346) * backport PR * finalizing docs and changing remaining groups * PR feedback + testing changes * PR feedback * reverting change * Revert "update changelog" This reverts commit ae4627c. * reverting changes * PR feedback * removing EgressLambda from doc * Update xml2js 0.4.22->0.5 strict (#3330) (#3339) * Update xml2js 0.4.22->0.5 strict * Address GHSA-776f-qx25-q3cc/update allow list * Update CHANGELOG * Update package pins to 0.5.0 for xmljs Co-authored-by: Jonathan Kovarik <kovarik@nsidc.org> * PR feedback * PR feedback * PR feedback * fixing documentation linting * PR feedback * PR feedback * PR feedback * adding variables to tf-modules/workflow * PR feedback * PR feedback * reverting previous change --------- Co-authored-by: jennyhliu <34660846+jennyhliu@users.noreply.github.com> Co-authored-by: Jonathan Kovarik <kovarik@nsidc.org> * PR feedback (docs+changelog) * fixing docs * undoing commit * PR feedback --------- Co-authored-by: jennyhliu <34660846+jennyhliu@users.noreply.github.com> Co-authored-by: Jonathan Kovarik <kovarik@nsidc.org> * CUMULUS-3121/3120 (#3360) * CUMULUS-3121/3120 v15.1.0 backport (#3346) * backport PR * finalizing docs and changing remaining groups * PR feedback + testing changes * PR feedback * reverting change * Revert "update changelog" This reverts commit ae4627c. * reverting changes * PR feedback * removing EgressLambda from doc * Update xml2js 0.4.22->0.5 strict (#3330) (#3339) * Update xml2js 0.4.22->0.5 strict * Address GHSA-776f-qx25-q3cc/update allow list * Update CHANGELOG * Update package pins to 0.5.0 for xmljs Co-authored-by: Jonathan Kovarik <kovarik@nsidc.org> * PR feedback * PR feedback * PR feedback * fixing documentation linting * PR feedback * PR feedback * PR feedback * adding variables to tf-modules/workflow * PR feedback * PR feedback * reverting previous change --------- Co-authored-by: jennyhliu <34660846+jennyhliu@users.noreply.github.com> Co-authored-by: Jonathan Kovarik <kovarik@nsidc.org> * PR feedback (docs+changelog) * fixing docs * undoing commit * PR feedback --------- Co-authored-by: jennyhliu <34660846+jennyhliu@users.noreply.github.com> Co-authored-by: Jonathan Kovarik <kovarik@nsidc.org> * CUMULUS-3121/3120 (#3360) * CUMULUS-3121/3120 v15.1.0 backport (#3346) * backport PR * finalizing docs and changing remaining groups * PR feedback + testing changes * PR feedback * reverting change * Revert "update changelog" This reverts commit ae4627c. * reverting changes * PR feedback * removing EgressLambda from doc * Update xml2js 0.4.22->0.5 strict (#3330) (#3339) * Update xml2js 0.4.22->0.5 strict * Address GHSA-776f-qx25-q3cc/update allow list * Update CHANGELOG * Update package pins to 0.5.0 for xmljs Co-authored-by: Jonathan Kovarik <kovarik@nsidc.org> * PR feedback * PR feedback * PR feedback * fixing documentation linting * PR feedback * PR feedback * PR feedback * adding variables to tf-modules/workflow * PR feedback * PR feedback * reverting previous change --------- Co-authored-by: jennyhliu <34660846+jennyhliu@users.noreply.github.com> Co-authored-by: Jonathan Kovarik <kovarik@nsidc.org> * PR feedback (docs+changelog) * fixing docs * undoing commit * PR feedback --------- Co-authored-by: jennyhliu <34660846+jennyhliu@users.noreply.github.com> Co-authored-by: Jonathan Kovarik <kovarik@nsidc.org> * CUMULUS-3121/3120 (#3360) * CUMULUS-3121/3120 v15.1.0 backport (#3346) * backport PR * finalizing docs and changing remaining groups * PR feedback + testing changes * PR feedback * reverting change * Revert "update changelog" This reverts commit ae4627c. * reverting changes * PR feedback * removing EgressLambda from doc * Update xml2js 0.4.22->0.5 strict (#3330) (#3339) * Update xml2js 0.4.22->0.5 strict * Address GHSA-776f-qx25-q3cc/update allow list * Update CHANGELOG * Update package pins to 0.5.0 for xmljs Co-authored-by: Jonathan Kovarik <kovarik@nsidc.org> * PR feedback * PR feedback * PR feedback * fixing documentation linting * PR feedback * PR feedback * PR feedback * adding variables to tf-modules/workflow * PR feedback * PR feedback * reverting previous change --------- Co-authored-by: jennyhliu <34660846+jennyhliu@users.noreply.github.com> Co-authored-by: Jonathan Kovarik <kovarik@nsidc.org> * PR feedback (docs+changelog) * fixing docs * undoing commit * PR feedback --------- Co-authored-by: jennyhliu <34660846+jennyhliu@users.noreply.github.com> Co-authored-by: Jonathan Kovarik <kovarik@nsidc.org> * fixing CHANGELOG.md * removing docs + updating changelog * changing doc link --------- Co-authored-by: Nate Pauzenga <npauzenga@gmail.com> Co-authored-by: Jonathan Kovarik <kovarik@nsidc.org> Co-authored-by: Marc <marc@element84.com> Co-authored-by: nasamoduyebo <123400008+nasamoduyebo@users.noreply.github.com> Co-authored-by: jennyhliu <34660846+jennyhliu@users.noreply.github.com> * CUMULUS-3243:Updated granule delete logic (#3338) (#3366) (#3367) * CUMULUS-3243:Updated granule delete logic (#3338) * CUMULUS-3243:Updated granule delete logic to delete granule which is not in DynamoDB * add unit tests * delete files not in master * update CHANGELOG * remove dynamodb from test * Release 16.0.0 (#3376) * update docs * Version up to v16.0.0 * Minor edits/update release doc * Add PI release version notes * Minor CL edit * Update CHANGELOG.md Co-authored-by: Nate Pauzenga <npauzenga@gmail.com> * Update CHANGELOG.md Co-authored-by: Nate Pauzenga <npauzenga@gmail.com> * Update CHANGELOG.md Co-authored-by: Nate Pauzenga <npauzenga@gmail.com> * Address PR feedback --------- Co-authored-by: Nate Pauzenga <npauzenga@gmail.com> * [CUMULUS-3172] Update data integrity/migration docs (#3387) * update docs * update docs * add new doc to sidebars * Update docs/upgrade-notes/rds-phase-3-data-migration-guidance.md Co-authored-by: Jonathan Kovarik <kovarik@nsidc.org> * Update docs/upgrade-notes/rds-phase-3-data-migration-guidance.md Co-authored-by: Jonathan Kovarik <kovarik@nsidc.org> * Update docs/upgrade-notes/rds-phase-3-data-migration-guidance.md Co-authored-by: Jonathan Kovarik <kovarik@nsidc.org> * Update docs/upgrade-notes/rds-phase-3-data-migration-guidance.md Co-authored-by: Jonathan Kovarik <kovarik@nsidc.org> --------- Co-authored-by: Jonathan Kovarik <kovarik@nsidc.org> * Fix lint issues (#3392) * Jk/cumulus 3307 (#3394) * Backport CUMULUS-3307/PR #3386 * Fixup * Re-issue v16 docs (#3400) * CUMULUS-3223: Fix failed granule stuck in queued (#3373) (#3402) * CUMULUS-3223:Fix failed granule stuck in queued * skip sqsQueueExists test * update getGranuleTemporalInfo * update test match schema * update getGranuleTemporalInfo * remove extra await * remove skip sqsQueueExists * update `@cumulus/cumulus-message-adapter-js` to `2.0.5` * update sfEventSqsToDbRecords to return partial batch failure * fix typo * handle process error seperately, multiple message test * update test to process multiple messages * Jk/cumulus 3315 (#3407) * CUMULUS-3135 - Update integration test scripts to fail on test timeout (#3401) * Update integration test scripts to fail on test timeout * Fixup * Fixup * Update script interpreter for test runs * Fix script * Fixup * Fixup * Update timeout pass/fail conditional * Jk/cumulus 3135 fix integration tests (#3403) * Update integration test scripts to fail on test timeout * Fixup * Fixup * Update script interpreter for test runs * Fix script * Fixup * Fixup * Update api/client and integration test usage of it to fix test failures * Fix formatting/lint/etc * Update test/minor fix --------- Co-authored-by: etcart <37375117+etcart@users.noreply.github.com> --------- Co-authored-by: etcart <37375117+etcart@users.noreply.github.com> * v16.0.1 alpha release for testing (#3409) * bump to 16.0.1-alpha.0 * resolve conflicts --------- Co-authored-by: Jonathan Kovarik <kovarik@nsidc.org> * Update serve.js to match main * bump version * Add missing method for erasing PG tables (#3419) * add missing method for erasing PG tables * Remove duplicate declaration * Remove duplicate declaration * Take serve.js and serveUtils.js from main * Update schema endpoint to utilize P3 api/lib schema --------- Co-authored-by: Jonathan Kovarik <jonathan.kovarik@gmail.com> * Release 16.0.0 (#3428) * version bump * update changelog version * changelog fixup * changelog fixup * update cumulus versions in new lambda * bump cumulus versions --------- Co-authored-by: nasamoduyebo <123400008+nasamoduyebo@users.noreply.github.com> Co-authored-by: jennyhliu <34660846+jennyhliu@users.noreply.github.com> Co-authored-by: Jonathan Kovarik <kovarik@nsidc.org> Co-authored-by: Marc <marc@element84.com> Co-authored-by: Naga Nages <66387215+Nnaga1@users.noreply.github.com> Co-authored-by: etcart <37375117+etcart@users.noreply.github.com> Co-authored-by: Jonathan Kovarik <jonathan.kovarik@gmail.com>
4 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary: Summary of changes
Addresses CUMULUS-3135
This addresses a breaking change introduced in #3000 and related PRs due to changes in the underlying bulk API calls.
This PR can be merged with or after #3401
Changes
@cumulus/api-client/granules.bulkOperationto removeidsparameter in favor of
granulesparameter, in the form of a@cumulus/types/ApiGranulethat requires the following keys:[granuleId, collectionId]PR Checklist