Skip to content

v1.10.0
Compare
Choose a tag to compare
@jennyhliu jennyhliu released this 31 Aug 15:21
· 12711 commits to master since this release
v1.10.0
4d0dc93

Removed

  • Removed unused and broken code from @cumulus/common
    • Removed @cumulus/common/test-helpers
    • Removed @cumulus/common/task
    • Removed @cumulus/common/message-source
    • Removed the getPossiblyRemote function from @cumulus/common/aws
    • Removed the startPromisedSfnExecution function from @cumulus/common/aws
    • Removed the getCurrentSfnTask function from @cumulus/common/aws

Changed

  • CUMULUS-839 - In @cumulus/sync-granule, 'collection' is now an optional config parameter

Fixed

  • CUMULUS-859 Moved duplicate code in @cumulus/move-granules and @cumulus/post-to-cmr to @cumulus/ingest. Fixed imports making assumptions about directory structure.
  • @cumulus/ingest/consumer correctly limits the number of messages being received and processed from SQS. Details:
    • Background: @cumulus/api includes a lambda <stack-name>-sqs2sf which processes messages from the <stack-name>-startSF SQS queue every minute. The sqs2sf lambda uses @cumulus/ingest/consumer to receive and process messages from SQS.
    • Bug: More than messageLimit number of messages were being consumed and processed from the <stack-name>-startSF SQS queue. Many step functions were being triggered simultaneously by the lambda <stack-name>-sqs2sf (which consumes every minute from the startSF queue) and resulting in step function failure with the error: An error occurred (ThrottlingException) when calling the GetExecutionHistory.
    • Fix: @cumulus/ingest/consumer#processMessages now processes messages until timeLimit has passed OR once it receives up to messageLimit messages. sqs2sf is deployed with a default messageLimit of 10.
    • IMPORTANT NOTE: consumer will actually process up to messageLimit * 2 - 1 messages. This is because sometimes receiveSQSMessages will return less than messageLimit messages and thus the consumer will continue to make calls to receiveSQSMessages. For example, given a messageLimit of 10 and subsequent calls to receiveSQSMessages returns up to 9 messages, the loop will continue and a final call could return up to 10 messages.
Assets 2
Loading