Add DSpace submit functionality

[hakbailey]

Why these changes are being introduced:

The core functionality of this app is to create new items with associated bitstreams in DSpace based on the messages retrieved from an input SQS queue. This commit adds the functionality to do so, along with some necessary functionality around configuration and secrets handling.

How this addresses that need:

• Adds a submission module with a Submission class that can be instantiated from an SQS message, with methods to facilitate creating a DSpace item with bitstreams and generating both success and error result messages
• Updates the sqs module process function to use the new submission module
• Adds an smm module with an SSM class that handles retrieving secret from AWS Systems Manager Parameter Store
• Adds a config module that configures the application based on env (set by the WORKSPACE env variable)
• Adds and updates relevant tests, including adding a conftest.py and fixtures
• Updates Dockerfile and setup.py to reflect new dependencies

Additional changes in this commit:

• Adds logging throughout the app with logging configuration set by the root logger
• Adds pytest-cov and updates make test command to create a coverage report
• Updates .gitignore

Side effects of this change:

• For the app to work in staging and production environments, the WORKSPACE and SSM_PATH env variables must be set and the application must have an IAM role with access to SSM. This has been handled in a separate ticket, just noting it here (also added this info to the README).

Includes new or updated dependencies?

YES

Changes expectations for external applications?

NO

Developer

• All new ENV is documented in README
• All new ENV has been added to staging and production environments (technically just staging but there's a ticket for this)
• All related Jira tickets are linked in commit message(s)

How can a reviewer manually see the effects of these changes?

It's a process, but:

1. Run a local moto server, create an input queue and an output queue, and set env variable DSS_INPUT_QUEUE to whatever the input queue name is.
2. Send a message to the input queue that follows our input message spec. Look at the input_message_good fixture in conftest.py for an example (but note you'll need to change things like the collection handle and the output queue name).
3. Make sure you're on the VPN and set the following env variables:
   DSPACE_API_URL=<dspace test instance API url>
DSPACE_USER=<a user with write access to a DSpace collection>
DSPACE_PASSWORD=<that user's password>
4. You will need to change the sqs calls in the sqs.py retrieve and write functions to sqs = boto3.resource("sqs", endpoint="<whatever your localhost moto server endpoint is>") so the app will read and write to the local SQS queues and not real AWS.
5. With all that done, you should be able to run pipenv run submitter start and watch the app process the message, post the item to DSpace, and send the result message to the output queue.

Code Reviewer

• The commit message is clear and follows our guidelines
  (not just this pull request message)
• There are appropriate tests covering any new functionality
• The documentation has been updated or is unnecessary
• The changes have been verified
• New dependencies are appropriate or there were no changes

[JPrevost]

Oh my... lots of stuff going on here :)

Firstly, nothing blocking feel free to merge whenever. A few questions and comments and requests for Jira tickets along the way. I didn't run this locally as I felt we'd get enough opportunities to confirm it actually works as we run through tests on staging (and the local setup is a pain until I land my secret branch I never bothered to finish because reasons).

[hakbailey]

I didn't run this locally as I felt we'd get enough opportunities to confirm it actually works as we run through tests on staging (and the local setup is a pain until I land my secret branch I never bothered to finish because reasons).

I did. So at least there's that ¯_(ツ)_/¯