Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add non-AWS configuration options #30

Merged
merged 18 commits into from Mar 17, 2020
Merged

Add non-AWS configuration options #30

merged 18 commits into from Mar 17, 2020

Conversation

fozboz
Copy link
Contributor

@fozboz fozboz commented Feb 14, 2020

What does this Pull Request do?

This PR adds additional configuration options in the AMA, so that one can configure non-AWS backend options.

When configuring Global Properties or Duracloud Mill, users now have the option to use:

  • RabbitMQ instead of SNS for notifications
  • RabbitMQ instead of SQS for the audit queue
  • S3+Swift as a storage option for audit logs

Users now have the option of setting "spring" for email notifications, in which case an SMTP server is used instead of SES.

We have also included an update for the SQL schema. The schema should be updated before attempting to run the compiled WAR, even if you are not using the non-AWS config options.

Finally, we include the required update to the POM so that it uses HTTPS for Maven.

How should this be tested?

This cannot be compiled without the changes that we made to the Duracloud repo, which will be in a forthcoming PR. Ideally, this and durastore/duradmin should be tested in tandem.

Pre-requisites for testing:

  • An SMTP server
  • A RabbitMQ server with:
    • rabbitmq_message_timestamp plugin installed
    • a fanout exchange
  1. Build the AMA WAR in the usual way.
  2. Ensure the database schema has been updated with update-non-aws.sql
  3. Edit notification section in config.properties file with:
notification.type=spring
notification.host=smtp.example.com
notification.port=25
notification.user='exampleuser'
notification.pass='examplepass'
notification.from-address=duracloud@example.com
notification.admin-address=admin@example.com
  1. Restart tomcat with new WAR
  2. Log in to Root Console and edit Duracloud Mill parameters with new options
  3. Edit Global Properties parameters with new options. Use the name of the fanout exchange as the exchange name
  4. Restart tomcat. You should see messages that we have successfully connected to the SMTP server, and RabbitMQ has been selected for notifications. You should also see that queues have been created in RabbitMQ and they have been bound to the fanout exchange.

Interested parties

@bbranan

Shibo Liu and others added 4 commits February 14, 2020 16:57
Add support for RabbitMQ and Spring email
4a183bc 
This commit contains all the changes necessary to enable us to configure
RabbitMQ or Spring email in the AMA.

Bumps all dependencies to 6.2.0-SNAPSHOT.
@adrwhong
Add Swift configuration options as an Audit Log Type
78a41de
Use HTTPS for Maven central
7bcb6a7 
Also updates organization from DuraSpace to LYRASIS and reference urls
to https
Add SQL schema update for non-AWS config
6f7fe22
@fozboz fozboz requested a review from bbranan February 14, 2020 22:53
Andy Foster added 2 commits February 18, 2020 14:20
Remove Swift config
12699ee 
We are going to store Swift config in the .properties file, so these
properties are no longer stored in the database, and therefore do not
need to be configured in the management console.
Remove Swift params from bean constructor
ed92a08 
Forms part of previous commit.
bbranan
bbranan requested changes Feb 27, 2020
View reviewed changes
resources/sql/update-non-aws.sql Outdated Show resolved Hide resolved
resources/sql/schema-with-swift.sql Outdated Show resolved Hide resolved
resources/sql/schema-with-rabbitmq.sql Outdated Show resolved Hide resolved
resources/sql/schema-3.1.6.sql Outdated Show resolved Hide resolved
resources/config-template/duracloud-config-non-aws.properties Outdated Show resolved Hide resolved
account-management-app/src/main/webapp/WEB-INF/applicationContext.xml Outdated Show resolved Hide resolved
account-management-app/src/main/webapp/WEB-INF/jspx/root/globalproperties/index.jspx Outdated Show resolved Hide resolved
account-management-app/src/main/webapp/WEB-INF/jspx/root/duracloudmill/index.jspx Outdated Show resolved Hide resolved
account-management-common/src/main/java/org/duracloud/account/config/McConfig.java Outdated Show resolved Hide resolved
...ement-util/src/main/java/org/duracloud/account/db/util/notification/NotificationMgrImpl.java Outdated Show resolved Hide resolved
Andy Foster added 6 commits March 2, 2020 13:27
Change SQL query to MODIFY
09e15fe 
Prevents repetition of column name.
Improve notification type log message
dc929c3
Remove commented bean definitions
17722ba
Permit empty values for McConfig bean parameters
36360d9 
Setting the `:` at the end of these variable allows them to be empty so
that IAM can set them at runtime.
Change imports from Spring to SMTP and set default to SES
707e059 
The modules in the Duracloud repo have been changed from
SpringNotificationFactory to SMTPNotificationFactory to more accurately
describe what they do, so we have changed the import here.

We have also ensured that the application will start without any changes
to the config of pre-existing installations, so email settings will
default to SES unless SMTP is explicitly set.
Use the spring:message model for all form fields
286beea 
We also remove some extraneous whitespace from the .jspx files.
Pull strings from common constants
a3791f0 
We now pull strings from a Constant whenever we do a string comparison.
Andy Foster and others added 4 commits March 10, 2020 12:17
Remove auditStoreType from tests
691474a 
This value is no longer necessary.
Switch constants for Enums and rename variables
59d7f17 
We now use Enums instead of constants to make it easier to see what the
valid values for each "type" are.

We have also changed any instance of auditQueueType to queueType, to
reduce the number of "types".

Any instance of "AWS" has been changed to the more specific AWS service
in question.
Change McConfig constructor to use emailer.type
3b5d9e1 
Forms part of previous commit.
@msarmie
Switch frequently used strings to constants in JavaScript
8ec745b
@dbernstein dbernstein self-requested a review March 13, 2020 19:48
dbernstein
dbernstein approved these changes Mar 13, 2020
View reviewed changes
Copy link
Member

@dbernstein dbernstein left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work.

@bbranan bbranan merged commit ac6ed7f into duracloud:develop Mar 17, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dbernstein dbernstein dbernstein approved these changes

@bbranan bbranan bbranan approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
5 participants
@fozboz @dbernstein @bbranan @adrwhong @msarmie