Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LF-3079 Replace Digital Ocean Spaces with MinIO for the development environment #2515

Conversation

kathyavini
Copy link
Collaborator

@kathyavini kathyavini commented Mar 24, 2023

Description

On beta & production, document upload and certification documents export both use a paid Digital Ocean Spaces service for storage. Developers currently can’t test these services locally without significant refactors to the user flow or using their own DIgital Oceans Spaces / AWS S3 accounts.

MinIO is a free and open-source drop-in replacement for AWS S3 that can be set up and run locally completely free of charge. Adding MinIO compatibility to the developer environment allows developers to locally test certification export and document upload and download in their local environments.

Instructions for using MinIO with the export server can be found on Confluence here: Running the export server locally

These code changes were made specifically for use with the guide above, and remove hardcoded references to the Digital Ocean Spaces bucket in the 1) certification document export, 2) download of certification documents, and 3) document upload & download.

A NOTE ABOUT IMAGES: Image upload and download are handled by the LiteFarm Imaginary which I believe is a Digital Ocean Droplet and not a Digital Ocean Space. Image upload and download don't currently work locally, and this PR will not fix that. That feature would be a great addition for a developer to pick up in the future!

Jira link: https://lite-farm.atlassian.net/jira/software/c/projects/LF/issues/LF-3079

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • This change requires a documentation update -- new variables have been documented in .env.default

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

  • Other (please explain)

This branch has been used by myself, @SayakaOno and @Duncan-Brain to test the SurveyStack ticket LF-2990.

Checklist:

  • I have commented my code, particularly in hard-to-understand areas
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • The precommit and linting ran successfully
  • I have added or updated language tags for text that's part of the UI
  • I have added "MISSING" for all new language tags to languages I don't speak
  • I have added the GNU General Public License to all new files

Use aws configure to set the access key and secret access key, and make sure that region is removed from the aws config
@Duncan-Brain
Copy link
Collaborator

@kathyavini

Thanks for this one Joyce and @arcijsg for suggesting MinIO.

Definitely already used this a few times For surveystack and document export reviews and has worked well!

  1. If there is any way to condense the Confluence document into a small ReadMe section, it could be good.

  2. You mentioned it doesn't handle the images: could you write up a Jira ticket for that one to incorporate MinIO as well?

Copy link
Collaborator

@SayakaOno SayakaOno left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for working on this Joyce!
I left a comment :)

Comment on lines 27 to 28
url.hostname = 'images.litefarm.workers.dev';
const url = new URL(fileUrl);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These two lines should be swapped, right...?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh goodness 🤦‍♀️ Thank you

@SayakaOno
Copy link
Collaborator

In case you haven't seen it, you have a conflict in packages/api/.env.default 😉

@kathyavini
Copy link
Collaborator Author

kathyavini commented Mar 27, 2023

@Duncan-Brain could you check the new README.md addition?

And here's a Jira ticket for localizing the Imaginary: LF-3081. Good idea.

Copy link
Collaborator

@Duncan-Brain Duncan-Brain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kathyavini

Looks great!

@SayakaOno SayakaOno merged commit b8d4e3f into integration Mar 28, 2023
2 checks passed
@iperdomo iperdomo deleted the LF-3079-Add-MinIO-integration-to-development-environment-to-allow-localhost-document-export-and-download branch June 28, 2023 11:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants