Major improvements to External Storage Support (BETA)

[ggallotti]

Improvements to GeneXus External Storage implementation.

(#249 and #283 and #314. All these PR are within this PR)

New

• Support For private storage
• New Providers support: Minio, any other S3 compatible SDK.
• Full Support for Storage Configuration through Environment Variables
• Support Encrypted or not encrypted sensitive properties (as Access Key, Secrey Key, bucket name, etc)
• Deleted IBM Bluemix deprecated provider
• Fixed some bugs detected In WebUnitTests
• Blobs, files and temporal files are now served as private resources

Improvements

• Added full UnitTesting to AWS S3, Azure, Minio as JUnit Tests
• Mayor refactor on how Storage Providers properties are read

New Property Name syntax

Provider Property Names were confusing, duplicate between providers so could not be set by ENV VARIABLES.
Now Properties follows the syntax: "STORAGE_%ProviderName%_%PropertyName%". (For compatibility reasons, old Properties name still supported).

In the future, we could support multiples providers as:
STORAGE_AWSS3_MyCustomProv_%PropertyName%

Examples (old property names)

• STORAGE_PROVIDER_SECRETACCESSKEY
• BUCKET_NAME
• FOLDER_NAME
• STORAGE_ENDPOINT

These properties are now:

• STORAGE_AWSS3_SECRET_ACCESSKEY
• STORAGE_AWSS3_BUCKET_NAME
• STORAGE_AWSS3_FOLDER_NAME
• STORAGE_AWSS3_ENDPOINT

How to run Unit Tests

** For security reasons, Secret Key has been removed from PR.

S3

SET AWSS3_TEST_ENABLED=true
SET STORAGE_AWSS3_ACCESS_KEY=AKIAJMQ6SF3Y4IULKD5A
SET STORAGE_AWSS3_SECRET_KEY=%REMOVED%
SET STORAGE_AWSS3_BUCKET_NAME=genexuss3test
SET STORAGE_AWSS3_FOLDER_NAME=gxclasses
SET STORAGE_AWSS3_REGION=us-east-1
mvn -pl gxexternalproviders -Dtest=TestExternalProviderS3 test

AZURE

SET AZUREBS_TEST_ENABLED=true
SET STORAGE_AZUREBS_ACCESS_KEY=%REMOVED%
SET STORAGE_AZUREBS_ACCOUNT_NAME=luistest1
SET STORAGE_AZUREBS_FOLDER_NAME=14_5_2021
SET STORAGE_AZUREBS_PUBLIC_CONTAINER_NAME=contluispublic
SET STORAGE_AZUREBS_PRIVATE_CONTAINER_NAME=contluisprivate
mvn -pl gxexternalproviders -Dtest=TestExternalProviderAzure test

MINIO

SET AWSS3_TEST_ENABLED=true
SET STORAGE_AWSS3_ACCESS_KEY=desaint
SET STORAGE_AWSS3_SECRET_KEY=%REMOVED%
SET STORAGE_AWSS3_BUCKET_NAME=java-classes-unittests
SET STORAGE_AWSS3_FOLDER_NAME=test-minio
SET STORAGE_AWSS3_ENDPOINT=custom
SET STORAGE_AWSS3_CUSTOM_ENDPOINT=http://192.168.254.78:9000
mvn -pl gxexternalproviders -Dtest=TestExternalProviderS3 test

Google Cloud Storage

SET STORAGE_GOOGLECS_KEY={ "type": "service_account", "project_id": "desarrollo-262414", "private_key_id": "d0c7d1ad72e5....}
SET STORAGE_GOOGLECS_PROJECT_ID=desarrollo-262414
SET STORAGE_GOOGLECS_BUCKET_NAME=desarrollo-262414-vcm
SET STORAGE_GOOGLECS_FOLDER_NAME=gxclasses
SET STORAGE_GOOGLECS_APPLICATION_NAME=desarrollo
mvn -pl gxexternalproviders -Dtest=TestExternalProviderGoogleCloudStorage test