Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Terrarium - Providers]: Implementation of Storage Feature - 3 #72

Merged
merged 19 commits into from
Apr 30, 2024
Merged

[Terrarium - Providers]: Implementation of Storage Feature - 3 #72

merged 19 commits into from
Apr 30, 2024

Conversation

anjanikshree12
Copy link
Contributor

@anjanikshree12 anjanikshree12 commented Apr 28, 2024 •
edited
Loading

Follow up PR to #71

This PR includes:

  • Definition of APIs and Implementation of handler functions that interacts with grpc services to fetch the required files.
  • Unit tests added for helper functions.
  • Created a cmd for provider-storage-service
  • Update allInOne, gateway, providers-v1-rest & browse services.

Description:

  • The plan is to extend the rest protocol implementation of providers v1, to design 3 APIs that could be used to download the required files/provider binaries as per the provider registry protocol from the storage system.
https://<host-name>/providers/v1/{organization_name}/{name}/{version}/{os}/{arch}/terraform-provider-{name}_{version}_{os}_{arch}.zip
https://<host-name>/providers/v1/{organization_name}/{name}/{version}/terraform-provider-{name}_{version}_SHA256SUMS
https://<host-name>/providers/v1/{organization_name}/{name}/{version}/terraform-provider-{name}_{version}_SHA256SUMS.sig
  • The files needed as per the provider registry protocol for download through the URL is provider binary archive, shasum file & shasum signature file.
"download_url": "https://<host-name>/providers/v1/{organization_name}/{name}/{version}/{os}/{arch}/terraform-provider-{name}_{version}_{os}_{arch}.zip",
"shasums_url": "https://<host-name>/providers/v1/{organization_name}/{name}/{version}/terraform-provider-{name}_{version}_SHA256SUMS",
"shasums_signature_url": "https://<host-name>/providers/v1/{organization_name}/{name}/{version}/terraform-provider-{name}_{version}_SHA256SUMS.sig"

S3 Directory tree structure to be followed.

--organisation1
     |---- provider1
              |------ version
                         |----- terraform-provider-<provider-name>_<version>_SHA256SUMS
                         |----- terraform-provider-<provider-name>_<version>_SHA256SUMS.sig
                         |----- terraform-provider-<provider-name>_<version>_<os>_<arch>.zip
                         |----- terraform-provider-<provider-name>_<version>_<os>_<arch>.zip
                         continued ...
              |------ version2
                         |----- terraform-provider-<provider-name>_<version>_SHA256SUMS
                         |----- terraform-provider-<provider-name>_<version>_SHA256SUMS.sig
                         |----- terraform-provider-<provider-name>_<version>_<os>_<arch>.zip
                         |----- terraform-provider-<provider-name>_<version>_<os>_<arch>.zip
                         continued ...
     |---- provider2
              |------ version
                         |----- terraform-provider-<provider-name>_<version>_SHA256SUMS
                         |----- terraform-provider-<provider-name>_<version>_SHA256SUMS.sig
                         |----- terraform-provider-<provider-name>_<version>_<os>_<arch>.zip
                         |----- terraform-provider-<provider-name>_<version>_<os>_<arch>.zip
                         continued ...
              |------ version2
                         |----- terraform-provider-<provider-name>_<version>_SHA256SUMS
                         |----- terraform-provider-<provider-name>_<version>_SHA256SUMS.sig
                         |----- terraform-provider-<provider-name>_<version>_<os>_<arch>.zip
                         |----- terraform-provider-<provider-name>_<version>_<os>_<arch>.zip
                         continued ...

Note: Shasum file and signature is common for each version, so getting stored at version level.

@anjanikshree12 anjanikshree12 self-assigned this Apr 28, 2024
adcharre
adcharre reviewed Apr 29, 2024
View reviewed changes
Copy link
Contributor

@adcharre adcharre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only a few minor comments, looking good

internal/restapi/providers/v1/handler.go Outdated Show resolved Hide resolved
internal/restapi/providers/v1/handler.go Outdated Show resolved Hide resolved
internal/restapi/providers/v1/handler.go Outdated Show resolved Hide resolved
anjanikshree12 and others added 16 commits April 30, 2024 00:36
@anjanikshree12 @adcharre
Update internal/restapi/providers/v1/handler.go
2ecf809 
Co-authored-by: Adam Charrett <73886859+adcharre@users.noreply.github.com>
@anjanikshree12 @adcharre
Update internal/provider/services/storage/storage.go
2a7a675 
Co-authored-by: Adam Charrett <73886859+adcharre@users.noreply.github.com>
updated storage.go
1281216
enhance error logs
26b635f
updated storage_test
7501142
fix
71911b8
fix
ca2601f
err2->err renaming
65c52bc
Revert "err2->err renaming"
be0fc69 
This reverts commit 65c52bc.
err2->err renaming
1cab2c8
Revert "fix"
868924a 
This reverts commit ca2601f.
Revert "fix"
5a67a5f 
This reverts commit 71911b8.
Revert "updated storage_test"
c5f6fa0 
This reverts commit 7501142.
Revert "enhance error logs"
7b34376 
This reverts commit 26b635f.
Revert "updated storage.go"
f851a2e 
This reverts commit 1281216.
Revert "Update internal/provider/services/storage/storage.go"
511a1b0 
This reverts commit 2a7a675.
@anjanikshree12 anjanikshree12 merged commit 8ee56e1 into ciedev-3259-2 Apr 30, 2024
1 check passed
@anjanikshree12 anjanikshree12 deleted the ciedev-3259-3 branch April 30, 2024 15:29
anjanikshree12 added a commit that referenced this pull request Apr 30, 2024
@anjanikshree12 @adcharre
[Terrarium - Providers]: Implementation of Storage Feature - 2 (#71)
bb6b5b7 
* implementation of consumer service

* update in functions as we remove consumer service

* go fmt

* Update internal/provider/services/storage/storage.go

Co-authored-by: Adam Charrett <73886859+adcharre@users.noreply.github.com>

* updated storage.go

* enhance error logs

* updated storage_test

* fix

* fix

* updated storage_test and mock server

* [Terrarium - Providers]: Implementation of Storage Feature - 3 (#72)

---------

Co-authored-by: Anjani Kumar Srivastava <anjani@anjanivm.com>
Co-authored-by: Adam Charrett <73886859+adcharre@users.noreply.github.com>
anjanikshree12 added a commit that referenced this pull request Apr 30, 2024
@anjanikshree12 @adcharre
[Terrarium - Providers]: Implementation of Storage Feature - 1 (#70)
7c5a688 
* protobuff definitions for consumer service

* updated gen_proto.sh

* removed ConsumerService as not required

* format

* [Terrarium - Providers]: Implementation of Storage Feature - 2 (#71)

* implementation of consumer service

* update in functions as we remove consumer service

* go fmt

* Update internal/provider/services/storage/storage.go

Co-authored-by: Adam Charrett <73886859+adcharre@users.noreply.github.com>

* updated storage.go

* enhance error logs

* updated storage_test

* fix

* fix

* updated storage_test and mock server

* [Terrarium - Providers]: Implementation of Storage Feature - 3 (#72)

---------

Co-authored-by: Anjani Kumar Srivastava <anjani@anjanivm.com>
Co-authored-by: Adam Charrett <73886859+adcharre@users.noreply.github.com>

---------

Co-authored-by: Anjani Kumar Srivastava <anjani@anjanivm.com>
Co-authored-by: Adam Charrett <73886859+adcharre@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@adcharre adcharre adcharre approved these changes

@odelanno odelanno Awaiting requested review from odelanno

@ilijad1 ilijad1 Awaiting requested review from ilijad1

@dkPranav dkPranav Awaiting requested review from dkPranav

@vmilovanovicc vmilovanovicc Awaiting requested review from vmilovanovicc

Assignees

@anjanikshree12 anjanikshree12

Labels
terrarium-providers
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@anjanikshree12 @adcharre