Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enable etcd-druid to make use of Azurite - the Azure Blob Storage Emulator #753

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

Enable etcd-druid to make use of Azurite - the Azure Blob Storage Emulator #753

wants to merge 4 commits into from

Conversation

renormalize
Copy link
Member

@renormalize renormalize commented Jan 10, 2024 •
edited

How to categorize this PR?

/area cost
/area dev-productivity
/area testing
/area usability
/kind enhancement

What this PR does / why we need it:
This PR adds support to etcd-druid to make use of Azurite the Azure Blob Storage emulator, which can be used to run end-to-end tests with Azure as the storage provider locally.

As a consequence of the above, developers can now spin up a kind cluster and run Azurite in this cluster - which enables developers to rapidly test changes made to etcd-druid or any of its dependencies which target Azure Blob Storage, either directly or indirectly, without the headache of managing Azure infrastructure.

Finally, cost to develop for Azure will reduce.

To make use of Azurite to run end-to-end tests with Azure Blob Storage, simply run:

make ci-e2e-kind-azure

To deploy Azurite in a kind cluster, run:

make kind-up
make deploy-azurite

Azure CLI az can be made use of to communicate with Azurite running in kind by exporting the relevant environment variable AZURE_STORAGE_CONNECTION_STRING emulating actual Azure infrastructure in all capabilities.

More information on how to develop for etcd-druid making use of Azurite can be found in docs/development/getting-started-locally-azurite.md!

Which issue(s) this PR fixes:
Fixes #740

Special notes for your reviewer:

This pull request is dependent on changes proposed in etcd-backup-restore #699.

Release note:

`etcd-druid` now supports end-to-end testing with `Azurite` - the Azure Blob Storage Emulator

@renormalize renormalize added area/cost Cost related area/dev-productivity Developer productivity related (how to improve development) kind/enhancement Enhancement, improvement, extension platform/azure Microsoft Azure platform/infrastructure area/testing Testing related labels Jan 10, 2024
@renormalize renormalize added this to the v0.23.0 milestone Jan 10, 2024
@renormalize renormalize requested a review from a team as a code owner January 10, 2024 13:51
@gardener-robot gardener-robot added needs/review Needs review area/usability Usability related labels Jan 10, 2024
@gardener-robot-ci-2 gardener-robot-ci-2 added the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Jan 10, 2024
@gardener-robot gardener-robot added size/l Size of pull request is large (see gardener-robot robot/bots/size.py) needs/second-opinion Needs second review by someone else labels Jan 10, 2024
@gardener-robot-ci-2 gardener-robot-ci-2 added needs/ok-to-test Needs approval for testing (check PR in detail before setting this label because PR is run on CI/CD) and removed reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) labels Jan 10, 2024
@renormalize
Copy link
Member Author

Thanks to @seshachalam-yv for his work on localstack in #713 for end-to-end testing which was the inspiration and baseline for this PR.

@gardener-robot-ci-1 gardener-robot-ci-1 added the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Jan 10, 2024
@gardener-robot-ci-2 gardener-robot-ci-2 removed the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Jan 10, 2024
@renormalize
Copy link
Member Author

/retest-required

@renormalize
Added support to etcd-druid to make use of Azurite - the Azure Bl…
edb1c49 
…ob Storage Emulator

* New documentation `docs/development/getting-started-locally-azurite.md`, to document
  the setup process for running `etcd-druid` with `Azurite`.

* New make target `deploy-azurite` to setup the `Azurite` emulator in the `kind` cluster.

* New make target `ci-e2e-kind-azurite` to run the e2e tests with `Azurite`.

* New script `hack/deploy-azurite.sh`, to setup the `Azurite` emulator in the `kind` cluster.

* New script `ci-e2e-kind-azurite`, to run the e2e tests with `Azurite`.

* New manifest `config/samples/druid_v1alpha1_etcd_azurite.yaml`, to run with `Azurite`.

* New manifest `config/samples/etcd-secret-azurite.yaml`, to run with with `Azurite`.

* New manifest `hack/e2e-test/infrastructure/azurite/azurite.yaml`, to setup `Azurite`.

* Port `10000` forward `hack/e2e-test/infrastructure/kind/cluster.yaml`, to setup `Azurite`.

* New environment variable `AZURE_STORAGE_API_ENDPOINT` to override ABS endpoint to `Azurite`.

* Environment variable `AZURE_STORAGE_CONNECTION_STRING` added to
  `hack/e2e-test/infrastructure/overlays/azure/common/azure.env` to enable the script which
  creates the ABS Container in `Azurite` to access `Azurite` using the relevant Storage Account,
  Storage Key, and the BlobEndpoint at which `Azurite`'s Blob Storage is hosted.

* Modified `hack/e2e-test/infrastructure/overlays/azure/common/files/common.sh` to create
  the ABS Container in `Azurite` when the relevant environment variable is present.

* Modified `pkg/utils/envvar.go` to read the Azure infrastructure `Secret` for the fields
  `enableAzurite` and `storageAPIEndpoint`, which are passed to the `etcd-backup-restore`
  container as environment variables, which are used to make use of the `Azurite` emulator.

* Modified `pkg/common/constants.go` to add `AZURE_ENABLE_STORAGE_EMULATOR` and
  `AZURE_STORAGE_API_ENDPOINT`, which are the enviroment variables passed to
  `etcd-backup-restore` to make use of the `Azurite` emulator.

* Modified `test/e2e/utils.go` to check for the `AZURITE_HOST` environment variable
  which creates the Azure `Secret`, to include additional fields `enableAzurite` and
  `storageAPIEndpoint` to enable usage of `Azurite`.
@renormalize
Changes to reflect additional changes proposed in etcd-backup-restore.
573fcbf 
* `make` target `ci-e2e-azurite` renamed to `ci-e2e-azure` since this
  target runs the end-to-end tests for Azure, using Azurite.

* The environment variable which is checked for the presence of the emulator
  `AZURE_ENABLE_STORAGE_EMULATOR` changed to `EMULATOR_ENABLED`.

* The field in the Secret for Azure which signifies that the Azurite
  emulator is being used is changed from `enableAzurite` to `emulatorEnabled`.
  All references to this are changed accordingly.
@gardener-robot-ci-1 gardener-robot-ci-1 added reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) and removed reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) labels Feb 7, 2024
@seshachalam-yv seshachalam-yv self-assigned this Feb 7, 2024
@renormalize
Copy link
Member Author

The output of end-to-end tests for etcd-druid that are run using Azurite as the object store; with changes made to etcd-backup-restore in 699 included in the vendor/ directory for etcd-backup-restore is at this gist.

The tests run successfully.

@renormalize
Fixed the comment which explains EnvEmulatorEnabled
6e3c880 
* Fixed the comment which explains `EnvEmulatorEnabled`.

* Better markdown formatting and practices in `getting-started-locally-azurite.md`.

* Removed an unnecessary comment in `etcd-secret-azurite.yaml`.
@gardener-robot-ci-2 gardener-robot-ci-2 added reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) and removed reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) labels Feb 9, 2024
seshachalam-yv
seshachalam-yv reviewed Feb 9, 2024
View reviewed changes
Copy link
Contributor

@seshachalam-yv seshachalam-yv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, @renormalize, for the excellent PR ❤️.

Please address my minor comments. Apart from that, the overall PR looks good to me.

hack/deploy-azurite.sh Outdated Show resolved Hide resolved
@renormalize
Simplify hack/ci-e2e-kind-azure.sh
ed3fa51 
* Simplify `hack/ci-e2e-kind-azure.sh` by defining variables which can be
  used to avoid repetition in `hack/ci-e2e-kind-azure.sh`

* Correct the License year in `hack/deploy-azurite.sh`
@gardener-robot-ci-1 gardener-robot-ci-1 added reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) and removed reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) labels Feb 9, 2024
@renormalize
Copy link
Member Author

renormalize commented Feb 9, 2024 •
edited

ci-e2e-kind-azure-output.txt

Output that can be viewable online at this gist

The e2e tests run fine with the changes requested.

seshachalam-yv
seshachalam-yv approved these changes Feb 9, 2024
View reviewed changes
Copy link
Contributor

@seshachalam-yv seshachalam-yv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/hold until this PR gardener/etcd-backup-restore#699 merge

@gardener-robot gardener-robot added the reviewed/do-not-merge Has no approval for merging as it may break things, be of poor quality or have (ext.) dependencies label Feb 9, 2024
@seshachalam-yv seshachalam-yv removed their assignment Feb 9, 2024
@renormalize
Copy link
Member Author

The dependency on etcd-backup-restore has to be upgraded to at least v0.29.0 (which would include gardener/etcd-backup-restore#699) which is yet to be released at the time of writing; before merging this PR.

If #748 is not merged before this PR, then the vendor directory must also be updated to the latest release of etcd-backup-restore.

@unmarshall
Copy link
Contributor

/hold util #777 is merged

@shreyas-s-rao shreyas-s-rao modified the milestones: v0.23.0, v0.24.0 Jun 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@seshachalam-yv seshachalam-yv seshachalam-yv approved these changes

Assignees

@renormalize renormalize

Labels
area/cost Cost related area/dev-productivity Developer productivity related (how to improve development) area/testing Testing related area/usability Usability related kind/enhancement Enhancement, improvement, extension needs/ok-to-test Needs approval for testing (check PR in detail before setting this label because PR is run on CI/CD) needs/review Needs review needs/second-opinion Needs second review by someone else platform/azure Microsoft Azure platform/infrastructure reviewed/do-not-merge Has no approval for merging as it may break things, be of poor quality or have (ext.) dependencies size/l Size of pull request is large (see gardener-robot robot/bots/size.py)
Projects
None yet
Milestone
v0.24.0
Development

Successfully merging this pull request may close these issues.

[Feature] Enable etcd-druid to make use of the Azurite emulator to run end-to-end (e2e) tests
7 participants
@renormalize @unmarshall @seshachalam-yv @gardener-robot-ci-1 @shreyas-s-rao @gardener-robot-ci-2 @gardener-robot