Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Start outlining the test environment #6

Open
hellais opened this issue Feb 23, 2024 · 1 comment
Open

Start outlining the test environment #6

hellais opened this issue Feb 23, 2024 · 1 comment

Comments

@hellais
Copy link
Member

hellais commented Feb 23, 2024

We should start outlining the structure of the test tier, so we can start migrating services over starting from the testing environment.

As part of this we should be refactoring the current resources and terraform definitions into separate modules (following how this has been done for example for clickhouse: #4)

@jbonisteel
Copy link

For Sprint starting March 11

  • Review the refactor PR
  • Add support for CI/CD
  • Take template + reproduce for Test stage

hellais added a commit that referenced this issue Mar 14, 2024
This PR is to refactor the various resource groups into modules to make
it easier to develop the test environment.

See: #6

Also includes changes implemented by @DecFox

feat: add OONI Dev User for GH actions (#17)

This adds a new IAM user: ooni_dev_user which shall be used as the
github user to run terraform apply. This IAM user has been given minimal
ReadOnly permissions to resources which allow it to successfully run
`terraform plan`

---------

Co-authored-by: DecFox <33030671+DecFox@users.noreply.github.com>
hellais added a commit that referenced this issue Mar 14, 2024
… Github User (#13)

Refactor the various resource groups into modules to make
it easier to develop the test environment.

See: #6

Convert to terraform the CodeBuild and CodePipeline projects that were
previously created through the web UI (fixes:
#12) and add support for an nginx
based API gateway (fixes: #7)

Also includes changes implemented by @DecFox

feat: add OONI Dev User for GH actions (#17)

This adds a new IAM user: ooni_dev_user which shall be used as the
github user to run terraform apply. This IAM user has been given minimal
ReadOnly permissions to resources which allow it to successfully run
`terraform plan`

Regarding CodeBuild and CodePipeline terraform conversion we made use the of the
terraformer tool running it like this:

```
terraformer import aws --resources=ec2,codebuild,codepipeline,iam --regions=eu-central-1
```

The output of this tool cannot be used as-is, but instead required a bit
of massaging to adapt it to the conventions used in the ooni terraform
scripts and hook it up to the existing variables, however it was very
helpful to bootstrap the config.

For the nginx API gateway, the actual configuration is still missing,
but the general idea is that we will have a Elastic Load Balancer rule
to forward requests to the nginx reverse proxy running in EC2 or the
dataapi running on ECS.

Note:

As part of this work I also did some refactoring of the names of some of
the resources and items inside of the terraform config. In doing so this
triggered a destruction and re-creation of all relevant resources, which
is I guess a nice lesson learned (I actually saw it from the plan, but
was curious to see what would happen).

In the future if we would like to do this kind of refactoring we should
be sure to make use of the `moved` syntax in terraform like this:
https://developer.hashicorp.com/terraform/language/modules/develop/refactoring#moved-block-syntax
or manually move the state by running `terraform mv`.

---------

Co-authored-by: DecFox <33030671+DecFox@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants