Skip to content

Allow deployment of multiple instances per server#1688

Merged
mxsasha merged 14 commits into
mainfrom
1669-routinator
Mar 21, 2025
Merged

Allow deployment of multiple instances per server#1688
mxsasha merged 14 commits into
mainfrom
1669-routinator

Conversation

@aequitas

@aequitas aequitas commented Mar 13, 2025

Copy link
Copy Markdown
Collaborator

This PR contains the following changes:

  • Allow running multiple instances on the same server
  • Expose the Routinator API for internal use
  • No longer require static IP configuration for container in production deployments notice: this requires a deviating deployment method once, see below
  • Improve development iteration speed by reducing initial healthcheck interval and using uv for Python installs
  • Reduce warnings during test/lint commands and make error more apparent
  • Make prometheus retention time configurable
  • Fix running make test-all in local development
  • Check routinator/monitoring profile configuration on deploy

Deployment notes:

A change in this PR will require one of the Docker networks to be recreated with a new subnet. So before deploying this version the entire environment should first be brought down using:

docker compose --project-name=internetnl-prod down

And made sure the internetnl-prod_public-internet Docker network is removed:

docker network rm internetnl-prod_public-internet

@aequitas aequitas force-pushed the 1669-routinator branch 16 times, most recently from 2897adf to 04c5245 Compare March 20, 2025 08:34
@aequitas aequitas force-pushed the 1669-routinator branch 5 times, most recently from dec0810 to c5eb77d Compare March 20, 2025 21:58

# set proxy_pass argument as variable, this makes sure nginx will still start even if this hostname is unresolvable due to routinator profile being disabled
set $routinator http://routinator:9556;
rewrite ^/routinator(.*)$ $1 break;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you test/tried using a trailing slash in the proxy_pass variable instead of a rewrite?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did and somehow I could not get it to work with any combination of slashes, might be how routinator is handling the request?

include http.headers;
include hsts_h3.headers;

# allow only local connections

@bwbroersma bwbroersma Mar 20, 2025

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This still requires some GB Routinator footprint per install .. maybe even optionally open this up even more? (e.g. make batch or dev use single-test Routinator)
I understand doing so creates new potential issues (version and connection wise), and probably it's best running at least one in dev, but batch might use single-test.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add this as a feature for a next iteration? I like to finish this PR as-is because it already has lots of big changes.

@aequitas aequitas force-pushed the 1669-routinator branch 4 times, most recently from 5bb7be2 to 58bb8f7 Compare March 21, 2025 09:00
@aequitas aequitas marked this pull request as ready for review March 21, 2025 09:01
- Move django_browser_reload into requirements.txt to remove complication of seperate image for development
- Don't require running statsd container for running tests
aequitas added 11 commits March 21, 2025 11:59
To allow multiple instances to be easily configured to run next to each
other on the same server we want to reduce the amount of static IP
configuration of containers.

- Move static files used for testing (hof, public_suffix_list) into
separate service to keep it distinct from test-target

- remove static IP's for almost all containers/services. Only a few
static IP's are kept, mostly for dev/test environments where these IP's
are referenced in the zone files for resolving by the mock-resolver.

- configure nginx to do DNS lookups for proxy_pass's to backend
containers for app and monitoring.

- no longer configure the internal resolver (resolver-validating) as
container resolver. Since the application configures a resolver for use
internally which explicitly uses the `resolver-validating` this
configuration is not needed, it also enables resolving containers by
name.

- configure the `app` resolver to lookup the `resolver-validating`
containers IP by container name.

- restart services that use `resolver-validating` when that services is
restarted (and potentially get's a different IP).
@mxsasha mxsasha merged commit 59c6492 into main Mar 21, 2025
@mxsasha mxsasha deleted the 1669-routinator branch March 21, 2025 12:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

3 participants