Conversation
640c4f9 to
4b68438
Compare
|
It's on my list for next week to fix the resolving with |
|
Ah! It turned out to be importing the env file from the wrong repo (I have 6 copies now, from testing git clone). With One bit of feedback: Most of the test results are worse than they should be because the IPv6 routing doesn't work, making the code think that I have a misconfigured IPv6 site. This makes it a bit hard to debug changes to the scoring. It would be nice if we could have an option that just ignores IPv6 results :) |
|
@atjn I'm curious. How did you end up importing te env file from the wrong repo? Do you maybe have shell settings that source them automatically from a project directory? |
|
To be more precise, when opening the env file in VSCode to edit it, VSCode opened the file from the wrong repo, so I never actually edited the file that Docker was importing 😅 |
|
Hi @atjn, besides the point you mentioned above. Where you able to use the Docker environment to develop/test your changes or did you run into any more issues? |
|
@aequitas my other issues are mainly with understanding the code base, not anything specific to the docker image. I would like to have a fully configurable test website where I can quickly change values and see the outcome in a test. That does not seem to exist, or at least I can't find proper documentation for it. Other than that, I have not come much further, because the IPv6 issue is a pretty big blocker. When I can't find any detailed documentation for the inner workings of INL, I have to manually try things and see what the result is, but I can't do that because of the IPv6 issue :) |
|
I'm working on that this week, so I'll let you know if I have an update on that regard. |
What kind of system are you developing on? Linux, Mac? Depending on that I might already have a solution to get IPv6 working. |
|
@aequitas Linux debian/ubuntu |
|
@atjn in that case you may want to refer to this chapter of the development environment docs: https://github.com/internetstandards/Internet.nl/blob/docker/documentation/Docker-development-environment.md#ipv6-subnet-forwarding-linux-only I have not had the chance to test this extensively on desktop. So there might be some things missing, let me know if your run into issues. |
|
Just had a discussion with @baknu about HSTS, we should add a comment not to use add_header 'Strict-Transport-Security' 'max-age=31536000; includeSubdomains' always;Update: see #1010 (a3035fc) for the comment to add around this setting. |
|
For the relevant Apache setting (without It should be made explicit in the the documentation to not use this includeSubdomains HSTS directive.
|
|
Did a bunch more testing on the latest status and docs. Switched to Colima on my mac (0.5.5) and also tested on an Arch linux with Docker 24.0.2.
Redis issue log: RabbitMQ CPU usage: |
|
Perhaps related to my earlier redis host resolution issue, here's an integration test on the linux instance failing at least in part due to host resolution errors: https://gist.github.com/mxsasha/72cb79584dcceebb012c017374df838a On my colima mac instance, there are connection timeouts and the test never finished: https://gist.github.com/mxsasha/dd63303a7c02439d9fbb63a6234b4108 Command was |
|
Post-it reminders:
Some other notes:
For the 'known issues/improvements' ticket:
|
|
Built a setup using the docs on a fresh Ubuntu 22 box. I've fixed and improved a number of things:
Builds are failing, but that's our legacy build because zlib deleted their tarball because they released a new version. My unbound/web IPv6 address wasn't reachable from outside, as the host was not responding to ND requests. I fiddled with sysconf settings for a while, but it did not work. I ended up running ndppd with the following config, which fixed it. Not sure if this is the best approach or I missed something else: Also noticed logs like these in the host's journal: |
|
I am also noticing some flakiness in the integration test on CI, e.g.: https://github.com/internetstandards/Internet.nl/actions/runs/5938938158/job/16104534541?pr=890 |
This is a known problem which I'm still investigating, no solution in sight yet, but often restarting the CI run make it pass. |
Which sysconf settings did you try?
These queries should not have to be forwarded as they are resolved internally. Do you see these coming back all the time or only at startup? Might be a race condition. |
Not only at startup, but also not continuously: https://gist.github.com/mxsasha/b35ee26a9090f0e4c6b2a8eda4de678b |
|
@mxsasha what version of Docker are you running? Did you configure the |
Yes, those settings are set. Running |
|
What is our backup procedure when we run on Docker? Is this documented? |
|
@baknu there is not a proper procedure yet. But all stateful data is stored in |
[1.7.x] Ref #941 - Quick fix for sectxt translation mixup Manage external repositories through submodules and include other external sources in this repository Refactor settings to be configurable by environment variables Remove old docker implementation Add Dockerfile for application and docker-compose.yml for application stack. Add integration test suite Miscellaneous cleanup Github action for integration tests Improve logging during tests Allow local development in Docker Compose Add unbound resolver for connection test Add test target for domain and mail tests Httpd webserver for testing against current configuration, to be replaced with nginx config WIP: Split workers into multiple services WIP: Use nginx as webserver instead of apache2/httpd WIP: beat scheduler/batch? WIP: testsuite WIP: connection test Add integration tests that test against live running instances instead of controlled environment. WIP: temporary letsencrypt TLS certs TODO: autoreload fix? Use test target that complies with current main branch CSP header rules WIP: disable advanced checks until test environment is fully simulated Test targets healthcheck Make Unbound use Docker resolver during tests Adjust test scores wip Add public suffix list to integration test environment for mail test TODO: why is this resolved during tests Further isolate DNS in test environment to speed up testing Disable forwarding of all DNS requests not explicitly handled by dnsmasq so tests won't wait for resolving. This speeds up the test suite from minutes to seconds. Add target for testing all browsers Used fixed ip for mail test target Use .test tld for test targets Unbound logging wip wip Keep docker images locally so they don't have to be pulled in the next steps Enable batch API Documentation Add monitoring Statsd monitoring examples Settings file for locohost.nl Make integration tests code configurable Remove the need for dist settings Disable tracking on test/development Fix connection test IPv6 resolving Skip test we expect to fail to improve test suite speed Add dependencies for integration test Add live tests TODO: fix unbound for batch on staging Add security and robots .txt files Add favicon Split integration tests into multiple files Split live tests into files Cleanup testsuite Deployment documentation Make live/integration tests more similar Split live tests Enable linting on integration tests code Move integration tests docker resources to docker folder and naming consistency Deployment docs IPv6 dev env docs Fix batch by enabling scheduler which kicks of tasks Add container logging to integration tests Development docs Switch from Apache to Nginx webserver, add letsencrypt, https in integration test environment Reduce expected test score after rebase Don't let build fail if Github has issues with their caching system Split build from test and make concurrent Split image pins in separate file, build versioned images in github actions Make compose project names more distinct Make reboot proof Production settings Celery exporter Metrics documentation Test connection test redirect, don't redirect for trailing slashes, redirect for subdomains as well Fix race condition for certbot Non conflicting address for staing unbound Add worker healthcheck Docs Run integration tests without DEBUG, serve static files from Django in production, add static files caching and expires. Solve redis connection leak issue, deprecate 3.7 build. Add short commands Move assets files out of python source directories, to improve docker build cache Small improvements in docker image build speed 404's should return proper status code, because otherwise they might be cached Work around precondition failed issue with celery workers and rabbitmq Use correct path for development volume mounts Fix grafana authenticaton nagging from websockets Disable includeSubdomains on HSTS header, add docs. Use Zone file notation for DNS example Use Dockerfile as extension Lower default healthcheck interval on production Disable grafana connecting out Docs Improve stability Add nowww class b complicance Build images for monitoring so Git repo should not be needed for deployment Silence warning Remove unused Make targets Version number Matomo settings description IP allow/deny lists and site wide basic auth Container troubleshooting logging Setuptools_scm version as tag Serve connection test result over https Fix nginx config issues WIP: sign connection test zones WIP: add test- commands WIP: rework deployment docs without Git, Make and with env file split Fix glue records for unbound zone Docs WIP: dev env volumes Update test domains/email Troubleshooting docs Cleanup typo fix Signed-off-by: Sasha Romijn <github@mxsasha.eu> Fix zones path for docker volume Docs Reduce port exposure Firewall docs Normalize env files Document compose spec Switch from gunicorn to uwsgi Fix unbound config file interpolation after restart Allow debug logging in test, even if DEBUG=False Debugging docs Devenv: autoreload and mounted sources as default Store compose logs as artifacts for CI debugging Integration test environment docs Add test summary to CI Disable trace logs by default Add Docker linting and unittests, split workflows Split docker and old CI Add unit tests Make CI test run more reliable Remove need to generate secret key during deployment Cleanup Builtin unbound verbose logging Allow shorthand 'env' for 'environment' in make commands Pull all images to keep test run output clean Consistency Improve caching Fix indenting Document live testing, simplify live testing setup, add batch test Docs Add option for worker concurrency Standalone live tests Make celery exporter more reliable Docs, env diff table, make dev env explicit Move env files into docker directory Fix unit test environment file Improve when tasks are shown Use layered approach for env files Make rpki test more reliable when results are incomplete, document issues with test isolation. Batch docs Use static IP's for internal connections instead of relying on Docker's flaky internal resolver Move logs capture lower to capture logs for all started services Remove duplicate config Make Hof seperately configurable Fix race condition with internal resolver to make CI tests more reliable Add test for development environment, split static files tests into existing and generated files, fix static files for develop environment. Pytest colored output in Github Suppres warnings in CI Adjust test score Add test report for live and develop tests Add IPv6 documentation for macOS All services should restart after 'reboot' Add unit tests to summary as well Default test-runner to live tests Batch docs Ignore test result parsing failures Log failures of up command Fix docs Add test-all command Cleanup host env file and improve batch docs Build all images Compare technical results for batch tests before asserting score Remove outdated file Run integration tests in ci verbose Fix IPv6 docs Make integration tests use unique id for each test to make correlating tests with logs easier Make startup more reliable using start_period instead of many retries Workaround for Docker not allowing intercontainer IPv6 traffic on 'internal' networks, will be fixed in Docker 25, see: moby/moby#45649 Fix allowed hosts after slimming down host-dist.env Make issues with existing environment variables during deploy less likely Make unbound verbose logging configurable Test environment context Revert "Pytest colored output in Github" This reverts commit 43f3dbd. WIP: Add second integration test with ip6tables enabled Make rabbitmq healthcheck and startup more reliable Remove outdated config file Routinator basic healthcheck Fix logging issues when bringing ci environment up Update deployment docs Disable routinator to pull in data during test and develop Add website test to develop Test development environment in CI using ipv6 Fix host-dist.env file Improve instructions for DNSSEC reset Expose DNS port on IPv6 Update live-tests docs for ipv6 Production and batch overview images Move test-runner into integration-test Architecture docs Remove invalid TLSA from zone files Add running message Fix documentation for IPv6 on macOs, add HE tunnel mention, simplify Linux instructions Disable waiting for healthy routinator as it takes to long of a time for routinator to start. Remove outdated file Allow configuring encrypted passwords Document different test suites Fix subdomain redirects in nginx (quick fix pending discussion) Signed-off-by: Sasha Romijn <github@mxsasha.eu> Fix nginx redirects, attempt 2 This reverts commit b959a54. Fix conn domain letsencrypt Add $IPV6_IP_PUBLIC to ALLOWED_HOSTS in host-dist.env Add note about IPv6 addresses Signed-off-by: Sasha Romijn <github@mxsasha.eu> Improve resilience of certbot entrypoint Documentation formatting fix Signed-off-by: Sasha Romijn <github@mxsasha.eu> Update integration test security.txt expectation ref #932 Add https scheme to CSP policy to meet #810 Set our Referrer-Policy to same-origin to meet #357 Make sure live tests examples always use latests Remove old config file Reduce complication in IPv6 setup for deployments - Remove the need to assign and forward IPv6 subnets to the internal docker networks - bind unbound ports explicitly to external addresses to not conflict with os resolvers on localhost - update deployment documentation Test conn. redirects Revert IP in allowed hosts, add testcase Fix develop environment ipv6 test Use internal resolver for all queries made by app/worker Rename test-resolver to mock-resolver for clarity Work around issue with resolving race condition in CI Fix documentation for nginx auth config Simplify compose commands in docs Pull test runner for develop test There is no public IPv6 network on Github actions, so these jobs make no sense. Enable DNSSEC validation for internal resolver Reduce maximum test duration and retest timeout on development Fix linting, fix linting errors Keep celery exporter running after crash Resign zones every time, but persist the keys, so we don't have to update DNS after IP/hostname changes. Update docs Split internal resolver in validating and permissive to both accomodate ldns-dane which requires validation and other queries which needs to be passed through to the client despite failing DNSSEC validation Add live test for sites with known score Don't search integration test directory during 'normal' tests Add Gixy check for nginx configs Nginx log vhost name, use chrome for playwright trace Issue 894, disable hsts on https://conn. domain and return 404 not found for all non-configured domains Use --project-name instead of --env-file for everything but up/pull/run Add node exporter Fix initial configuration for batch Rewrite basic auth, require authentication when DEBUG is enabled Integrate certbot with webserver service, auto reload nginx on certificate changes Test-all Set max tasks per child, to reduce memory issues on ipv6kaartmetingen.
5 participants
Getting started documentation on setting up a developer environment and integration tests: https://github.com/internetstandards/Internet.nl/blob/docker/documentation/Docker-getting-started.md
Other documents can be found at: https://github.com/internetstandards/Internet.nl/blob/docker/documentation/Docker.md
Collective todo:
Edit: to 'fix' the hundreds hidden items, Load more… a poor man's solution is this js: