e2e integration tests with docker and testcontainers

[0xERR0R]

I would like to introduce first step end-to-end integration tests. We do already have good basic unit test coverage, but I think we should also test more complex scenarios with real running system and external dependencies.

Basic idea:

• start blocky as docker container with defined configuration (as YAML)
• start also dependent services as docker containers in the same docker network, e.g.
  • database (mysql, postgres) for query log
  • redis
  • upstream DNS server(s)
  • http server for list downloads
  • ...
• do tests as black box (by using DNS queries (plain, DoH, DoT) or by calling REST/Cli endpoints)

I created a project DNS MOKKA which can be used as external upstream DNS server. This tool returns predefined answers (DNS responses, DNS errors or answers with delay), so it is easy to assert results.

This PR adds basic tests with all external services (mostly only happy path).

Tests can be execution with make e2e-test

GitHub action executes all e2e-tests on each push and PR

Opinions, ideas and suggestions are welcome 🤝

[codecov]

Codecov Report

Base: 92.82% // Head: 88.23% // Decreases project coverage by -4.59% ⚠️

Coverage data is based on head (87e9200) compared to base (d4813a6).
Patch coverage: 0.00% of modified lines in pull request are covered.

❗ Current head 87e9200 differs from pull request most recent head 4cd28db. Consider uploading reports for the commit 4cd28db to get more accurate results

Additional details and impacted files

@@               Coverage Diff               @@
##           development     #753      +/-   ##
===============================================
- Coverage        92.82%   88.23%   -4.60%     
===============================================
  Files               41       41              
  Lines             4644     4794     +150     
===============================================
- Hits              4311     4230      -81     
- Misses             264      494     +230     
- Partials            69       70       +1     

Impacted Files	Coverage Δ
e2e/containers.go	0.00% <0.00%> (ø)
redis/redis.go	66.32% <0.00%> (-25.95%)	⬇️
querylog/database_writer.go	82.17% <0.00%> (-0.18%)	⬇️
config/config.go	88.73% <0.00%> (ø)
querylog/file_writer.go	100.00% <0.00%> (ø)
querylog/logger_writer.go	100.00% <0.00%> (ø)
util/hostname.go

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

[kwitsch]

Looks very nice. 👍
Will review it during the week.

[kwitsch]

Looks consistent 👍