Skip to content

E2E testing and start tracking coverage - #403

Merged
NGTmeaty merged 22 commits into
mainfrom
e2e-testing
Jul 22, 2025
Merged

E2E testing and start tracking coverage#403
NGTmeaty merged 22 commits into
mainfrom
e2e-testing

Conversation

@yzqzss

@yzqzss yzqzss commented Jul 20, 2025

Copy link
Copy Markdown
Collaborator

issue: #162


The main idea of this PR is to use logs for E2E testing, this approach doesn't require mocking anything.

The method is to start up the main Zeno program during testing and let Zeno push logs to a socket. Then the e2e test suite connects to the socket to get the log stream and run customizable Match() and Assert() functions to check if each log record contains expected/unexpected content.

I didn't choose to read the existing ZENO-{time}.log files because "file is not stream". If we want to read logs in a streaming fashion, we'd need to implement behavior like tail -f, periodically checking the file size and seeking to that position to read new content, which is quite cumbersome. Additionally, Zeno's log files get rotated after some time, making parsing somewhat complicated.

So I added the --log-socket for e2e testing purposes.

  • To allow Go test features like -race and -coverage to cover Zeno program being tested, we can't execve or fork the Zeno's binary to start a new process. We have to call Zeno's entry function within the Test* functions.
  • Since go test compiles all Test* functions in the same test package into a single test binary and run it in the same process. We have to write each E2E test in different packages. Otherwise, global variable reuse would cause tests to fail.

Other changes:

  • Start tracking coverage with codecov.io
  • Make CI run faster

@yzqzss yzqzss changed the title E2E test WIP: E2E test Jul 20, 2025
@yzqzss

yzqzss commented Jul 20, 2025

Copy link
Copy Markdown
Collaborator Author

demo:

2025-07-21.04-14-22.mp4

@yzqzss yzqzss changed the title WIP: E2E test E2E testing Jul 20, 2025
@yzqzss
yzqzss requested a review from Copilot July 20, 2025 22:23

This comment was marked as outdated.

@yzqzss yzqzss changed the title E2E testing E2E testing and start tracking coverage Jul 21, 2025
@codecov-commenter

Copy link
Copy Markdown

Welcome to Codecov 🎉

Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests.

Thanks for integrating Codecov - We've got you covered ☂️

Comment thread .github/workflows/go.yml Outdated
Comment thread cmd/cmd.go
Comment thread e2e/e2e.go Outdated
Comment thread e2e/test/cloudflare204/cloudflare204_test.go Outdated
@yzqzss

yzqzss commented Jul 22, 2025

Copy link
Copy Markdown
Collaborator Author

ready for review:)

@yzqzss
yzqzss requested a review from Copilot July 22, 2025 01:11

This comment was marked as outdated.

yzqzss and others added 3 commits July 22, 2025 09:13
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@yzqzss
yzqzss requested a review from Copilot July 22, 2025 01:31

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR implements end-to-end (E2E) testing functionality and adds coverage tracking to the CI pipeline. The implementation introduces socket-based logging to enable real-time log monitoring during tests and adds comprehensive test infrastructure.

  • Adds Unix socket logging capability for E2E test communication
  • Implements E2E test framework with log parsing and command execution utilities
  • Integrates coverage tracking for both unit tests and E2E tests in CI workflow

Reviewed Changes

Copilot reviewed 15 out of 16 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
internal/pkg/log/log.go Adds socket cleanup logic in the Stop function
internal/pkg/log/config.go Implements Unix socket logging configuration and connection handling
internal/pkg/controler/signal.go Removes os.Exit(0) call to allow graceful shutdown
internal/pkg/config/config.go Adds socket logging configuration fields
go.mod Adds logfmt dependency for log parsing
e2e/test/nxdomain/nxdomain_test.go E2E test for DNS resolution failure scenarios
e2e/test/cloudflare204/cloudflare204_test.go E2E test for successful URL archiving
e2e/log/log.go Log parsing utilities and record matching interface
e2e/e2e.go Core E2E testing framework with command execution and log handling
cmd/cmd.go Refactors command preparation to support E2E testing
.github/workflows/go.yml Updates CI to run E2E tests and track coverage
Comments suppressed due to low confidence (3)

e2e/test/nxdomain/nxdomain_test.go:1

  • The package name 'boot' does not match the directory name 'nxdomain'. The package should be named 'nxdomain' to follow Go conventions.
package boot

e2e/test/cloudflare204/cloudflare204_test.go:1

  • The package name 'boot' does not match the directory name 'cloudflare204'. The package should be named 'cloudflare204' to follow Go conventions.
package boot

e2e/test/nxdomain/nxdomain_test.go:43

  • The function name 'TestCloudFlare204' doesn't match the test purpose. This test is for NXDOMAIN handling, so it should be named something like 'TestNXDomain'.
func TestCloudFlare204(t *testing.T) {

Comment thread internal/pkg/log/config.go Outdated
Comment thread internal/pkg/log/config.go Outdated
Comment thread internal/pkg/log/config.go
Comment thread e2e/log/log.go Outdated
Comment thread e2e/log/log.go Outdated
yzqzss and others added 3 commits July 22, 2025 09:33
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@yzqzss
yzqzss requested a review from NGTmeaty July 22, 2025 03:24
@yzqzss yzqzss added coverage github_actions Pull requests that update GitHub Actions code labels Jul 22, 2025
NGTmeaty
NGTmeaty previously approved these changes Jul 22, 2025

@NGTmeaty NGTmeaty left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initial review looks good! I'm going to take some time to run these locally as well before I merge, but great job so far!

Oh and a couple of small comments but nothing major!

Comment thread e2e/test/cloudflare204/cloudflare204_test.go
Comment thread e2e/test/nxdomain/nxdomain_test.go Outdated
Comment thread .github/workflows/go.yml Fixed
Comment thread .github/workflows/go.yml Fixed

@NGTmeaty NGTmeaty left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great job! This looks great and is a great start for additional end to end testing!

@NGTmeaty
NGTmeaty merged commit 62be009 into main Jul 22, 2025
5 checks passed
@yzqzss
yzqzss deleted the e2e-testing branch July 22, 2025 23:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

coverage github_actions Pull requests that update GitHub Actions code GSoC tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants