Skip to content

Improve Archiver error reporting#482

Merged
vbanos merged 1 commit intointernetarchive:mainfrom
vbanos:warc-panic
Sep 21, 2025
Merged

Improve Archiver error reporting#482
vbanos merged 1 commit intointernetarchive:mainfrom
vbanos:warc-panic

Conversation

@vbanos
Copy link
Copy Markdown
Collaborator

@vbanos vbanos commented Sep 19, 2025

Instead of crashing startWARCWriter via os.Exit(1), we return the error which is passed via archiver.Start in the pipeline. There, the program is terminated via panic as it happens with all component errors.

Relevant to: #473

Instead of crashing `startWARCWriter` via `os.Exit(1)`, we return the error
which is passed via `archiver.Start` in the pipeline.
There, the program is terminated via `panic` as it happens with all
component errors.
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Sep 19, 2025

Codecov Report

❌ Patch coverage is 30.76923% with 9 lines in your changes missing coverage. Please review.
✅ Project coverage is 56.41%. Comparing base (0837f4d) to head (7e2abd8).
⚠️ Report is 63 commits behind head on main.

Files with missing lines Patch % Lines
internal/pkg/archiver/worker.go 22.22% 5 Missing and 2 partials ⚠️
internal/pkg/archiver/warc.go 50.00% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #482      +/-   ##
==========================================
- Coverage   56.42%   56.41%   -0.01%     
==========================================
  Files         130      130              
  Lines        8091     8099       +8     
==========================================
+ Hits         4565     4569       +4     
- Misses       3161     3162       +1     
- Partials      365      368       +3     
Flag Coverage Δ
e2etests 40.71% <30.76%> (+0.05%) ⬆️
unittests 29.30% <0.00%> (-0.08%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Copy Markdown
Collaborator

@NGTmeaty NGTmeaty left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good besides one minor comment!


// Setup WARC writing HTTP clients
startWARCWriter()
if err := startWARCWriter(); err != nil {
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it make more sense to define this as WARCWriterErr or something similar?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now, we return the err returned by warc.NewWARCWritingHTTPClient.

How can we change the type of that to WARCWriterErr ?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was just talking about the variable name itself not the type, but on second glance I think it's fine as is.

@vbanos vbanos merged commit e8fcb38 into internetarchive:main Sep 21, 2025
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants