Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

logging: added log rotation and improved predictability of log sweep #1562

Merged
merged 2 commits into from Dec 4, 2021

Conversation

jkowalski
Copy link
Contributor

With this change logs will be rotated every 50 MB, which prevents
accumulation of giant files while server is running.

This change will also guarantee that log sweep completes at least once
before each invocation of Kopia finishes. Previously it was a goroutine
that was not monitored for completion.

Flags can be used to override default behaviors:

  • --max-log-file-segment-size
  • --no-wait-for-log-sweep - disables waiting for full log sweep

Fixes #1561

@codecov
Copy link

codecov bot commented Dec 3, 2021

Codecov Report

Merging #1562 (0ce3ec0) into master (920341c) will increase coverage by 0.08%.
The diff coverage is 85.71%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1562      +/-   ##
==========================================
+ Coverage   69.18%   69.26%   +0.08%     
==========================================
  Files         375      375              
  Lines       29795    29866      +71     
==========================================
+ Hits        20614    20688      +74     
+ Misses       7501     7496       -5     
- Partials     1680     1682       +2     
Impacted Files Coverage Δ
internal/logfile/logfile.go 79.06% <83.95%> (+3.85%) ⬆️
cli/app.go 74.02% <100.00%> (+0.76%) ⬆️
cli/command_content_verify.go 73.68% <0.00%> (-5.27%) ⬇️
repo/content/merged.go 86.11% <0.00%> (-2.78%) ⬇️
repo/blob/filesystem/filesystem_storage.go 63.68% <0.00%> (-1.50%) ⬇️
internal/epoch/epoch_manager.go 88.61% <0.00%> (ø)
repo/content/content_manager_iterate.go 91.71% <0.00%> (+2.20%) ⬆️
internal/ownwrites/ownwrites.go 84.90% <0.00%> (+3.77%) ⬆️
internal/listcache/listcache.go 83.33% <0.00%> (+3.92%) ⬆️
... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 920341c...0ce3ec0. Read the comment docs.

With this change logs will be rotated every 50 MB, which prevents
accumulation of giant files while server is running.

This change will also guarantee that log sweep completes at least once
before each invocation of Kopia finishes. Previously it was a goroutine
that was not monitored for completion.

Flags can be used to override default behaviors:

* `--max-log-file-segment-size`
* `--no-wait-for-log-sweep` - disables waiting for full log sweep

Fixes kopia#1561
This limits the total size of all logs in a directory to 1 GB.
@jkowalski jkowalski merged commit 2cb05f7 into kopia:master Dec 4, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

KopiaUI: CLI logs using 250+ GBs?!
1 participant