Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

improve default configuration #1508

Merged
merged 19 commits into from
Aug 7, 2023
Merged

improve default configuration #1508

merged 19 commits into from
Aug 7, 2023

Conversation

bruno-at-bareos
Copy link
Contributor

@bruno-at-bareos bruno-at-bareos commented Jul 11, 2023 •
edited by sebsura
Loading

This PR serve several improvements

  • OP#5304 Add Recycle Pool = Scratch to default Scratch Pool.
  • OP#5459 Replace gzip and md5 by more modern lz4 and xxh128 in default configuration and systemtest
    • We extend fileset-multiple-blocks to test all signature and all compression options we have.
  • OP#5432 Change default for "HardLinks" CFG_TYPE_OPTION to no and explain in documentation the behavior

It also updates the systemtests to use newer compression/checksum algorithms.

Please check

  • Short description and the purpose of this PR is present above this paragraph
  • Your name is present in the AUTHORS file (optional)

If you have any questions or problems, please give a comment in the PR.

Helpful documentation and best practices

Checklist for the reviewer of the PR (will be processed by the Bareos team)

Make sure you check/merge the PR using devtools/pr-tool to have some simple automated checks run and a proper changelog record added.

General
  • Is the PR title usable as CHANGELOG entry?
  • Purpose of the PR is understood
  • Commit descriptions are understandable and well formatted
Source code quality
  • Source code changes are understandable
  • Variable and function names are meaningful
  • Code comments are correct (logically and spelling)
  • Required documentation changes are present and part of the PR

@bruno-at-bareos bruno-at-bareos marked this pull request as ready for review July 12, 2023 14:54
sebsura
sebsura requested changes Jul 13, 2023
View reviewed changes
Copy link
Contributor

@sebsura sebsura left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some small things need to be addressed. Keep in mind that documentation files should also have only 80 characters per line.

docs/manuals/source/Configuration/Director.rst Outdated Show resolved Hide resolved
docs/manuals/source/Configuration/Director.rst Outdated Show resolved Hide resolved
docs/manuals/source/Configuration/Director.rst Outdated Show resolved Hide resolved
docs/manuals/source/Configuration/Director.rst Outdated Show resolved Hide resolved
.../source/manually_added_config_directive_descriptions/dir-director-LogTimestampFormat.rst.inc Outdated Show resolved Hide resolved
...source/manually_added_config_directive_descriptions/fd-filedaemon-LogTimestampFormat.rst.inc Outdated Show resolved Hide resolved
...ls/source/manually_added_config_directive_descriptions/sd-storage-LogTimestampFormat.rst.inc Outdated Show resolved Hide resolved
systemtests/tests/virtualfull-bscan/etc/bareos/bareos-dir.d/pool/Differential.conf Outdated Show resolved Hide resolved
systemtests/tests/virtualfull-deletedfiles/etc/bareos/bareos-dir.d/pool/Differential.conf Outdated Show resolved Hide resolved
core/src/defaultconfigs/bareos-dir.d/fileset/LinuxAll.conf.in Show resolved Hide resolved
@bruno-at-bareos bruno-at-bareos force-pushed the dev/bruno/master/improve-default-configuration branch from 7688f74 to 44631a9 Compare July 13, 2023 13:54
sebsura
sebsura requested changes Jul 14, 2023
View reviewed changes
Copy link
Contributor

@sebsura sebsura left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found some small things.

docs/manuals/source/Configuration/Director.rst Outdated Show resolved Hide resolved
docs/manuals/source/Configuration/Director.rst Outdated Show resolved Hide resolved
.../source/manually_added_config_directive_descriptions/dir-director-LogTimestampFormat.rst.inc Outdated Show resolved Hide resolved
...source/manually_added_config_directive_descriptions/fd-filedaemon-LogTimestampFormat.rst.inc Outdated Show resolved Hide resolved
...ls/source/manually_added_config_directive_descriptions/sd-storage-LogTimestampFormat.rst.inc Outdated Show resolved Hide resolved
@bruno-at-bareos bruno-at-bareos force-pushed the dev/bruno/master/improve-default-configuration branch from 29dcdbe to 4832ed8 Compare July 17, 2023 08:54
sebsura
sebsura requested changes Jul 24, 2023
View reviewed changes
Copy link
Contributor

@sebsura sebsura left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can set the default log timestamp format in bareos itself.

docs/manuals/source/Configuration/Director.rst Outdated Show resolved Hide resolved
.../source/manually_added_config_directive_descriptions/dir-director-LogTimestampFormat.rst.inc Outdated Show resolved Hide resolved
...source/manually_added_config_directive_descriptions/fd-filedaemon-LogTimestampFormat.rst.inc Outdated Show resolved Hide resolved
...ls/source/manually_added_config_directive_descriptions/sd-storage-LogTimestampFormat.rst.inc Outdated Show resolved Hide resolved
sebsura
sebsura requested changes Jul 24, 2023
View reviewed changes
Copy link
Contributor

@sebsura sebsura left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found some trailing whitespace

docs/manuals/source/Configuration/Director.rst Outdated Show resolved Hide resolved
docs/manuals/source/Configuration/Director.rst Outdated Show resolved Hide resolved
docs/manuals/source/Configuration/Director.rst Outdated Show resolved Hide resolved
docs/manuals/source/Configuration/Director.rst Outdated Show resolved Hide resolved
docs/manuals/source/Configuration/Director.rst Outdated Show resolved Hide resolved
@joergsteffens joergsteffens dismissed their stale review July 27, 2023 08:44

My topics are covered and sebsura is already doing a good review job.

@joergsteffens joergsteffens removed their request for review July 27, 2023 08:44
@bruno-at-bareos
doc: Add LogTimestampFormat description, default
835a263 
Signed-off-by: Bruno Friedmann <bruno.friedmann@bareos.com>
@bruno-at-bareos
defaultconfig: add RecyclePool=Scratch to Scratch
3081354 
- It is mandatory for data pool, because the recyclepoolid is not
  updated when a media is affected to this pool data.

Signed-off-by: Bruno Friedmann <bruno.friedmann@bareos.com>
bruno-at-bareos and others added 16 commits August 7, 2023 10:58
@bruno-at-bareos
replace MD5 by XXH128 signatures in systemtests
aaea473 
Signed-off-by: Bruno Friedmann <bruno.friedmann@bareos.com>
@bruno-at-bareos
core + doc: use lz4 and xxh128 by default
3da4cc5 
- replace compression GZIP by LZ4
- replace signature MD5 by XXH128

Signed-off-by: Bruno Friedmann <bruno.friedmann@bareos.com>
@bruno-at-bareos
systemtests: extend fileset-multiple-blocks
0852739 
- test all signature options MD5,SHA1,SHA256,SHA512,XXH128
- test all compression options GZIP,GZIP(LEVEL),LZO,LZ4,LZ4HC

Signed-off-by: Bruno Friedmann <bruno.friedmann@bareos.com>
@bruno-at-bareos
tests: bareos-basic change AutoDeflateAlgorithm
a4ad6e8 
- set lz4 as Auto Deflate Algorithm in FileStorage2
  so different compression are tested.

Signed-off-by: Bruno Friedmann <bruno.friedmann@bareos.com>
@arogge @bruno-at-bareos
fastlz: remove unaligned memory accesses
9209b25 
- some of the helper functions used unaligned memory access as an
  optimization. A quick look at godbolt.org showed that modern
  compilers emit the same code with or without those manual
  optimizations, so the unaligned accesses were removed to avoid
  relying on undefined behaviour.
- core: compression free internal fastlz buffers

Signed-off-by: Bruno Friedmann <bruno.friedmann@bareos.com>
@bruno-at-bareos
core: set hardlinks=no as default
6b3d031 
When you have a file with multiple hardlinks and the first hardlink
encountered gets excluded, then no actual file data is send but the
backup is still finishes without errors.

On restore you will get an error.

- hardlink fix camelcase output in dir_conf.cc
- Add HardLink = No for config-dump and gluster systemtest

Signed-off-by: Bruno Friedmann <bruno.friedmann@bareos.com>
@bruno-at-bareos @sebsura
doc: rework hardlink section
da2bc7d 
- unify the use of "hard links" over mixed "hardlinks"
- reformat reviewed paragraphs to line length 80
- apply check-sources recommendation and format

Signed-off-by: Bruno Friedmann <bruno.friedmann@bareos.com>
Co-authored-by: Sebastian Sura <124262655+sebsura@users.noreply.github.com>
@bruno-at-bareos @sebsura @joergsteffens
apply suggestions from code review
ba90ccb 
Co-authored-by: Sebastian Sura <124262655+sebsura@users.noreply.github.com>
Co-authored-by: Jörg Steffens <joergsteffens@users.noreply.github.com>
@bruno-at-bareos
core: add LogTimestampFormat description & default
100679d 
- create new default json export

Signed-off-by: Bruno Friedmann <bruno.friedmann@bareos.com>
@bruno-at-bareos
doc: apply suggestions hard links
75db36e
@bruno-at-bareos
apply suggestion from review
97eb178
@bruno-at-bareos
doc: hardlink set warning and sinceversion correctly
6191554 
Signed-off-by: Bruno Friedmann <bruno.friedmann@bareos.com>
@bruno-at-bareos
apply strict formating rule for parameter and value
87dcd9a 
- parameter start with Capital letter
- value for compression and signature are fully uppercase
- remove redondant default value in LogTimestampFormat

Signed-off-by: Bruno Friedmann <bruno.friedmann@bareos.com>
@bruno-at-bareos
add trailing / to fix redirection issue
9aa0805 
Signed-off-by: Bruno Friedmann <bruno.friedmann@bareos.com>
@bruno-at-bareos @sebsura
Update docs UsingTapeDrivesWithoutAutochanger.rst
7c8fb75 
Co-authored-by: Sebastian Sura <124262655+sebsura@users.noreply.github.com>
@sebsura @bruno-at-bareos
Update CHANGELOG.md
717509c
@bruno-at-bareos bruno-at-bareos force-pushed the dev/bruno/master/improve-default-configuration branch from 78e1c92 to 717509c Compare August 7, 2023 09:30
@bruno-at-bareos bruno-at-bareos mentioned this pull request Aug 7, 2023
Merged
10 tasks
@BareosBot BareosBot merged commit 502c2c8 into bareos:master Aug 7, 2023
@bruno-at-bareos bruno-at-bareos deleted the dev/bruno/master/improve-default-configuration branch August 14, 2023 12:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@joergsteffens joergsteffens joergsteffens left review comments

@sebsura sebsura sebsura approved these changes

Assignees

@sebsura sebsura

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
5 participants
@bruno-at-bareos @joergsteffens @sebsura @BareosBot @arogge