Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

scylla_coredump_setup should be called with --compress to enable zstd compression of the core dumps #462

Closed
mykaul opened this issue Jun 29, 2023 · 9 comments
Closed

scylla_coredump_setup should be called with --compress to enable zstd compression of the core dumps #462

mykaul opened this issue Jun 29, 2023 · 9 comments
Assignees
@syuu1228

Comments

@mykaul
Copy link

mykaul commented Jun 29, 2023

Need to verify, but if Ubuntu 22.2 LTS systemd is compiled with zstd as default compression, we must configure the coredump to be compressed.
This will reduce significantly the time it takes to write the coredump, to transfer it, etc.
Example:

ykaul@ykaul Downloads]$ file ip-172-18-2-215
ip-172-18-2-215: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from '/usr/bin/scylla --log-to-syslog 1 --log-to-stdout 0 --default-log-level info --', real uid: 114, effective uid: 114, real gid: 120, effective gid: 120, execfn: '/opt/scylladb/libexec/scylla', platform: 'x86_64'
[ykaul@ykaul Downloads]$ du -ch ip-172-18-2-215
14G	ip-172-18-2-215
14G	total
[ykaul@ykaul Downloads]$ time zstd ip-172-18-2-215
ip-172-18-2-215      : 11.17%   (  13.8 GiB =>   1.55 GiB, ip-172-18-2-215.zst)
real	0m19.028s
user	0m20.921s
sys	0m3.490s
[ykaul@ykaul Downloads]$ du -ch ip-172-18-2-215.zst
1.6G	ip-172-18-2-215.zst
1.6G	total

I had an ever more drastic example of 111GB compressed by 90% reduction of size.
@mykaul mykaul mentioned this issue Feb 29, 2024
Closed
@roydahan roydahan removed their assignment Mar 4, 2024
@yaronkaikov yaronkaikov assigned syuu1228 and unassigned yaronkaikov Mar 4, 2024
@mykaul
Copy link
Author

mykaul commented May 5, 2024

Example for benefit (not only faster, but takes far less space) - see scylladb/scylla-cluster-tests#7234 (comment)

syuu1228 added a commit to syuu1228/scylla that referenced this issue May 23, 2024
@syuu1228
scylla_coredump_setup: enable compress by default when zstd support d…
c1fb2c9 
…etected

We disabled coredump compression by default because it was too slow,
but recent versions of systemd-coredump supports faster zstd based compression,
so let's enable compression by default when zstd support detected.

Related scylladb/scylla-machine-image#462
@syuu1228 syuu1228 mentioned this issue May 23, 2024
Closed
@syuu1228
Copy link
Contributor

Sent patch to enable compress by default when ZSTD detected:
scylladb/scylladb#18854

Implemented it on scylla_coredump_setup not machine-image script, since this looks like generic issue not just machine-image.

fruch added a commit to fruch/scylla-cluster-tests that referenced this issue Jun 18, 2024
@fruch
feature(coredumps): support zstd compressed coredumps
e85aced 
if we have coredumps compressed with zstd, we should compress them again
also we should write the proper tool in the download instructions

Ref: scylladb/scylla-machine-image#462
Ref: scylladb/scylladb#18854
@fruch fruch mentioned this issue Jun 18, 2024
Merged
2 tasks
fruch added a commit to fruch/scylla-cluster-tests that referenced this issue Jun 23, 2024
@fruch
feature(coredumps): support zstd compressed coredumps
7bfbf31 
if we have coredumps compressed with zstd, we shouldn't compress them again
also we should write the proper tool in the download instructions

Ref: scylladb/scylla-machine-image#462
Ref: scylladb/scylladb#18854
fruch added a commit to scylladb/scylla-cluster-tests that referenced this issue Jun 23, 2024
@fruch
feature(coredumps): support zstd compressed coredumps
20587f4 
if we have coredumps compressed with zstd, we shouldn't compress them again
also we should write the proper tool in the download instructions

Ref: scylladb/scylla-machine-image#462
Ref: scylladb/scylladb#18854
mergify bot pushed a commit to scylladb/scylla-cluster-tests that referenced this issue Jun 23, 2024
@fruch @mergify
feature(coredumps): support zstd compressed coredumps
59ad7c1 
if we have coredumps compressed with zstd, we shouldn't compress them again
also we should write the proper tool in the download instructions

Ref: scylladb/scylla-machine-image#462
Ref: scylladb/scylladb#18854
(cherry picked from commit 20587f4)
@mergify mergify bot mentioned this issue Jun 23, 2024
Merged
2 tasks
fruch added a commit to scylladb/scylla-cluster-tests that referenced this issue Jun 23, 2024
@fruch
feature(coredumps): support zstd compressed coredumps
5a00aba 
if we have coredumps compressed with zstd, we shouldn't compress them again
also we should write the proper tool in the download instructions

Ref: scylladb/scylla-machine-image#462
Ref: scylladb/scylladb#18854
(cherry picked from commit 20587f4)
denesb pushed a commit to scylladb/scylladb that referenced this issue Jul 4, 2024
@syuu1228 @denesb
scylla_coredump_setup: enable compress by default when zstd support d…
09e2269 
…etected

We disabled coredump compression by default because it was too slow,
but recent versions of systemd-coredump supports faster zstd based compression,
so let's enable compression by default when zstd support detected.

Related scylladb/scylla-machine-image#462

Closes #18854
@yaronkaikov
Copy link
Collaborator

@syuu1228 Can this be closed? i saw that the PR was promoted

@fruch
Copy link
Collaborator

fruch commented Jul 7, 2024

and this is breaking artifacts jobs
https://jenkins.scylladb.com/job/scylla-master/job/artifacts/job/artifacts-ubuntu2204-test/576/

@mykaul
Copy link
Author

mykaul commented Jul 8, 2024

and this is breaking artifacts jobs https://jenkins.scylladb.com/job/scylla-master/job/artifacts/job/artifacts-ubuntu2204-test/576/

@yaronkaikov - please revert.

@yaronkaikov
Copy link
Collaborator

and this is breaking artifacts jobs https://jenkins.scylladb.com/job/scylla-master/job/artifacts/job/artifacts-ubuntu2204-test/576/

@yaronkaikov - please revert.

I can't revert, it's in Scylla core scylladb/scylladb#18854

@yaronkaikov
Copy link
Collaborator

@mykaul , @syuu1228 is working on a fix now, so i expect it to be ready today (hopefully)

@syuu1228 please update

@yaronkaikov
Copy link
Collaborator

And he just sent it scylladb/scylladb#19648

@syuu1228
Copy link
Contributor

syuu1228 commented Aug 5, 2024

Closing, since the patch is merged scylladb/scylladb@cbf33ab

@syuu1228 syuu1228 closed this as completed Aug 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@syuu1228 syuu1228

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@syuu1228 @fruch @mykaul @yaronkaikov @roydahan