-
Notifications
You must be signed in to change notification settings - Fork 343
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(cli): C: Add ability to dump pprof data to logs ... #3473
Conversation
137fd5e
to
1c97f40
Compare
…to pprof_extensions-C
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is moving in the right direction and getting closer to "landing" it.
Please see inline comments.
Co-authored-by: Julio López <1953782+julio-lopez@users.noreply.github.com>
Co-authored-by: Julio López <1953782+julio-lopez@users.noreply.github.com>
Co-authored-by: Julio López <1953782+julio-lopez@users.noreply.github.com>
Co-authored-by: Julio López <1953782+julio-lopez@users.noreply.github.com>
…to pprof_extensions-C
…to pprof_extensions-C
…to pprof_extensions-C
…to pprof_extensions-C
@@ -242,6 +242,7 @@ func (c *commandRepositorySyncTo) runSyncBlobs(ctx context.Context, src blob.Rea | |||
|
|||
numBlobs, bytesCopied := totalCopied.Add(m.Length) | |||
progressMutex.Lock() | |||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit; drop this addition of line
@@ -0,0 +1,46 @@ | |||
//go:build darwin || linux |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This can become a separate PR.
// run the tests | ||
result := m.Run() | ||
|
||
os.Exit(result) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should check that the dump was created after SIGTERM.
@@ -54,7 +54,6 @@ func (e *CLIExeRunner) Start(t *testing.T, args []string, env map[string]string) | |||
return stdoutPipe, stderrPipe, c.Wait, func(sig os.Signal) { | |||
if sig == os.Kill { | |||
c.Process.Kill() | |||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit; drop this change
This PR adds the ability to dump pprof data to logs for debugging.
This is a continuation of #3454
This is one of 4 PRs in a PR train:
aaron-kasten/kopia:pprof-extensions-A
aaron-kasten/kopia:pprof-extensions-B
aaron-kasten/kopia:pprof-extensions-C
aaron-kasten/kopia:pprof-extensions-D
Usage
pprof dumps are configured using the
KOPIA_DEBUG_PPROF
environment variable. The variable is a list of pprof profile names (seepprof.Lookup
) separated by,
. Optional parameters can be set with '=', delimited by ':'.example:
export KOPIA_DEBUG_PPROF=cpu,heap=debug=1,mutex=debug=1:rate=1000
The above setting will produce CPU, heap and mutex profiles. The block profile will have its debug parameter set to 1 and its sample rate set to 1000
Once run, profile data will be output in the Kopia logs on termination. Profile dumps are generated as base64 output (PEM) to the log on termination.
You should consider captureing logs to a file when running the Kopia command:
Once the logs are captured, a dump can created by terminating the command:
The following signals (on Linux and macos) can be used to dump profiles: SIGTERM, SIGINT, and SIGUSR1.
Captured standard-output should look similar to:
The captured output can then be converted to a pprof binary by using
kats
. The Kopiakats
tool can be used to convert the PEM file into a binary:When successful, kats will output the file found in the capture file.
kats expects that there is a well formed PEM record in the capture file.
Once successful, the binary can be used in PPROF:
Configuration Options
TBD