Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pkg/parca: Add --http-read-timeout and --http-write-timeout options #3689

Conversation

quite4work
Copy link
Contributor

Hi,

I had troubles with loading debug information of a big Rust binary (~1.2Gb).
Because debug information wasn't loaded, there were no function names in the samples.

Every time the upload failed after 5 seconds with following error:

{
  "caller": "server.go:321",
  "grpc.code": "Internal",
  "grpc.component": "server",
  "grpc.error": "rpc error: code = Internal desc = upload debuginfo: copy to /var/lib/parca/debuginfo/.../debuginfo: get next upload chunk (343932928 bytes read so far): receive from stream: rpc error: code = Unavailable desc = i/o timeout",
  "grpc.method": "Upload",
  "grpc.method_type": "client_stream",
  "grpc.request.deadline": "2023-08-03T10:53:19Z",
  "grpc.service": "parca.debuginfo.v1alpha1.DebuginfoService",
  "grpc.start_time": "2023-08-03T10:38:19Z",
  "grpc.time_ms": "5000.786", // coresponds to `ReadTimeout:  5 * time.Second`
  "level": "error",
  "msg": "finished call",
  "name": "parca",
  "peer.address": "10.0.30.5:38962",
  "protocol": "grpc",
  "ts": "2023-08-03T10:38:24.685793547Z"
}

I was able to solve this by increasing HTTP read and write timeouts.

My issue has some similarity to #2208, but that one has different errors and different solution.
I haven't encountered errors described there, so I am not sure if it solves #2208.

@quite4work quite4work requested review from a team as code owners August 25, 2023 14:52
@CLAassistant
Copy link

CLAassistant commented Aug 25, 2023

CLA assistant check
All committers have signed the CLA.

Copy link
Member

@brancz brancz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you so much for the contribution! Lgtm

@quite4work quite4work force-pushed the add-read-write-timeout-options-for-parca-server branch from 9c6606b to 87430b4 Compare August 29, 2023 09:09
@brancz
Copy link
Member

brancz commented Aug 30, 2023

The docs are auto-generated, do you mind running make README.md? Then I'm happy to merge this.

auto-merge was automatically disabled August 30, 2023 11:46

Head branch was pushed to by a user without write access

@quite4work
Copy link
Contributor Author

@brancz done

Also I have removed Defaults to ... at the end of the descriptions, because it just repeats the same information as at the begining. For example:

--http-read-timeout=5s Timeout duration for HTTP server to read request body. Defaults to 5s

@brancz brancz merged commit c306fcd into parca-dev:main Aug 31, 2023
34 checks passed
@manojVivek
Copy link
Contributor

@all-contributors Please add @quite4work for code.

@allcontributors
Copy link
Contributor

@manojVivek

I've put up a pull request to add @quite4work! 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants