docs: Add sampling documentation #2854
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
knylander-grafana
added
the
type/docs
|
|
||
| # Sampling scrape targets | ||
|
|
||
| It's not uncommon for an application to have many instances deployed. While Pyroscope is designed specifically to handle large amounts of profiling data, you may have other considerations which demand that only a subset of the application's instances should be scraped. For example, the sheer volume of profiling data your application generates may make it unreasonable to profile every instance or you might be targeting cost-reduction. Whatever the case may be, it's possible to sample scrape targets through configuration of the Grafana Agent. |
There was a problem hiding this comment.
What do you think about this? I'm not crazy about the last sentence in this suggestion. It still sounds awkward.
Suggested change
| It's not uncommon for an application to have many instances deployed. While Pyroscope is designed specifically to handle large amounts of profiling data, you may have other considerations which demand that only a subset of the application's instances should be scraped. For example, the sheer volume of profiling data your application generates may make it unreasonable to profile every instance or you might be targeting cost-reduction. Whatever the case may be, it's possible to sample scrape targets through configuration of the Grafana Agent. | |
| Applications often have many instances deployed. | |
| While Pyroscope is designed to handle large amounts of profiling data, you may want only a subset of the application's instances to be scraped. | |
| For example, the volume of profiling data your application generates may make it unreasonable to profile every instance or you might be targeting cost-reduction. | |
| By configuring Grafana Agent, Pyroscope can sample scrape targets through the Grafana Agent's configuration. |
There was a problem hiding this comment.
I also like this updated version -- better for skimming. I do plan to do a video on this at some point and would use the initial version for that
There was a problem hiding this comment.
I added this change in In 18daf62. I tweaked the last sentence a little because saying "Grafana Agent" twice felt awkward.
|
|
||
| The `hashmod` action and the `modulus` argument are used in conjunction to enable sampling behavior. To read further on these concepts, see [rule block documentation]({{< relref "/docs/agent/latest/flow/reference/components/discovery.relabel#rule-block" >}}). In short, `hashmod` will perform an MD5 hash on the source labels and `modulus` will perform a modulus operation on the output. | ||
|
|
||
| Together, these can be leveraged to shard targets into shards. One or more shards can be selected to be the sampled targets to be scraped. Here is an example: |
There was a problem hiding this comment.
I'm not sure I understand the phrase "share targets into shards." If you have better wording, please feel free to edit this suggestion to update.
Suggested change
| Together, these can be leveraged to shard targets into shards. One or more shards can be selected to be the sampled targets to be scraped. Here is an example: | |
| Together, they can be leveraged to shard targets into shards. One or more shards can be selected to be the sampled targets to be scraped. Here is an example: |
There was a problem hiding this comment.
Suggested change
| Together, these can be leveraged to shard targets into shards. One or more shards can be selected to be the sampled targets to be scraped. Here is an example: | |
| Together, these can be leveraged to divide targets into shards. Once divided, you can configure what percentage of the shards should be scraped. Here is an example: |
There was a problem hiding this comment.
In In 18daf62, I also reworked this part. I went into a little more detail about how hashmod will shard the labels and keep will select shards.
It might still need some work.
knylander-grafana
approved these changes
Dec 15, 2023
knylander-grafana
pushed a commit
that referenced
this pull request
Dec 15, 2023
* Add sampling doc * Tweak wording and layout * Fix links * Fix absolute links * Variety of small tweaks in structure and wording
knylander-grafana
added a commit
that referenced
this pull request
Dec 16, 2023
* Changes from 2855 intro doc * docs: update go-pull.md (#2844) * docs: update go-pull.md * fix links * rm comments * Update docs/sources/configure-client/grafana-agent/go_pull.md Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com> * Update docs/sources/configure-client/grafana-agent/go_pull.md Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com> * Update docs/sources/configure-client/grafana-agent/go_pull.md Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com> * Update docs/sources/configure-client/grafana-agent/go_pull.md Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com> * Update docs/sources/configure-client/grafana-agent/go_pull.md Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com> * Update docs/sources/configure-client/grafana-agent/go_pull.md Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com> * Update docs/sources/configure-client/grafana-agent/go_pull.md Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com> * Update docs/sources/configure-client/grafana-agent/go_pull.md Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com> * Update docs/sources/configure-client/grafana-agent/go_pull.md Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com> * Update docs/sources/configure-client/grafana-agent/go_pull.md Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com> * typo * Update docs/sources/configure-client/grafana-agent/go_pull.md Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com> --------- Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com> Co-authored-by: Ryan Perry <Rperry2174@gmail.com> * Create release notes for v1.2.1 (#2860) * docs: Add sampling documentation (#2854) * Add sampling doc * Tweak wording and layout * Fix links * Fix absolute links * Variety of small tweaks in structure and wording * docs: Refactor ebpf installation (#2849) * docs: Refactor ebpf installation * docs: Refactor ebpf installation * Remove not useful sections. * review feedbac * remove absolute link to grafana.com * lint * review feedback * Add diagram for ebpf * linted * Update docs/sources/configure-client/grafana-agent/ebpf/_index.md Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com> * Update docs/sources/configure-client/grafana-agent/ebpf/_index.md Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com> * Update docs/sources/configure-client/grafana-agent/ebpf/_index.md Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com> * Update docs/sources/configure-client/grafana-agent/ebpf/configuration/_index.md Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com> * Update docs/sources/configure-client/grafana-agent/ebpf/configuration/_index.md Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com> * Update docs/sources/configure-client/grafana-agent/ebpf/setup-docker.md Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com> * Update docs/sources/configure-client/grafana-agent/ebpf/setup-docker.md Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com> * Update docs/sources/configure-client/grafana-agent/ebpf/setup-kubernetes.md Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com> * Update docs/sources/configure-client/grafana-agent/ebpf/setup-kubernetes.md Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com> * Update docs/sources/configure-client/grafana-agent/ebpf/setup-linux.md Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com> * Update docs/sources/configure-client/grafana-agent/ebpf/setup-linux.md Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com> * Update docs/sources/configure-client/grafana-agent/ebpf/setup-linux.md Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com> * Update docs/sources/configure-client/grafana-agent/ebpf/troubleshooting.md Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com> * Update docs/sources/configure-client/grafana-agent/ebpf/troubleshooting.md Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com> * Update docs/sources/configure-client/grafana-agent/ebpf/troubleshooting.md Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com> * Update docs/sources/configure-client/grafana-agent/ebpf/troubleshooting.md Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com> * Update docs/sources/configure-client/grafana-agent/ebpf/setup-linux.md Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com> * Update docs/sources/configure-client/grafana-agent/ebpf/setup-linux.md Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com> * Update docs/sources/configure-client/grafana-agent/ebpf/setup-kubernetes.md Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com> --------- Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com> Co-authored-by: Ryan Perry <Rperry2174@gmail.com> * docs: language SDK page updates (#2851) * docs changes * improvements * wip * fixes * fixes * fixes * Update docs/sources/configure-client/language-sdks/go_push.md * Update docs/sources/configure-client/language-sdks/go_push.md * Apply suggestions from code review * Update Before you begin sections --------- Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com> Co-authored-by: Kim Nylander <kim.nylander@grafana.com> * doc: Add sampling diagram (#2861) --------- Co-authored-by: Tolya Korniltsev <korniltsev.anatoly@gmail.com> Co-authored-by: Ryan Perry <Rperry2174@gmail.com> Co-authored-by: Bryan Huhta <32787160+bryanhuhta@users.noreply.github.com> Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: Dmitry Filimonov <dmitry.filimonov@grafana.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
closes #2767
Adds documentation on how to sample scrape targets using the Grafana Agent. This is applicable for eBPF profiling as well as the typical Pyroscope pull mode profiling.