Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

app/vmagent: added remote write context groups #6344

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

AndrewChubatiuk
Copy link
Contributor

Describe Your Changes

Added remote write context groups

Checklist

The following checks are mandatory:

@AndrewChubatiuk AndrewChubatiuk marked this pull request as draft May 24, 2024 12:30
@AndrewChubatiuk AndrewChubatiuk self-assigned this May 24, 2024
@AndrewChubatiuk AndrewChubatiuk force-pushed the remote-write-groups branch 2 times, most recently from 8479a50 to 9d612bf Compare May 24, 2024 13:28
@AndrewChubatiuk AndrewChubatiuk force-pushed the remote-write-groups branch 3 times, most recently from df61e8b to c236f8b Compare May 26, 2024 06:57
@tenmozes tenmozes changed the title app/vmagent: added added remote write context groups app/vmagent: added remote write context groups May 26, 2024
@hagen1778
Copy link
Collaborator

@AndrewChubatiuk do I understand correctly that with this PR I can have the following config:

# global relabeling for all incoming data
-remoteWrite.relabelConfig=/local/relabel_config.yml

# shard 1
-remoteWrite.urlRelabelConfig=shard-1/local/1_url_relabel.yml
-remoteWrite.url=shard-1/http://shard1-1/insert/0/prometheus/api/v1/write
-remoteWrite.url=shard-1/http://shard1-2/insert/0/prometheus/api/v1/write

# shard 2
-remoteWrite.urlRelabelConfig=shard-2/local/2_url_relabel.yml
-remoteWrite.url=shard-2/http://shard2-1/insert/0/prometheus/api/v1/write
-remoteWrite.url=shard-2/http://shard2-2/insert/0/prometheus/api/v1/write

based on #6212 (comment)
, where data is replicated across shards, and then replicated to each rwctx within the shard. The individual relabeling per rwctx then acts as a sharding mechanism.

In general, the workflow with new architecture is something like below:
image


Please note, specifying group name in -remoteWrite.urlRelabelConfig seems conflicting with file path.

@AndrewChubatiuk
Copy link
Contributor Author

AndrewChubatiuk commented Jun 5, 2024

diagram doesn't describe code changes precise enough

  1. streaming aggregation and relabeling for rwctx and group are mutually exclusive, if group name is set aggregation and relabeling will happen on a group level if not - on rwctx for a backward compatibility. there're no possible scenarios for 3 consecutive aggregations or relabelings
  2. copy to each rwctx should be rather changed to copy to or shard among rwctxs

@AndrewChubatiuk AndrewChubatiuk force-pushed the remote-write-groups branch 2 times, most recently from da8c51c to a5269ef Compare June 7, 2024 16:55
@AndrewChubatiuk AndrewChubatiuk marked this pull request as ready for review June 11, 2024 10:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

vmagent: support multiple groups of remoteWrite.urls to support directing samples to different groups
2 participants