Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

app/vmalert: add retries to remotewrite #605

Merged
merged 7 commits into from
Jul 5, 2020
Merged

Conversation

hagen1778
Copy link
Collaborator

Remotewrite pkg now does limited number of retries if write request failed.
This suppose to make vmalert state persisting more reliable.

New metrics were added to remotewrite in order to track rows/bytes sent/dropped.

defaultFlushInterval was increased from 1s to 5s for sanity reasons.

Remotewrite pkg now does limited number of retries if write request failed.
This suppose to make vmalert state persisting more reliable.

New metrics were added to remotewrite in order to track rows/bytes sent/dropped.

defaultFlushInterval was increased from 1s to 5s for sanity reasons.
@hagen1778 hagen1778 force-pushed the vmalert/remotewrite-retries branch from ddf36ab to cc26cb7 Compare July 4, 2020 10:59
@codecov
Copy link

codecov bot commented Jul 5, 2020

Codecov Report

Merging #605 into master will decrease coverage by 0.03%.
The diff coverage is 56.25%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #605      +/-   ##
==========================================
- Coverage   68.08%   68.05%   -0.04%     
==========================================
  Files         154      156       +2     
  Lines       21298    21431     +133     
==========================================
+ Hits        14500    14584      +84     
- Misses       5633     5675      +42     
- Partials     1165     1172       +7     
Impacted Files Coverage Δ
app/vmalert/group.go 75.62% <ø> (+0.46%) ⬆️
app/vmalert/remotewrite/remotewrite.go 62.71% <56.25%> (ø)
app/vmalert/remotewrite/init.go 0.00% <0.00%> (ø)
lib/mergeset/table.go 68.29% <0.00%> (+0.27%) ⬆️
lib/storage/partition.go 72.01% <0.00%> (+0.47%) ⬆️
lib/storage/merge.go 82.64% <0.00%> (+3.30%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ff6a095...f3616ee. Read the comment docs.

@valyala valyala merged commit 703def4 into master Jul 5, 2020
@valyala valyala deleted the vmalert/remotewrite-retries branch July 5, 2020 15:46
@valyala
Copy link
Collaborator

valyala commented Jul 5, 2020

Thanks!

valyala pushed a commit that referenced this pull request Jul 5, 2020
* app/vmalert: add retries to remotewrite

Remotewrite pkg now does limited number of retries if write request failed.
This suppose to make vmalert state persisting more reliable.

New metrics were added to remotewrite in order to track rows/bytes sent/dropped.

defaultFlushInterval was increased from 1s to 5s for sanity reasons.

* fix

* wip

* wip

* wip

* fix bits alignment bug for 32-bit systems

* fix mistakenly dropped field
@valyala
Copy link
Collaborator

valyala commented Jul 8, 2020

The commit has been included in v1.38.0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants