Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adjust ElasticSearch index refresh period as INT(flushInterval * 2/3) #7310

Merged
merged 2 commits into from
Jul 16, 2021

Conversation

wu-sheng
Copy link
Member

  • If this pull request closes/resolves/fixes an existing issue, replace the issue number. Closes #.
  • Update the CHANGES log.

Adjust index refresh period as INT(flushInterval * 2/3), it used to be as same as bulk flush period. At the edge case, in low traffic(traffic < bulkActions in the whole period), there is a possible case, 2 period bulks are included in one index refresh rebuild operation, which could cause version conflicts. And this case can't be fixed through core/persistentPeriod as the bulk fresh is not controlled by the persistent timer anymore.

This PR should avoid the following exception in the low load case, especially when bulkActions is set larger than the number of a metric type.
image

@wu-sheng wu-sheng added core feature Core and important feature. Sometimes, break backwards compatibility. backend OAP backend related. labels Jul 16, 2021
@wu-sheng wu-sheng added this to the 8.7.0 milestone Jul 16, 2021
@codecov
Copy link

codecov bot commented Jul 16, 2021

Codecov Report

Merging #7310 (dc56d4f) into master (408bbf8) will decrease coverage by 14.73%.
The diff coverage is 50.00%.

❗ Current head dc56d4f differs from pull request most recent head 663a8bd. Consider uploading reports for the commit 663a8bd to get more accurate results
Impacted file tree graph

@@              Coverage Diff              @@
##             master    #7310       +/-   ##
=============================================
- Coverage     59.03%   44.29%   -14.74%     
+ Complexity     4373     2747     -1626     
=============================================
  Files          1027     1890      +863     
  Lines         26449    40598    +14149     
  Branches       2618     4749     +2131     
=============================================
+ Hits          15614    17983     +2369     
- Misses         9457    21508    +12051     
+ Partials       1378     1107      -271     
Impacted Files Coverage Δ
...lasticsearch/StorageModuleElasticsearchConfig.java 100.00% <ø> (ø)
.../plugin/elasticsearch/base/StorageEsInstaller.java 79.67% <50.00%> (-2.00%) ⬇️
...skywalking/oap/server/core/alarm/AlarmMessage.java 0.00% <0.00%> (-100.00%) ⬇️
...ywalking/apm/agent/core/plugin/EnhanceContext.java 0.00% <0.00%> (-100.00%) ⬇️
...walking/apm/agent/core/plugin/match/NameMatch.java 0.00% <0.00%> (-100.00%) ⬇️
...ing/oap/server/core/storage/IHistoryDeleteDAO.java 0.00% <0.00%> (-100.00%) ⬇️
...ng/oap/server/core/analysis/config/NoneStream.java 0.00% <0.00%> (-100.00%) ⬇️
...log/analyzer/provider/log/ILogAnalyzerService.java 0.00% <0.00%> (-100.00%) ⬇️
...ement/ui/template/UITemplateManagementService.java 0.00% <0.00%> (-100.00%) ⬇️
...gin/interceptor/enhance/MethodInterceptResult.java 0.00% <0.00%> (-100.00%) ⬇️
... and 1455 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 408bbf8...663a8bd. Read the comment docs.

@wu-sheng wu-sheng requested a review from kezhenxu94 July 16, 2021 03:01
@wu-sheng wu-sheng merged commit 8df362b into master Jul 16, 2021
@wu-sheng wu-sheng deleted the polish-refresh branch July 16, 2021 04:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend OAP backend related. core feature Core and important feature. Sometimes, break backwards compatibility.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants