[Backport] [2.x] Enabling Compression Level Index Settings for ZSTD and ZSTD Codecs #8558

sarthakaggarwal97 · 2023-07-09T08:59:38Z

Description

Backport #8312

Related Issues

Resolves #7555

Check List

New functionality includes testing.
- All tests pass
New functionality has been documented.
- New functionality has javadoc added
Commits are signed per the DCO using --signoff
Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

github-actions · 2023-07-09T09:04:57Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/19525/
CommitID: 0b15bf2
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-07-09T09:12:15Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/19526/
CommitID: 5559781
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-07-09T09:55:02Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/19528/
CommitID: 2c8486f

codecov · 2023-07-09T09:55:41Z

Codecov Report

Merging #8558 (d829bda) into 2.x (aecf435) will increase coverage by 0.09%.
The diff coverage is 93.54%.

❗ Current head d829bda differs from pull request most recent head eb87477. Consider uploading reports for the commit eb87477 to get more accurate results

@@             Coverage Diff              @@
##                2.x    #8558      +/-   ##
============================================
+ Coverage     70.59%   70.68%   +0.09%     
- Complexity    57082    57105      +23     
============================================
  Files          4744     4744              
  Lines        270579   270606      +27     
  Branches      39939    39943       +4     
============================================
+ Hits         191003   191286     +283     
+ Misses        63139    62894     -245     
+ Partials      16437    16426      -11

Impacted Files	Coverage Δ
...pensearch/common/settings/IndexScopedSettings.java	`100.00% <ø> (ø)`
.../java/org/opensearch/index/codec/CodecService.java	`95.65% <92.00%> (-4.35%)`	⬇️
...customcodecs/Lucene95CustomStoredFieldsFormat.java	`74.07% <100.00%> (+2.07%)`	⬆️
...opensearch/index/codec/customcodecs/ZstdCodec.java	`100.00% <100.00%> (ø)`
...arch/index/codec/customcodecs/ZstdNoDictCodec.java	`85.71% <100.00%> (-14.29%)`	⬇️
...java/org/opensearch/index/engine/EngineConfig.java	`98.51% <100.00%> (+0.01%)`	⬆️
...in/java/org/opensearch/index/shard/IndexShard.java	`69.53% <100.00%> (+0.54%)`	⬆️

... and 495 files with indirect coverage changes

github-actions · 2023-07-09T11:19:20Z

Gradle Check (Jenkins) Run Completed with:

RESULT: UNSTABLE ❕
TEST FAILURES:

      1 org.opensearch.action.admin.cluster.node.tasks.ResourceAwareTasksTests.testBasicTaskResourceTracking

URL: https://build.ci.opensearch.org/job/gradle-check/19534/
CommitID: 9cf569b
Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

github-actions · 2023-07-09T12:39:32Z

Gradle Check (Jenkins) Run Completed with:

RESULT: UNSTABLE ❕
TEST FAILURES:

      1 org.opensearch.search.ConcurrentSegmentSearchTimeoutIT.testSimpleTimeout

URL: https://build.ci.opensearch.org/job/gradle-check/19536/
CommitID: dbbc877
Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

server/src/main/java/org/opensearch/index/codec/CodecService.java

github-actions · 2023-07-10T05:30:22Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/19546/
CommitID: 616419f
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

…ject#8312) * Enabling compression levels as index settings Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * addressing review comments Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * refactoring codec service Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * index settings assertion Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * default compression level change in the settings Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> --------- Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> (cherry picked from commit ca7045e)

github-actions · 2023-07-10T08:31:39Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/19554/
CommitID: cf3e913
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-07-10T09:13:32Z

Gradle Check (Jenkins) Run Completed with:

RESULT: UNSTABLE ❕
TEST FAILURES:

      1 org.opensearch.repositories.azure.AzureBlobContainerRetriesTests.testWriteBlobWithRetries
      1 org.opensearch.remotestore.SegmentReplicationUsingRemoteStoreIT.testDropPrimaryDuringReplication

URL: https://build.ci.opensearch.org/job/gradle-check/19564/
CommitID: d829bda
Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

server/src/main/java/org/opensearch/index/codec/CodecService.java

github-actions · 2023-07-10T13:33:00Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/19576/
CommitID: 6ff8abf
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-07-10T13:52:20Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/19594/
CommitID: eb87477
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-07-10T13:57:46Z

Gradle Check (Jenkins) Run Completed with:

RESULT: null ❌
URL: null
CommitID: c573940
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-07-10T14:00:54Z

Gradle Check (Jenkins) Run Completed with:

RESULT: null ❌
URL: null
CommitID: b7ae674
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

github-actions · 2023-07-10T15:10:21Z

Gradle Check (Jenkins) Run Completed with:

RESULT: UNSTABLE ❕
TEST FAILURES:

      2 org.opensearch.cluster.service.MasterServiceTests.classMethod
      1 org.opensearch.cluster.service.MasterServiceTests.testThrottlingForMultipleTaskTypes

URL: https://build.ci.opensearch.org/job/gradle-check/19633/
CommitID: 4bb4c09
Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

reta · 2023-07-10T17:55:25Z

There are flaky tests out of this pull request: #8593, @sarthakaggarwal97 please take a look

sarthakaggarwal97 force-pushed the backport/backport-8312-to-2.x branch from 0b15bf2 to 5559781 Compare July 9, 2023 09:00

sarthakaggarwal97 force-pushed the backport/backport-8312-to-2.x branch from 5559781 to 2c8486f Compare July 9, 2023 09:13

sarthakaggarwal97 changed the title ~~[Backport] Enabling Compression Level Index Settings for ZSTD and ZSTD Codecs~~ [Backport] [2.x] Enabling Compression Level Index Settings for ZSTD and ZSTD Codecs Jul 9, 2023

sarthakaggarwal97 force-pushed the backport/backport-8312-to-2.x branch from 2c8486f to 9cf569b Compare July 9, 2023 10:35

sarthakaggarwal97 force-pushed the backport/backport-8312-to-2.x branch from 9cf569b to dbbc877 Compare July 9, 2023 12:00

sarthakaggarwal97 marked this pull request as ready for review July 9, 2023 12:41

sarthakaggarwal97 requested review from reta, anasalkouz, andrross, Bukhtawar, CEHENKLE, dblock, gbbafna, setiah, kartg, kotwanikunal, mch2, nknize, owaiskazi19, Rishikesh1159, ryanbogan, saratvemulapalli, shwetathareja and dreamer-89 as code owners July 9, 2023 12:41

sarthakaggarwal97 requested review from tlfeng, VachaShah, dbwiddis and sachinpkale as code owners July 9, 2023 12:41

reta reviewed Jul 9, 2023

View reviewed changes

server/src/main/java/org/opensearch/index/codec/CodecService.java Outdated Show resolved Hide resolved

sarthakaggarwal97 force-pushed the backport/backport-8312-to-2.x branch from dbbc877 to 616419f Compare July 10, 2023 04:15

sarthakaggarwal97 force-pushed the backport/backport-8312-to-2.x branch from 616419f to cf3e913 Compare July 10, 2023 07:20

sarthakaggarwal97 force-pushed the backport/backport-8312-to-2.x branch from cf3e913 to d829bda Compare July 10, 2023 08:31

mgodwan approved these changes Jul 10, 2023

View reviewed changes

reta reviewed Jul 10, 2023

View reviewed changes

server/src/main/java/org/opensearch/index/codec/CodecService.java Show resolved Hide resolved

sarthakaggarwal97 force-pushed the backport/backport-8312-to-2.x branch from d829bda to 6ff8abf Compare July 10, 2023 13:16

reta approved these changes Jul 10, 2023

View reviewed changes

sarthakaggarwal97 force-pushed the backport/backport-8312-to-2.x branch from 6ff8abf to eb87477 Compare July 10, 2023 13:38

sarthakaggarwal97 force-pushed the backport/backport-8312-to-2.x branch from eb87477 to c573940 Compare July 10, 2023 13:57

sarthakaggarwal97 force-pushed the backport/backport-8312-to-2.x branch from c573940 to b7ae674 Compare July 10, 2023 14:00

refactoring CodecService for 2.x

4bb4c09

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

sarthakaggarwal97 force-pushed the backport/backport-8312-to-2.x branch from b7ae674 to 4bb4c09 Compare July 10, 2023 14:28

reta merged commit fc30d6c into opensearch-project:2.x Jul 10, 2023
6 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Backport] [2.x] Enabling Compression Level Index Settings for ZSTD and ZSTD Codecs #8558

[Backport] [2.x] Enabling Compression Level Index Settings for ZSTD and ZSTD Codecs #8558

sarthakaggarwal97 commented Jul 9, 2023 •

edited

github-actions bot commented Jul 9, 2023

github-actions bot commented Jul 9, 2023

github-actions bot commented Jul 9, 2023

codecov bot commented Jul 9, 2023 •

edited

github-actions bot commented Jul 9, 2023

github-actions bot commented Jul 9, 2023

github-actions bot commented Jul 10, 2023

github-actions bot commented Jul 10, 2023

github-actions bot commented Jul 10, 2023

github-actions bot commented Jul 10, 2023

github-actions bot commented Jul 10, 2023

github-actions bot commented Jul 10, 2023

github-actions bot commented Jul 10, 2023

github-actions bot commented Jul 10, 2023

reta commented Jul 10, 2023

[Backport] [2.x] Enabling Compression Level Index Settings for ZSTD and ZSTD Codecs #8558

[Backport] [2.x] Enabling Compression Level Index Settings for ZSTD and ZSTD Codecs #8558

Conversation

sarthakaggarwal97 commented Jul 9, 2023 • edited

Description

Related Issues

Check List

github-actions bot commented Jul 9, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Jul 9, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Jul 9, 2023

Gradle Check (Jenkins) Run Completed with:

codecov bot commented Jul 9, 2023 • edited

Codecov Report

github-actions bot commented Jul 9, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Jul 9, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Jul 10, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Jul 10, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Jul 10, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Jul 10, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Jul 10, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Jul 10, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Jul 10, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Jul 10, 2023

Gradle Check (Jenkins) Run Completed with:

reta commented Jul 10, 2023

sarthakaggarwal97 commented Jul 9, 2023 •

edited

codecov bot commented Jul 9, 2023 •

edited