[TEST] Fix Unit and Integration Tests as part of Removal of XPack code #23

adnapibar · 2021-02-01T18:45:42Z

This meta issue tracks fixing all the tests which are failing as part of the removal of references of Elastic licensed code (CR #2)

@adnapibar Compile Issues (Fix compilation issues for tests. #29)
Build Tools Integration Tests (Gradle Task :build-tools:integTest)

REST High Level Client Tests (Gradle Task :client:rest-high-level:test)
Tests with failures:

org.elasticsearch.client.RequestConvertersTests.testExists (Revert back refresh policy in RequestConverters. #55)
org.elasticsearch.client.RequestConvertersTests.testGetSource (Revert back refresh policy in RequestConverters. #55)
org.elasticsearch.client.RequestConvertersTests.testMultiGet (Revert back refresh policy in RequestConverters. #55)
org.elasticsearch.client.RequestConvertersTests.testSourceExistsWithType (Revert back refresh policy in RequestConverters. #55)
org.elasticsearch.client.indices.GetDataStreamResponseTests.testFromXContent

REST High Level Client Integ Tests (Gradle Task :client:rest-high-level:asyncIntegTest)
Tests with failures:

org.elasticsearch.client.IndicesClientIT.testFreezeAndUnfreeze(Removing FreezeIndex related code since its x-pack counterpart is removed #85)
org.elasticsearch.client.IndicesClientIT.testReloadAnalyzer(Removing ReloadSearchAnalyzers related code since its x-pack counterpart is removed #48)
org.elasticsearch.client.ReindexIT.testDeleteByQuery
org.elasticsearch.client.analytics.AnalyticsAggsIT.testTopMetricsDoubleMetric(Remove x-pack aggregations hlrc. #59)
org.elasticsearch.client.analytics.AnalyticsAggsIT.testStringStats (Remove x-pack aggregations hlrc. #59)
org.elasticsearch.client.analytics.AnalyticsAggsIT.testTopMetricsSizeTwo (Remove x-pack aggregations hlrc. #59)
org.elasticsearch.client.analytics.AnalyticsAggsIT.testTopMetricsLongMetric (Remove x-pack aggregations hlrc. #59)
org.elasticsearch.client.analytics.AnalyticsAggsIT.testTopMetricsDateMetric (Remove x-pack aggregations hlrc. #59)
org.elasticsearch.client.analytics.AnalyticsAggsIT.testTopMetricsManyMetrics (Remove x-pack aggregations hlrc. #59)
org.elasticsearch.client.UpdateByQueryIT.testUpdateByQuery
org.elasticsearch.client.documentation.IndicesClientDocumentationIT.testFreezeIndex(Removing FreezeIndex related code since its x-pack counterpart is removed #85)
org.elasticsearch.client.documentation.IndicesClientDocumentationIT.testUnfreezeIndex(Removing FreezeIndex related code since its x-pack counterpart is removed #85)
org.elasticsearch.client.documentation.IndicesClientDocumentationIT.testReloadSearchAnalyzers(Removing ReloadSearchAnalyzers related code since its x-pack counterpart is removed #48)
org.elasticsearch.client.documentation.CRUDDocumentationIT.testGetSource
org.elasticsearch.client.SearchIT.testSearchWithBasicLicensedQuery (IT Failure "testSearchWithBasicLicensedQuery" #60)

The text was updated successfully, but these errors were encountered:

tlfeng · 2021-02-02T06:20:38Z

verification tasks (gradle check) pass in the below sub-projects:

benchmarks (./gradlew :benchmarks:check)
example-plugins (./gradlew :example-plugins:xxx:check)
libs (./gradlew :libs:xxx:check)
modules (./gradlew :modules:xxx:check)

tlfeng · 2021-02-02T23:19:40Z

There is one test failed in "Build Tools Integration Tests" [Update: It was fixed by PR #33]

> Task :build-tools:integTest

org.elasticsearch.gradle.internal.InternalDistributionArchiveCheckPluginFuncTest > fails on unexpected notice content FAILED
    org.spockframework.runtime.ConditionNotSatisfiedError at InternalDistributionArchiveCheckPluginFuncTest.groovy:100

67 tests completed, 1 failed

To fix the test "fails on unexpected notice content" in InternalDistributionArchiveCheckPluginFuncTest.groovy,
The product name in this line, which is "OSS Search" now:
https://github.com/opendistro-for-elasticsearch/search/blob/d496925a32385f30d0e4e6a546a2848bf1b0b83f/buildSrc/src/integTest/groovy/org/elasticsearch/gradle/internal/InternalDistributionArchiveCheckPluginFuncTest.groovy#L82
needs to be same with here, which is "Elasticsearch" now:
https://github.com/opendistro-for-elasticsearch/search/blob/d496925a32385f30d0e4e6a546a2848bf1b0b83f/buildSrc/src/main/java/org/elasticsearch/gradle/internal/InternalDistributionArchiveCheckPlugin.java#L93

Note:
Setting JAVA14_HOME environment variable is a prerequisite of the tests, according to:
https://github.com/elastic/elasticsearch/issues/ 63303 , and the contribution guide

Without setting JAVA14_HOME, there will be 3 other tests failure:

org.elasticsearch.gradle.internal.InternalDistributionBwcSetupPluginFuncTest > builds distribution from branches via archives assemble FAILED
    org.gradle.testkit.runner.UnexpectedBuildFailure at InternalDistributionBwcSetupPluginFuncTest.groovy:51

org.elasticsearch.gradle.internal.InternalDistributionBwcSetupPluginFuncTest > bwc distribution archives can be resolved as bwc project artifact FAILED
    org.gradle.testkit.runner.UnexpectedBuildFailure at InternalDistributionBwcSetupPluginFuncTest.groovy:87

org.elasticsearch.gradle.internal.InternalDistributionBwcSetupPluginFuncTest > bwc expanded distribution folder can be resolved as bwc project artifact FAILED
    org.gradle.testkit.runner.UnexpectedBuildFailure at InternalDistributionBwcSetupPluginFuncTest.groovy:125

tlfeng · 2021-02-03T06:30:29Z

There are two permanent tests failure in "REST Hight Level Client Tests", which are not failed in Elasticsearch 7.10.2

> Task :client:rest-high-level:test

Tests with failures:
 - org.elasticsearch.client.RestHighLevelClientTests.testProvidedNamedXContents
 - org.elasticsearch.client.indices.GetDataStreamResponseTests.testFromXContent
 - 
265 tests completed, 2 failed

The test testProvidedNamedXContents is fixed by PR [TEST] Fix unit test failure in RestHighLevelClientTests #36
The test testFromXContent:
There is an error mesage [data_stream] unknown field [ilm_policy], it is due to ILM policy field definition missing in DataStream.java, the original PR of adding the definition is: https://github.com/elastic/elasticsearch/pull/ 59128
If adding the field definition back, like the commit dcf00b6, the failure will be disappear.
But currently whether to leave of remove the whole DataStreams codes needs to be determined.

Besides, there are several flaky tests:

Tests with failures:
 - org.elasticsearch.client.RequestConvertersTests.testSourceExistsWithType
 - org.elasticsearch.client.RequestConvertersTests.testGetSource
 - org.elasticsearch.client.RequestConvertersTests.testMultiGet
 - org.elasticsearch.client.RequestConvertersTests.testSourceExists

These are caused by the removal of the "RefreshPolicy" in this commit, and are addressed by PR #42

tlfeng · 2021-02-04T06:07:50Z

15 test failures in Gradle Task :client:rest-high-level:asyncIntegTest, and the tests pass in Elasticsearch v7.10.2

Example:

REPRODUCE WITH: ./gradlew ':client:rest-high-level:asyncIntegTest' --tests "org.elasticsearch.client.UpdateByQueryIT.testUpdateByQuery" -Dtests.seed=36DBCD1668F2A29 -Dtests.security.manager=true -Dtests.locale=no -Dtests.timezone=EAT -Druntime.java=15

org.elasticsearch.client.UpdateByQueryIT > testUpdateByQuery FAILED
    java.lang.AssertionError: expected:<2> but was:<1>
        at __randomizedtesting.SeedInfo.seed([36DBCD1668F2A29:F93DCA21C2AB256F]:0)
        at org.junit.Assert.fail(Assert.java:88)
        at org.junit.Assert.failNotEquals(Assert.java:834)
        at org.junit.Assert.assertEquals(Assert.java:645)
        at org.junit.Assert.assertEquals(Assert.java:631)
        at org.elasticsearch.client.UpdateByQueryIT.testUpdateByQuery(UpdateByQueryIT.java:105)

Full errors list are omitted.

saratvemulapalli · 2021-02-04T21:49:59Z

For 2. Build Tools Integration Tests (Gradle Task :build-tools:integTest)
I have see all the tests passing.

vemsarat@3c22fb88f11c odfe-search % ./gradlew :build-tools:integTest
Elasticsearch Build Hamster says Hello!
  Gradle Version        : 6.6.1
  OS Info               : Mac OS X 10.15.7 (x86_64)
  JDK Version           : 14 (Oracle JDK)
  JAVA_HOME             : /Library/Java/JavaVirtualMachines/jdk-14.0.2.jdk/Contents/Home
  Random Testing Seed   : AA1DDF5D1C3EC860
  In FIPS 140 mode      : false

BUILD SUCCESSFUL in 3m 54s
18 actionable tasks: 4 executed, 14 up-to-date

saratvemulapalli · 2021-02-04T23:42:42Z

For 3. REST Hight Level Client Tests (Gradle Task :client:rest-high-level:test)
There is a single test which is failing org.elasticsearch.client.indices.GetDataStreamResponseTests.testFromXContent which is probably related to DataStreams issue. #40

adnapibar · 2021-02-05T00:06:16Z

For 3. REST Hight Level Client Tests (Gradle Task :client:rest-high-level:test)
There is a single test which is failing org.elasticsearch.client.indices.GetDataStreamResponseTests.testFromXContent which is probably related to DataStreams issue. #40

You also need to run ./gradlew :client:rest-high-level:asyncIntegTest (updated the description)

nknize · 2021-02-12T00:17:08Z

Closing issue as all test issues are complete! 🎉

adnapibar added :xpack-removal Related to removal of x-pack Meta Meta issue, not directly linked to a PR :test Adding or fixing a test labels Feb 1, 2021

opensearch-project deleted a comment from ftianli-amzn Feb 2, 2021

nknize mentioned this issue Feb 3, 2021

[PURIFY] Cleanup build and unblock HLRC integration tests #33

Merged

This was referenced Feb 3, 2021

[TEST] Fix unit test failure in RestHighLevelClientTests #36

Merged

[TEST] fix RequestConvertersTests failure #42

Closed

adnapibar changed the title ~~Fix Unit and Integration Tests as part of Removal of XPack code~~ [TEST] Fix Unit and Integration Tests as part of Removal of XPack code Feb 4, 2021

adnapibar added the >test-failure Test failure from CI, local build, etc. label Feb 4, 2021

This was referenced Feb 5, 2021

Remove ReloadSearchAnalyzers #47

Closed

Removing ReloadSearchAnalyzers related code since its x-pack counterpart is removed #48

Merged

adnapibar mentioned this issue Feb 5, 2021

[TEST] AnalyticsAggsIT #51

Closed

saratvemulapalli mentioned this issue Feb 5, 2021

Fixing Rest Converters Tests after x-pack removal #54

Merged

setiah added this to To do in beta release Feb 8, 2021

setiah moved this from To do to Backlog in beta release Feb 8, 2021

setiah mentioned this issue Feb 8, 2021

[TEST] FreezeUnfreezeIT #57

Closed

adnapibar moved this from Backlog to In progress in beta release Feb 8, 2021

adnapibar mentioned this issue Feb 9, 2021

Remove ILM policy from GetDataStreamAction Response. #63

Merged

abbashus mentioned this issue Feb 11, 2021

Removing FreezeIndex related code since its x-pack counterpart is removed #85

Merged

nknize closed this as completed Feb 12, 2021

beta release automation moved this from In progress to Done Feb 12, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[TEST] Fix Unit and Integration Tests as part of Removal of XPack code #23

[TEST] Fix Unit and Integration Tests as part of Removal of XPack code #23

adnapibar commented Feb 1, 2021 •

edited by nknize

tlfeng commented Feb 2, 2021 •

edited

tlfeng commented Feb 2, 2021 •

edited

tlfeng commented Feb 3, 2021 •

edited

tlfeng commented Feb 4, 2021 •

edited

saratvemulapalli commented Feb 4, 2021 •

edited

saratvemulapalli commented Feb 4, 2021

adnapibar commented Feb 5, 2021

nknize commented Feb 12, 2021

[TEST] Fix Unit and Integration Tests as part of Removal of XPack code #23

[TEST] Fix Unit and Integration Tests as part of Removal of XPack code #23

Comments

adnapibar commented Feb 1, 2021 • edited by nknize

tlfeng commented Feb 2, 2021 • edited

tlfeng commented Feb 2, 2021 • edited

tlfeng commented Feb 3, 2021 • edited

tlfeng commented Feb 4, 2021 • edited

saratvemulapalli commented Feb 4, 2021 • edited

saratvemulapalli commented Feb 4, 2021

adnapibar commented Feb 5, 2021

nknize commented Feb 12, 2021

adnapibar commented Feb 1, 2021 •

edited by nknize

tlfeng commented Feb 2, 2021 •

edited

tlfeng commented Feb 2, 2021 •

edited

tlfeng commented Feb 3, 2021 •

edited

tlfeng commented Feb 4, 2021 •

edited

saratvemulapalli commented Feb 4, 2021 •

edited