Build of QA: Smoke Test Shaded Jar fails under maven 3.3.x #12791
Assignees
Labels
>bug
:Delivery/Build
Build or test infrastructure
Team:Delivery
Meta label for Delivery team
>test
Issues or PRs that are addressing/adding tests
Comments
imotov
added
>bug
>test
|
I can confirm and reproduce the issue. Funny thing: when you do that again from root dir but "resume from" smoke-test-shaded module, you don't hit the issue...
|
|
I found a way to quickly reproduce the issue. It makes obvious that when shaded is added to the reactor, it fails then smoke tests. |
|
So the difference between both execution plan is obviously the The wrong one when you run The right one: Trying to find a fix now... |
|
Adding here some notes on how to quickly debug that (might help for the future). Just run: mvn dependency:tree -pl :smoke-test-shadedand mvn dependency:tree -pl org.elasticsearch.distribution.shaded:elasticsearch,:smoke-test-shaded |
dadoonet
added a commit
to dadoonet/elasticsearch
that referenced
this issue
Aug 13, 2015
Build fails with maven 3.3.1 and 3.3.3. To reproduce, install one of the 3.3.x versions of maven and run `mvn clean verify` in the root directory of the project. The build will fail in the QA: Smoke Test Shaded Jar module with the following error: ``` Started J0 PID(99979@flea.local). Suite: org.elasticsearch.shaded.test.ShadedIT 2> NOTE: reproduce with: ant test -Dtestcase=ShadedIT -Dtests.method=testJodaIsNotOnTheCP -Dtests.seed=2F4D23A7462CF921 -Dtests.locale= -Dtests.timezone=Asia/Baku -Dtests.asserts=true -Dtests.file.encoding=UTF-8 FAILURE 0.06s | ShadedIT.testJodaIsNotOnTheCP <<< > Throwable #1: junit.framework.AssertionFailedError: Expected an exception but the test passed: java.lang.ClassNotFoundException > at __randomizedtesting.SeedInfo.seed([2F4D23A7462CF921:3A9404F1F69FD80]:0) > at junit.framework.Assert.fail(Assert.java:57) > at java.lang.Thread.run(Thread.java:745) 2> NOTE: reproduce with: ant test -Dtestcase=ShadedIT -Dtests.method=testGuavaIsNotOnTheCP -Dtests.seed=2F4D23A7462CF921 -Dtests.locale= -Dtests.timezone=Asia/Baku -Dtests.asserts=true -Dtests.file.encoding=UTF-8 FAILURE 0.01s | ShadedIT.testGuavaIsNotOnTheCP <<< > Throwable #1: junit.framework.AssertionFailedError: Expected an exception but the test passed: java.lang.ClassNotFoundException > at __randomizedtesting.SeedInfo.seed([2F4D23A7462CF921:C2502FD54D83433D]:0) > at junit.framework.Assert.fail(Assert.java:57) > at java.lang.Thread.run(Thread.java:745) 2> NOTE: reproduce with: ant test -Dtestcase=ShadedIT -Dtests.method=testjsr166eIsNotOnTheCP -Dtests.seed=2F4D23A7462CF921 -Dtests.locale= -Dtests.timezone=Asia/Baku -Dtests.asserts=true -Dtests.file.encoding=UTF-8 FAILURE 0.01s | ShadedIT.testjsr166eIsNotOnTheCP <<< > Throwable #1: junit.framework.AssertionFailedError: Expected an exception but the test passed: java.lang.ClassNotFoundException > at __randomizedtesting.SeedInfo.seed([2F4D23A7462CF921:35593286F4269392]:0) > at junit.framework.Assert.fail(Assert.java:57) > at java.lang.Thread.run(Thread.java:745) 2> NOTE: leaving temporary files on disk at: /Users/Shared/Jenkins/Home/workspace/elasticsearch-master/qa/smoke-test-shaded/target/J0/temp/org.elasticsearch.shaded.test.ShadedIT_2F4D23A7462CF921-001 2> NOTE: test params are: codec=CheapBastard, sim=DefaultSimilarity, locale=, timezone=Asia/Baku 2> NOTE: Mac OS X 10.10.4 x86_64/Oracle Corporation 1.8.0_25 (64-bit)/cpus=8,threads=1,free=482137936,total=514850816 2> NOTE: All tests run in this JVM: [ShadedIT] Completed [1/1] in 6.61s, 5 tests, 3 failures <<< FAILURES! Tests with failures: - org.elasticsearch.shaded.test.ShadedIT.testJodaIsNotOnTheCP - org.elasticsearch.shaded.test.ShadedIT.testGuavaIsNotOnTheCP - org.elasticsearch.shaded.test.ShadedIT.testjsr166eIsNotOnTheCP ``` Please note that build doesn't fail with maven 3.2.x and it doesn't fail if mvn command is executed inside the qa/smoke-test-shaded directory. Only when the build is started from the root directory the error above can be observed. The reason is because of the shaded version which depends on elasticsearch core. When Maven build the module only, then elasticsearch core is not added to the dependency tree. ```sh mvn dependency:tree -pl :smoke-test-shaded ``` ``` [INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ smoke-test-shaded --- [INFO] org.elasticsearch.qa:smoke-test-shaded:jar:2.0.0-beta1-SNAPSHOT [INFO] +- org.elasticsearch.distribution.shaded:elasticsearch:jar:2.0.0-beta1-SNAPSHOT:compile [INFO] | +- org.apache.lucene:lucene-core:jar:5.2.1:compile [INFO] | +- org.apache.lucene:lucene-backward-codecs:jar:5.2.1:compile [INFO] | +- org.apache.lucene:lucene-analyzers-common:jar:5.2.1:compile [INFO] | +- org.apache.lucene:lucene-queries:jar:5.2.1:compile [INFO] | +- org.apache.lucene:lucene-memory:jar:5.2.1:compile [INFO] | +- org.apache.lucene:lucene-highlighter:jar:5.2.1:compile [INFO] | +- org.apache.lucene:lucene-queryparser:jar:5.2.1:compile [INFO] | +- org.apache.lucene:lucene-sandbox:jar:5.2.1:compile [INFO] | +- org.apache.lucene:lucene-suggest:jar:5.2.1:compile [INFO] | +- org.apache.lucene:lucene-misc:jar:5.2.1:compile [INFO] | +- org.apache.lucene:lucene-join:jar:5.2.1:compile [INFO] | +- org.apache.lucene:lucene-grouping:jar:5.2.1:compile [INFO] | +- org.apache.lucene:lucene-spatial:jar:5.2.1:compile [INFO] | \- com.spatial4j:spatial4j:jar:0.4.1:compile [INFO] +- org.hamcrest:hamcrest-all:jar:1.3:test [INFO] \- org.apache.lucene:lucene-test-framework:jar:5.2.1:test [INFO] +- org.apache.lucene:lucene-codecs:jar:5.2.1:test [INFO] +- com.carrotsearch.randomizedtesting:randomizedtesting-runner:jar:2.1.16:test [INFO] +- junit:junit:jar:4.11:test [INFO] \- org.apache.ant:ant:jar:1.8.2:test ``` But if shaded plugin is involved during the build, it modifies the `projectArtifactMap`: ```sh mvn dependency:tree -pl org.elasticsearch.distribution.shaded:elasticsearch,:smoke-test-shaded ``` ``` [INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ smoke-test-shaded --- [INFO] org.elasticsearch.qa:smoke-test-shaded:jar:2.0.0-beta1-SNAPSHOT [INFO] +- org.elasticsearch.distribution.shaded:elasticsearch:jar:2.0.0-beta1-SNAPSHOT:compile [INFO] | \- org.elasticsearch:elasticsearch:jar:2.0.0-beta1-SNAPSHOT:compile [INFO] | +- org.apache.lucene:lucene-backward-codecs:jar:5.2.1:compile [INFO] | +- org.apache.lucene:lucene-analyzers-common:jar:5.2.1:compile [INFO] | +- org.apache.lucene:lucene-queries:jar:5.2.1:compile [INFO] | +- org.apache.lucene:lucene-memory:jar:5.2.1:compile [INFO] | +- org.apache.lucene:lucene-highlighter:jar:5.2.1:compile [INFO] | +- org.apache.lucene:lucene-queryparser:jar:5.2.1:compile [INFO] | | \- org.apache.lucene:lucene-sandbox:jar:5.2.1:compile [INFO] | +- org.apache.lucene:lucene-suggest:jar:5.2.1:compile [INFO] | | \- org.apache.lucene:lucene-misc:jar:5.2.1:compile [INFO] | +- org.apache.lucene:lucene-join:jar:5.2.1:compile [INFO] | | \- org.apache.lucene:lucene-grouping:jar:5.2.1:compile [INFO] | +- org.apache.lucene:lucene-spatial:jar:5.2.1:compile [INFO] | | \- com.spatial4j:spatial4j:jar:0.4.1:compile [INFO] | +- com.google.guava:guava:jar:18.0:compile [INFO] | +- com.carrotsearch:hppc:jar:0.7.1:compile [INFO] | +- joda-time:joda-time:jar:2.8:compile [INFO] | +- org.joda:joda-convert:jar:1.2:compile [INFO] | +- com.fasterxml.jackson.core:jackson-core:jar:2.5.3:compile [INFO] | +- com.fasterxml.jackson.dataformat:jackson-dataformat-smile:jar:2.5.3:compile [INFO] | +- com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:jar:2.5.3:compile [INFO] | | \- org.yaml:snakeyaml:jar:1.12:compile [INFO] | +- com.fasterxml.jackson.dataformat:jackson-dataformat-cbor:jar:2.5.3:compile [INFO] | +- io.netty:netty:jar:3.10.3.Final:compile [INFO] | +- com.ning:compress-lzf:jar:1.0.2:compile [INFO] | +- com.tdunning:t-digest:jar:3.0:compile [INFO] | +- org.hdrhistogram:HdrHistogram:jar:2.1.6:compile [INFO] | +- org.apache.commons:commons-lang3:jar:3.3.2:compile [INFO] | +- commons-cli:commons-cli:jar:1.3.1:compile [INFO] | \- com.twitter:jsr166e:jar:1.1.0:compile [INFO] +- org.hamcrest:hamcrest-all:jar:1.3:test [INFO] \- org.apache.lucene:lucene-test-framework:jar:5.2.1:test [INFO] +- org.apache.lucene:lucene-codecs:jar:5.2.1:test [INFO] +- org.apache.lucene:lucene-core:jar:5.2.1:compile [INFO] +- com.carrotsearch.randomizedtesting:randomizedtesting-runner:jar:2.1.16:test [INFO] +- junit:junit:jar:4.11:test [INFO] \- org.apache.ant:ant:jar:1.8.2:test ``` A fix could consist of fixing something on Maven side. Probably something changed in a recent version and introduced this "issue" but it might be not really an issue. More a fix. There are two workarounds: 1) exclude manually elasticsearch core from shaded version in smoke-test-shaded module and add manually each lucene lib needed by elasticsearch 2) add a new `elasticsearch-lucene` (lucene) POM module which simply declares all needed lucene libs in subprojects (such as the smoke tester one). I choose the later. Closes elastic#12791.
|
Reopening as the change has been reverted |
HarishAtGitHub
pushed a commit
to HarishAtGitHub/elasticsearch
that referenced
this issue
Aug 18, 2015
issue url : elastic#12790 issue id : 12790 Problem Statement: =================== When creating index, we are now able to give the version created as input in the rest api content. But it is not supposed to be fed by users. But at the same time we cannot remove it completely as it is used internally as pointed out by @rjernst in elastic#12790 (comment) and elastic#12790 (comment). So we should find a way that only avoids elasticsearch api users from exploiting this. Approach: ========= We should not allow users to use this feature. so one way would be to filter the params given by the user before it is structured and set in CreateIndexRequest. This is like "Nip it in the Bud" . Will this affect other internal workings that are dependent on this version.created ? No. because by doing it in the CreateIndexRequest we are only avoding every one who hits the top most interface leaving the inner side untouched. Code: ==== modified: core/src/main/java/org/elasticsearch/action/admin/indices/create/CreateIndexRequest.java The filtering is done after the request.content() is processed and before it is set in the request.settings (as once set we cannot do anything as it is immutable). This is the last and safe place to filter as we get input in right form to filter. This is specific to IndexCreation thats why it is done here and only here. And this is the place where all request.content() processing is done whenever create index is requested. Did this work ? ============== Here are the observations: Manual Test: ************ I tested it with 5 possible settings as follows : ```javascript settings1 = {'settings': { 'index': { 'number_of_replicas': '1', 'version': {'created': '2000009'}, 'creation_date': '1439384094544', 'uuid': 'ayzHgwB3Sgey-Pk_Okgwyg', 'number_of_shards': '5' }, 'version': {'created': '2000010'}, 'creation_date': '1439384094544', 'uuid': 'ayzHgwB3Sgey-Pk_Okgwyg', 'number_of_shards': '3' } } settings2 = {'settings': { 'index': { 'number_of_replicas': '1', 'version': {'created': '2000009'}, 'creation_date': '1439384094544', 'uuid': 'ayzHgwB3Sgey-Pk_Okgwyg', 'number_of_shards': '5' } } } settings3 = {'settings': { 'version': {'created': '2000009'}, 'creation_date': '1439384094544', 'uuid': 'ayzHgwB3Sgey-Pk_Okgwyg', 'number_of_shards': '3' } } settings4 = {} settings5 = { 'settings' : {}} settings6 = { 'settings' : {'index': {}}} ``` and it seems to ignore the version.created whether it is given within index or outside index. So it works. Automated Test: *************** I built and all tests PASSED except the "Smoke test shaded jar" see : https://gist.github.com/HarishAtGitHub/aa8539a5c56cb4ad48b9 . and I hope this has nothing to do with my code. and suspecting some other problem I resumed the build and I landed in https://gist.github.com/HarishAtGitHub/fcc519bd9863ab8a95af which is reported in elastic#12791 .
|
We removed the shaded module so it's not an issue anymore... :p Closing |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Build fails with maven 3.3.1 and 3.3.3. To reproduce, install one of the 3.3.x versions of maven and run
mvn clean verifyin the root directory of the project. The build will fail in the QA: Smoke Test Shaded Jar module with the following error:Please note that build doesn't fail with maven 3.2.x and it doesn't fail if mvn command is executed inside the qa/smoke-test-sharded directory. Only when the build is started from the root directory the error above can be observed.
The text was updated successfully, but these errors were encountered: