Re-enable tests in CI (Eclipse Jenkins) #1166

javierron · 2021-01-04T12:16:16Z

I'm hijacking @andre15silva's PR #1164 to attempt some fixes myself. Any progress will be rebased to his PR.

Signed-off-by: André Silva <andre15andre@hotmail.com>

Signed-off-by: Javier Ron <javierron90@gmail.com>

andre15silva · 2021-01-04T12:40:00Z

Just to keep you up to date then, the current problems are:

The pipeline stage works on the local docker image, but on the Jenkins CI it gives those errors.
maven-repair has a problem, when running locally in the docker container, with spoon-core jar file not being available in the local repository. I'm trying to reproduce that in Jenkins.
Yesterday maven-repair hit an infinite loop. I'm not sure what is causing it. I'm adding a timeout so that doesn't happen again.

javierron · 2021-01-04T12:46:15Z

Just to keep you up to date then, the current problems are:

1. The `pipeline` stage works on the local docker image, but on the Jenkins CI it gives those errors.

2. `maven-repair` has a problem, when running locally in the docker container, with `spoon-core` jar file not being available in the local repository. I'm trying to reproduce that in Jenkins.

3. Yesterday `maven-repair` hit an infinite loop. I'm not sure what is causing it. I'm adding a timeout so that doesn't happen again.

Cool! thanks for the info! I am focusing on point 3 for now. I believe the problem might be here:

repairnator/src/maven-repair/src/test/java/com/github/tdurieux/repair/maven/plugin/NPEfixMojoTest.java

Line 14 in fc73f4a

mvn_clean_test.waitFor();

My guess is that the mvn clean test command is failing and the tests waits forever. I'll keep you updated.

andre15silva · 2021-01-04T12:48:17Z

Could be. When running on the local docker container the only problem is with the Nopol test not finding the spoon-core jar, so should be related to the environment in Jenkins.

javierron · 2021-01-04T20:25:11Z

Solved 2 & 3 (log). I will attempt point 1 tomorrow and hopefully the modification on the Jenkinsfile will fix some of the problems of the pipeline. @andre15silva

Add explicit dependecies on maven-reapiar pom to avoid conflicts and missing jars on tests Signed-off-by: Javier Ron <javierron90@gmail.com>

javierron · 2021-01-05T13:46:57Z

This PR is passing the tests now.
@andre15silva are there any new changes on your branch? If not, we can skip the rebase and merge this PR instead.

andre15silva · 2021-01-05T14:44:32Z

Jenkinsfile

+        container('maven') {
+          sh 'bash ./.ci/ci-maven-repair.sh'
        }


Is there really a need to use a separate script for maven-repair? It would be nice to merge this with run-with-core. The only difference is the NPEFix version option.

If we change this we can eliminate xmlstarlet from the Docker image too.

I'm not really sure of what does that version property does, have you tried to run the tests without it?

Yes, and they work. I also don't know why that is needed. I haven't found any usage of a property NPEFIX_VERSION.

@cokefh @monperrus WDYT?

andre15silva · 2021-01-05T14:46:01Z

src/maven-repair/pom.xml

+        <dependency>
+            <groupId>fr.inria.gforge.spoon</groupId>
+            <artifactId>spoon-core</artifactId>
+            <version>7.0.0</version>
+        </dependency>
+        <dependency>
+            <groupId>commons-cli</groupId>
+            <artifactId>commons-cli</artifactId>
+            <version>1.3</version>
+        </dependency>
+        <dependency>
+            <groupId>org.json</groupId>
+            <artifactId>json</artifactId>
+            <version>20160810</version>
+        </dependency>


I'm not sure this is the best approach, albeit it works. Shouldn't the dependencies be fetched transitively?

I'm not sure this is the best approach

I agree with you. At least in the case of spoon-core I could see that the dependency metadata (pom and sha files) is downloaded, but unless I put these here explicitly it does not download the jar. I assumed the same for commons-cli and json.

Also, I checked and the maven-repair tests work locally in my pc only because those jars were already there as a result of compiling another module.

If there is a better way to trigger the jar download, we can replace this.

Yes, it's very weird behavior indeed. While debugging I also saw that pom and sha files were downloaded, just not the jar file.

I think it's important to get this right tho, because having sub-dependencies hardcoded makes it hard to maintain.

I will look further into it after my exam tomorrow.

mvn dependency:tree for maven-repair is in https://gist.github.com/andre15silva/254c0de8ef2d2a7ac45c834f404119c4

The issue is that these three dependencies are being omitted because of conflicts with sub-dependencies from other repair tools that use these libraries but with a different version.

[INFO] +- (fr.inria.gforge.spoon:spoon-core:jar:7.0.0:compile - omitted for conflict with 6.0.0) [INFO] +- (commons-cli:commons-cli:jar:1.3:compile - omitted for conflict with 1.2) [INFO] \- (org.json:json:jar:20160810:compile - omitted for conflict with 20160212)

If we added the Nopol dependency first, it would fetch these versions. It's also not a very elegant solution tho. Yours is probably better, as it is explicitly written.

Another option would be to change getClassPathFromPom to take into account that the version explicited in the repair tools pom files may not be available locally and that another version has to be used. I'm not sure this is a very good idea tho.

@javierron WDYT?

If we added the Nopol dependency first, it would fetch these versions. It's also not a very elegant solution tho. Yours is probably better, as it is explicitly written.

I believe I tried this, and it did not resolve the conflicts.

Another option would be to change getClassPathFromPom to take into account that the version explicited in the repair tools pom files may not be available locally and that another version has to be used.

I don't think this is a good idea either, we may create an even more complicated compatibility/dependency issue.

andre15silva · 2021-01-05T14:49:36Z

This PR is passing the tests now.
@andre15silva are there any new changes on your branch? If not, we can skip the rebase and merge this PR instead.

The only changes were the timeouts, but you have already included that here. I added some comments, tell me what you think.

Other than that, we can skip the rebase and merge this yes.

monperrus · 2021-01-13T09:10:41Z

Thanks a lot for the heavy work. LGTM to merge and proceed. We can fine-tune later.

OK for you?

javierron · 2021-01-13T10:52:47Z

OK for you?

OK for me

andre15silva · 2021-01-13T10:57:55Z

OK for you?

OK for me

OK for me too

monperrus · 2021-01-13T11:45:53Z

Thanks a lot for the foundational PR

andre15silva and others added 20 commits December 22, 2020 10:52

Trigger CI

51d864d

Signed-off-by: André Silva <andre15andre@hotmail.com>

Fix M2_HOME

cc345cf

Signed-off-by: André Silva <andre15andre@hotmail.com>

Add libgomp1 to docker image

3ec2282

Signed-off-by: André Silva <andre15andre@hotmail.com>

Add missing parameter registration

e6559c2

Signed-off-by: André Silva <andre15andre@hotmail.com>

Fix build arg not being specified

30828f0

Signed-off-by: André Silva <andre15andre@hotmail.com>

Add tmp folder as workspace

7068eea

Signed-off-by: André Silva <andre15andre@hotmail.com>

Propagate tmp dir change

fdcd945

Signed-off-by: André Silva <andre15andre@hotmail.com>

Move args test to new class

3441a05

Signed-off-by: André Silva <andre15andre@hotmail.com>

Remove imports

ff6e908

Signed-off-by: André Silva <andre15andre@hotmail.com>

Add clean up before args test

1738c2b

Signed-off-by: André Silva <andre15andre@hotmail.com>

Add launcher with args

9814a21

Signed-off-by: André Silva <andre15andre@hotmail.com>

Add temporary output folder to tests

caa999c

Signed-off-by: André Silva <andre15andre@hotmail.com>

Fix null pointer exception

007346b

Signed-off-by: André Silva <andre15andre@hotmail.com>

Cache dependencies in ci docker image

c9d0a9b

Signed-off-by: André Silva <andre15andre@hotmail.com>

Revert cache

14e4ded

Signed-off-by: André Silva <andre15andre@hotmail.com>

Fix GitRepositoryLauncher not initializing specific config parameters

152950f

Signed-off-by: André Silva <andre15andre@hotmail.com>

Fix leaky tests

ab509b0

Signed-off-by: André Silva <andre15andre@hotmail.com>

Move default ODSPath from home dir to output dir

e35cdae

Signed-off-by: André Silva <andre15andre@hotmail.com>

Disable pipeline to test realtime

86ea07a

Signed-off-by: André Silva <andre15andre@hotmail.com>

more memory to ci container

e4f5d47

Signed-off-by: Javier Ron <javierron90@gmail.com>

javierron force-pushed the ci-fix branch 7 times, most recently from f189028 to 7b9bc08 Compare January 4, 2021 15:54

javierron force-pushed the ci-fix branch 12 times, most recently from 9a180a5 to f001b0d Compare January 4, 2021 20:19

javierron force-pushed the ci-fix branch 2 times, most recently from c85daf2 to 91977a1 Compare January 5, 2021 12:11

Add user.home config to Jenkinsfile

61a9bbf

Add explicit dependecies on maven-reapiar pom to avoid conflicts and missing jars on tests Signed-off-by: Javier Ron <javierron90@gmail.com>

javierron force-pushed the ci-fix branch from 91977a1 to 61a9bbf Compare January 5, 2021 12:27

andre15silva reviewed Jan 5, 2021

View reviewed changes

javierron marked this pull request as ready for review January 6, 2021 14:44

javierron requested a review from monperrus January 6, 2021 14:45

monperrus changed the title ~~Enable tests in CI~~ Re-enable tests in CI (Eclipse Jenkins) Jan 13, 2021

monperrus merged commit 7e4637a into eclipse:master Jan 13, 2021

monperrus mentioned this pull request Jan 13, 2021

Get a proper CI with all functions #1152

Closed

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Re-enable tests in CI (Eclipse Jenkins) #1166

Re-enable tests in CI (Eclipse Jenkins) #1166

javierron commented Jan 4, 2021

andre15silva commented Jan 4, 2021

javierron commented Jan 4, 2021

andre15silva commented Jan 4, 2021

javierron commented Jan 4, 2021 •

edited

javierron commented Jan 5, 2021

andre15silva Jan 5, 2021

javierron Jan 5, 2021

andre15silva Jan 5, 2021

javierron Jan 6, 2021

andre15silva Jan 5, 2021

javierron Jan 5, 2021

andre15silva Jan 6, 2021

andre15silva Jan 8, 2021 •

edited

javierron Jan 13, 2021

andre15silva commented Jan 5, 2021

monperrus commented Jan 13, 2021

javierron commented Jan 13, 2021

andre15silva commented Jan 13, 2021 •

edited

monperrus commented Jan 13, 2021

Re-enable tests in CI (Eclipse Jenkins) #1166

Re-enable tests in CI (Eclipse Jenkins) #1166

Conversation

javierron commented Jan 4, 2021

andre15silva commented Jan 4, 2021

javierron commented Jan 4, 2021

andre15silva commented Jan 4, 2021

javierron commented Jan 4, 2021 • edited

javierron commented Jan 5, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

andre15silva Jan 8, 2021 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

andre15silva commented Jan 5, 2021

monperrus commented Jan 13, 2021

javierron commented Jan 13, 2021

andre15silva commented Jan 13, 2021 • edited

monperrus commented Jan 13, 2021

javierron commented Jan 4, 2021 •

edited

andre15silva Jan 8, 2021 •

edited

andre15silva commented Jan 13, 2021 •

edited