Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update pr-validation job and tycho scm parameters #2

Merged
merged 18 commits into from
Apr 11, 2022
Merged

Update pr-validation job and tycho scm parameters #2

merged 18 commits into from
Apr 11, 2022

Conversation

sravanlakkimsetti
Copy link
Member

Fixes #1

@sravanlakkimsetti sravanlakkimsetti changed the title Update pr-validation job abd tycho scm parameters Update pr-validation job and tycho scm parameters Apr 9, 2022
Jenkinsfile Show resolved Hide resolved
@iloveeclipse
Copy link
Member

@sravanlakkimsetti : last gerrit failed wit OOM. Any idea what was default on Jenkins and what is default right now? Should we explicitly specify max heap?

@sravanlakkimsetti
Copy link
Member Author

sravanlakkimsetti commented Apr 9, 2022

@sravanlakkimsetti : last gerrit failed wit OOM. Any idea what was default on Jenkins and what is default right now? Should we explicitly specify max heap?

As far as I remember default was 2GB. That should be sufficient. I am unsetting JAVA_TOOL_OPTIONS and _JAVA_OPTIONS. These parameters are set by default in our okd cluster. We had issues with the defaults passed to these in the past. specifically with heap settings.

In our gerrit job we were unsetting these params. lets see next build.

@iloveeclipse
Copy link
Member

OOM again. Total was 1 GB if I read it right.

@sravanlakkimsetti
Copy link
Member Author

OOM again. Total was 1 GB if I read it right.

Added 4G heap now. Lets see

@iloveeclipse
Copy link
Member

iloveeclipse commented Apr 9, 2022

@sravanlakkimsetti : I see my jenkins build started by gerrit is also failing with OOM and other obscure error:

https://git.eclipse.org/r/c/jdt/eclipse.jdt.core/+/185619

https://ci.eclipse.org/jdt/job/eclipse.jdt.core-Gerrit/6045/artifact/org.eclipse.jdt.core.tests.model/target/work/data/.metadata/.log

!ENTRY org.eclipse.jdt.core 1 0 2022-04-09 16:59:21.972
!MESSAGE SETUP test0411

!ENTRY org.eclipse.swt.cocoa.macosx.x86_64 2 0 2022-04-09 16:59:25.810
!MESSAGE Could not resolve module: org.eclipse.swt.cocoa.macosx.x86_64 [125]
  Unresolved requirement: Require-Capability: eclipse.platform; filter:="(& (osgi.ws=cocoa) (osgi.os=macosx) (osgi.arch=x86_64) )"
  Unresolved requirement: Require-Capability: eclipse.platform; filter:="(& (osgi.ws=cocoa) (osgi.os=macosx) (osgi.arch=x86_64) )"


!ENTRY org.eclipse.swt.gtk.linux.aarch64 2 0 2022-04-09 16:59:25.810
!MESSAGE Could not resolve module: org.eclipse.swt.gtk.linux.aarch64 [123]
  Unresolved requirement: Require-Capability: eclipse.platform; filter:="(& (osgi.ws=gtk) (osgi.os=linux) (osgi.arch=aarch64))"
  Unresolved requirement: Require-Capability: eclipse.platform; filter:="(& (osgi.ws=gtk) (osgi.os=linux) (osgi.arch=aarch64))"


!ENTRY org.eclipse.swt.cocoa.macosx.aarch64 2 0 2022-04-09 16:59:25.811
!MESSAGE Could not resolve module: org.eclipse.swt.cocoa.macosx.aarch64 [126]
  Unresolved requirement: Require-Capability: eclipse.platform; filter:="(& (osgi.ws=cocoa) (osgi.os=macosx) (osgi.arch=aarch64) )"
  Unresolved requirement: Require-Capability: eclipse.platform; filter:="(& (osgi.ws=cocoa) (osgi.os=macosx) (osgi.arch=aarch64) )"


!ENTRY org.eclipse.swt.gtk.linux.ppc64le 2 0 2022-04-09 16:59:25.811
!MESSAGE Could not resolve module: org.eclipse.swt.gtk.linux.ppc64le [122]
  Unresolved requirement: Require-Capability: eclipse.platform; filter:="(& (osgi.ws=gtk) (osgi.os=linux) (osgi.arch=ppc64le))"
  Unresolved requirement: Require-Capability: eclipse.platform; filter:="(& (osgi.ws=gtk) (osgi.os=linux) (osgi.arch=ppc64le))"


!ENTRY org.eclipse.swt.win32.win32.x86_64 2 0 2022-04-09 16:59:25.811
!MESSAGE Could not resolve module: org.eclipse.swt.win32.win32.x86_64 [124]
  Unresolved requirement: Require-Capability: eclipse.platform; filter:="(& (osgi.ws=win32) (osgi.os=win32) (osgi.arch=x86_64))"
  Unresolved requirement: Require-Capability: eclipse.platform; filter:="(& (osgi.ws=win32) (osgi.os=win32) (osgi.arch=x86_64))"


!ENTRY org.eclipse.osgi 4 0 2022-04-09 16:59:25.811
!MESSAGE Application error
!STACK 1
java.lang.reflect.InvocationTargetException
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.apache.maven.surefire.api.util.ReflectionUtils.invokeMethodWithArray2(ReflectionUtils.java:183)
	at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:161)
	at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
	at org.eclipse.tycho.surefire.osgibooter.OsgiSurefireBooter.run(OsgiSurefireBooter.java:128)
	at org.eclipse.tycho.surefire.osgibooter.HeadlessTestApplication.start(HeadlessTestApplication.java:28)
	at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:203)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:136)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:104)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:402)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:255)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:659)
	at org.eclipse.equinox.launcher.Main.basicRun(Main.java:596)
	at org.eclipse.equinox.launcher.Main.run(Main.java:1467)
	at org.eclipse.equinox.launcher.Main.main(Main.java:1440)
Caused by: java.lang.OutOfMemoryError: Java heap space
Root exception:
java.lang.OutOfMemoryError: Java heap space

May be we have issue with last platform build?

@sravanlakkimsetti
Copy link
Member Author

sravanlakkimsetti commented Apr 9, 2022

@sravanlakkimsetti : I see my jenkins build started by gerrit is also failing with OOM and other obscure error:

https://git.eclipse.org/r/c/jdt/eclipse.jdt.core/+/185619

https://ci.eclipse.org/jdt/job/eclipse.jdt.core-Gerrit/6045/artifact/org.eclipse.jdt.core.tests.model/target/work/data/.metadata/.log

!ENTRY org.eclipse.jdt.core 1 0 2022-04-09 16:59:21.972
!MESSAGE SETUP test0411

!ENTRY org.eclipse.swt.cocoa.macosx.x86_64 2 0 2022-04-09 16:59:25.810
!MESSAGE Could not resolve module: org.eclipse.swt.cocoa.macosx.x86_64 [125]
  Unresolved requirement: Require-Capability: eclipse.platform; filter:="(& (osgi.ws=cocoa) (osgi.os=macosx) (osgi.arch=x86_64) )"
  Unresolved requirement: Require-Capability: eclipse.platform; filter:="(& (osgi.ws=cocoa) (osgi.os=macosx) (osgi.arch=x86_64) )"


!ENTRY org.eclipse.swt.gtk.linux.aarch64 2 0 2022-04-09 16:59:25.810
!MESSAGE Could not resolve module: org.eclipse.swt.gtk.linux.aarch64 [123]
  Unresolved requirement: Require-Capability: eclipse.platform; filter:="(& (osgi.ws=gtk) (osgi.os=linux) (osgi.arch=aarch64))"
  Unresolved requirement: Require-Capability: eclipse.platform; filter:="(& (osgi.ws=gtk) (osgi.os=linux) (osgi.arch=aarch64))"


!ENTRY org.eclipse.swt.cocoa.macosx.aarch64 2 0 2022-04-09 16:59:25.811
!MESSAGE Could not resolve module: org.eclipse.swt.cocoa.macosx.aarch64 [126]
  Unresolved requirement: Require-Capability: eclipse.platform; filter:="(& (osgi.ws=cocoa) (osgi.os=macosx) (osgi.arch=aarch64) )"
  Unresolved requirement: Require-Capability: eclipse.platform; filter:="(& (osgi.ws=cocoa) (osgi.os=macosx) (osgi.arch=aarch64) )"


!ENTRY org.eclipse.swt.gtk.linux.ppc64le 2 0 2022-04-09 16:59:25.811
!MESSAGE Could not resolve module: org.eclipse.swt.gtk.linux.ppc64le [122]
  Unresolved requirement: Require-Capability: eclipse.platform; filter:="(& (osgi.ws=gtk) (osgi.os=linux) (osgi.arch=ppc64le))"
  Unresolved requirement: Require-Capability: eclipse.platform; filter:="(& (osgi.ws=gtk) (osgi.os=linux) (osgi.arch=ppc64le))"


!ENTRY org.eclipse.swt.win32.win32.x86_64 2 0 2022-04-09 16:59:25.811
!MESSAGE Could not resolve module: org.eclipse.swt.win32.win32.x86_64 [124]
  Unresolved requirement: Require-Capability: eclipse.platform; filter:="(& (osgi.ws=win32) (osgi.os=win32) (osgi.arch=x86_64))"
  Unresolved requirement: Require-Capability: eclipse.platform; filter:="(& (osgi.ws=win32) (osgi.os=win32) (osgi.arch=x86_64))"


!ENTRY org.eclipse.osgi 4 0 2022-04-09 16:59:25.811
!MESSAGE Application error
!STACK 1
java.lang.reflect.InvocationTargetException
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.apache.maven.surefire.api.util.ReflectionUtils.invokeMethodWithArray2(ReflectionUtils.java:183)
	at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:161)
	at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
	at org.eclipse.tycho.surefire.osgibooter.OsgiSurefireBooter.run(OsgiSurefireBooter.java:128)
	at org.eclipse.tycho.surefire.osgibooter.HeadlessTestApplication.start(HeadlessTestApplication.java:28)
	at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:203)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:136)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:104)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:402)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:255)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:659)
	at org.eclipse.equinox.launcher.Main.basicRun(Main.java:596)
	at org.eclipse.equinox.launcher.Main.run(Main.java:1467)
	at org.eclipse.equinox.launcher.Main.main(Main.java:1440)
Caused by: java.lang.OutOfMemoryError: Java heap space
Root exception:
java.lang.OutOfMemoryError: Java heap space

May be we have issue with last platform build?

You may be right here. I gave explicit 4GB heap still having this problem. I cannot go more than this as the container has only 4GB of space.

Lets try after next platform build.

Any other suggestion are welcome.

@iloveeclipse
Copy link
Member

Or is the jenkins instance itself has issues?
@sravanlakkimsetti : can you restart jenkins for jdt?

https://ci.eclipse.org/jdt/job/eclipse.jdt.core-Github/job/PR-2/11/console

org.eclipse.jdt.core.tests.model.JavaSearchBugsTests#testBug266582()
19:27:33  Exception in thread "Java indexing" java.lang.OutOfMemoryError: Java heap space
19:27:33  	at org.eclipse.jdt.internal.compiler.util.HashtableOfObject.<init>(HashtableOfObject.java:56)
19:27:33  	at org.eclipse.jdt.internal.compiler.util.HashtableOfObject.rehash(HashtableOfObject.java:184)
19:27:33  	at org.eclipse.jdt.internal.compiler.util.HashtableOfObject.putUnsafely(HashtableOfObject.java:157)
19:27:33  	at org.eclipse.jdt.internal.core.index.DiskIndex.mergeCategory(DiskIndex.java:508)
19:27:33  	at org.eclipse.jdt.internal.core.index.DiskIndex.mergeCategories(DiskIndex.java:477)
19:27:33  	at org.eclipse.jdt.internal.core.index.DiskIndex.mergeWith(DiskIndex.java:579)
19:27:33  	at org.eclipse.jdt.internal.core.index.Index.save(Index.java:230)
19:27:33  	at org.eclipse.jdt.internal.core.index.MetaIndex.save(MetaIndex.java:100)
19:27:33  	at org.eclipse.jdt.internal.core.search.indexing.IndexManager.saveMetaIndex(IndexManager.java:1256)
19:27:33  	at org.eclipse.jdt.internal.core.search.indexing.IndexManager.saveIndexes(IndexManager.java:1232)
19:27:33  	at org.eclipse.jdt.internal.core.search.indexing.IndexManager.notifyIdle(IndexManager.java:826)
19:27:33  	at org.eclipse.jdt.internal.core.search.processing.JobManager.run(JobManager.java:425)
19:27:33  	at java.base/java.lang.Thread.run(Thread.java:833)

@sravanlakkimsetti
Copy link
Member Author

Or is the jenkins instance itself has issues? @sravanlakkimsetti : can you restart jenkins for jdt?

https://ci.eclipse.org/jdt/job/eclipse.jdt.core-Github/job/PR-2/11/console

org.eclipse.jdt.core.tests.model.JavaSearchBugsTests#testBug266582()
19:27:33  Exception in thread "Java indexing" java.lang.OutOfMemoryError: Java heap space
19:27:33  	at org.eclipse.jdt.internal.compiler.util.HashtableOfObject.<init>(HashtableOfObject.java:56)
19:27:33  	at org.eclipse.jdt.internal.compiler.util.HashtableOfObject.rehash(HashtableOfObject.java:184)
19:27:33  	at org.eclipse.jdt.internal.compiler.util.HashtableOfObject.putUnsafely(HashtableOfObject.java:157)
19:27:33  	at org.eclipse.jdt.internal.core.index.DiskIndex.mergeCategory(DiskIndex.java:508)
19:27:33  	at org.eclipse.jdt.internal.core.index.DiskIndex.mergeCategories(DiskIndex.java:477)
19:27:33  	at org.eclipse.jdt.internal.core.index.DiskIndex.mergeWith(DiskIndex.java:579)
19:27:33  	at org.eclipse.jdt.internal.core.index.Index.save(Index.java:230)
19:27:33  	at org.eclipse.jdt.internal.core.index.MetaIndex.save(MetaIndex.java:100)
19:27:33  	at org.eclipse.jdt.internal.core.search.indexing.IndexManager.saveMetaIndex(IndexManager.java:1256)
19:27:33  	at org.eclipse.jdt.internal.core.search.indexing.IndexManager.saveIndexes(IndexManager.java:1232)
19:27:33  	at org.eclipse.jdt.internal.core.search.indexing.IndexManager.notifyIdle(IndexManager.java:826)
19:27:33  	at org.eclipse.jdt.internal.core.search.processing.JobManager.run(JobManager.java:425)
19:27:33  	at java.base/java.lang.Thread.run(Thread.java:833)

I don't have permissions to restart. Opened Restart JDT JIPP with foundation.

@iloveeclipse
Copy link
Member

Tests fail also independently from this PR on master. That must be something with Jenkins. I would unset timeout back to 1 hour, typically JDT tests are done in 30 minutes. Eaiting longer just wastes our time and Jenkins resources.

@stephan-herrmann
Copy link
Contributor

You may be right here. I gave explicit 4GB heap still having this problem. I cannot go more than this as the container has only 4GB of space.

I would be surprised if these tests need more than 2GB. They shouldn't.

Verifying the size on mount points. Need to check the size of tmp
@sravanlakkimsetti
Copy link
Member Author

You may be right here. I gave explicit 4GB heap still having this problem. I cannot go more than this as the container has only 4GB of space.

I would be surprised if these tests need more than 2GB. They shouldn't.

I am wondering this could be problem with /tmp path. I see that most of these tests create project in /tmp path. Is there a way to force creation of test projects at some other place?

@iloveeclipse
Copy link
Member

I am wondering this could be problem with /tmp path

If so, it's Jenkins problem, not our issue. If /tmp is broken, lot of things will go wrong not only in tests.
We need to wait for Jenkins restart. If that not helps, we have another problem with JDT infrastructure.
May be some container change or new installed package etc.

using older tycho version
@sravanlakkimsetti
Copy link
Member Author

Trying with Tycho 2.7.0 now

Removed explicit tycho version
using tycho-version 2.7.0 for jdt validation
@iloveeclipse
Copy link
Member

Just for the record: created tycho ticket eclipse-tycho/tycho#879


unset JAVA_TOOL_OPTIONS
unset _JAVA_OPTIONS
MAVEN_OPTS="-Xmx4G"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wasn't / shouldn't that be 2G ?

@sravanlakkimsetti sravanlakkimsetti deleted the pr-validation branch April 11, 2022 12:44
iloveeclipse added a commit to iloveeclipse/eclipse.jdt.core that referenced this pull request Apr 11, 2022
iloveeclipse added a commit to iloveeclipse/eclipse.jdt.core that referenced this pull request Apr 11, 2022
iloveeclipse added a commit to iloveeclipse/eclipse.jdt.core that referenced this pull request Apr 11, 2022
iloveeclipse added a commit to iloveeclipse/eclipse.jdt.core that referenced this pull request Apr 11, 2022
- log memory total/used/free on starting every suite
- increased max heap for model tests to 2GB
- print system properties / environment once for started JVM

See eclipse-jdt#2 troubles and eclipse-tycho/tycho#879
iloveeclipse added a commit to iloveeclipse/eclipse.jdt.core that referenced this pull request Apr 11, 2022
- log memory total/used/free on starting every suite
- increased max heap for model tests to 2GB
- print system properties / environment once for started JVM

See eclipse-jdt#2 troubles and eclipse-tycho/tycho#879
iloveeclipse added a commit to iloveeclipse/eclipse.jdt.core that referenced this pull request Apr 11, 2022
- log memory total/used/free on starting every suite
- increased max heap for model tests to 2GB
- print system properties / environment once for started JVM

See eclipse-jdt#2 troubles and eclipse-tycho/tycho#879
iloveeclipse added a commit to iloveeclipse/eclipse.jdt.core that referenced this pull request Apr 11, 2022
- log memory total/used/free on starting every suite
- increased max heap for model tests to 2GB
- print system properties / environment once for started JVM

See eclipse-jdt#2 troubles and eclipse-tycho/tycho#879
@iloveeclipse
Copy link
Member

For the record, if one properly sets max heap to 2 GB for tycho, OOM errors disappear (should be investigated in eclipse-tycho/tycho#879) but then new test fail appear: #12.

iloveeclipse added a commit to iloveeclipse/eclipse.jdt.core that referenced this pull request Apr 11, 2022
- log memory total/used/free on starting every suite
- increased max heap for model tests to 2GB
- print system properties / environment once for started JVM

See eclipse-jdt#2 troubles and eclipse-tycho/tycho#879
iloveeclipse added a commit to iloveeclipse/eclipse.jdt.core that referenced this pull request Apr 11, 2022
- log memory total/used/free on starting every model test suite
- set explicit max heap for model tests to 1 GB (to identify memory
leaks, they need max 0.5 GB)
- print system properties / environment once for started JVM

See eclipse-jdt#2 troubles and eclipse-tycho/tycho#879
iloveeclipse added a commit that referenced this pull request Apr 11, 2022
- log memory total/used/free on starting every model test suite
- set explicit max heap for model tests to 1 GB (to identify memory
leaks, they need max 0.5 GB)
- print system properties / environment once for started JVM

See #2 troubles and eclipse-tycho/tycho#879
merks pushed a commit to merks/eclipse.jdt.core that referenced this pull request Jan 11, 2023
* Create Jenkinsfile

* Update Jenkinsfile
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Update pr-validation job and tycho scm parameters
5 participants