Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

initial investigation to create a travis.ci build #25

Merged
merged 1 commit into from Sep 18, 2017

Conversation

DanHeidinga
Copy link
Member

Initial investigation to get a travis.ci build up and running as per #24.

Commits will be squashed and cleaned up before merging. PR is just so others can follow along (or make suggestions on) the approach being taken.

@mgaudet
Copy link
Contributor

mgaudet commented Sep 15, 2017

Regarding the re-clone thing -- If you know what dir Travis has cloned into, i have previously successfully used get_sources.sh with a local directory; it means two copies on disk, yes, but the clone is nearly instantaneous.

@DanHeidinga
Copy link
Member Author

@mgaudet Travis supplies the build directory in one of the env vars:

TRAVIS_BUILD_DIR: The absolute path to the directory where the repository being built has been copied on the worker.

Do you have a command line you can share that shows the local clone?

@mgaudet
Copy link
Contributor

mgaudet commented Sep 15, 2017

So, I wasn't working on travis, so this is a small shot in the dark, but I was thinking this

 bash ./get_source.sh -openj9-repo=$TRAVIS_BUILD_DIR -openj9-branch=$TRAVIS_BRANCH -openj9-sha=$TRAVIS_COMMIT

Assuming travis doesn't do something silly on the clone (like nuke the .git dir), it should be fine to just provide the filesystem path as the argument to get_sources.sh

@DanHeidinga
Copy link
Member Author

Thanks @mgaudet! That worked much better than my earlier attempts to craft a file:// url.

@DanHeidinga
Copy link
Member Author

Reclone now works. Next challenge: boot idk not being recognized.

@DanHeidinga
Copy link
Member Author

Boot JDK is recognized and supported now.
Had to disable ccache due to the openjdk configure script rejecting the wrapped compiler.

Current status: build failed due to g++ internal compiler error. Relaunching

@DanHeidinga
Copy link
Member Author

First successful compile attempt in the travis env using 1 core for both the configure and for the make commands.
Next step: try to increase that and see if the builds are faster while still passing.

@DanHeidinga
Copy link
Member Author

Complete build and 'java -version' runs in ~30 min on travis.ci without ccache enabled.

The initial travis yml is based off the Eclipse OMR .travis.yml and the
the OpenJ9 docker file.  This does the necessary apt-gets and wgets to
create the build environment.

The build runs get_sources.sh to set up the 3 required repos.  As
Travis has already cloned the openj9 repo locally, we pass that
copy of the repo's location, sha, and branch to the get_sources.sh
script.

This results in a duplicate copy of the openj9 repo on disk but avoids
network traffic - overall a win.

The build ran into some strange issues:

* the bootjdk detection fails unless the '_JAVA_OPTIONS=' env var is
unset

* the compiles fail with a compiler internal error unless the ojdk
JOBS are limited and the `UMA_WINDOWS_PARRALLEL_HACK` is used to
control the `-jN`.  We can't directly set `-jN` in the MAKE env as
the OJDK build scripts fail when they detect it.  Using -j4 as the
travis container build env is 2 core / 4 gig memory.

* ccache is also disabled as the ojdk build scripts don't support
using a wrapped compiler.  They generate the following error:
   configure: error: /usr/lib/ccache/gcc is a symbolic link to ccache.
              This is not supported.
   configure exiting with result code 1

TODOs:
* Re-enable ccache
* Add testing to this build

Signed-off-by: Dan Heidinga <daniel_heidinga@ca.ibm.com>
@DanHeidinga DanHeidinga changed the title WIP: initial investigation to create a travis.ci build initial investigation to create a travis.ci build Sep 18, 2017
@DanHeidinga
Copy link
Member Author

I think this is ready to go as an initial travis build. It only ensures the code compiles and can run -version but we can look at enabling testing (and ccache) in separate PRs.

@DanHeidinga
Copy link
Member Author

@mgaudet Can you review this and call out any issues? Once its approved, the one of the other committers can also review and merge.

# Based on https://blog.travis-ci.com/2014-12-17-faster-builds-with-container-based-infrastructure/ travis container builds
# have 2 cores and 4 gigs of memory. Attempt to double provision the number of cores for the make...
# Limit number of jobs to work around g++ internal compiler error
- export UMA_WINDOWS_PARRALLEL_HACK="-j4"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the hack actually misspelled? (Two Rs)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sadly, that matches what's in the tr.source makefiles: https://github.com/eclipse/openj9/blob/master/runtime/tr.source/makefile.ftl

(edited to add: and in the rest of our code... see https://github.com/eclipse/openj9/search?utf8=✓&q=UMA_WINDOWS_PARRALLEL_HACK&type=)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

facepalm

@mgaudet
Copy link
Contributor

mgaudet commented Sep 18, 2017

To be clear, in the final merged version you do, or do not want ccache? (I know the OpenJDK build scripts demand --with-ccache )

Also, what's the distinction between make all and make images?

@DanHeidinga
Copy link
Member Author

ccache isn't enabled yet. When ccache is enabled on travis, it wraps the gcc/g++ executables and the ojdk scripts reject that. I suspect 3-ish commands to specify the unwrapped gcc/g++ and enable ccache should work but wanted to land this first.

@mgaudet
Copy link
Contributor

mgaudet commented Sep 18, 2017

Ok -- with that in mind, I think this is fine to land as is, with the potential to iterate

👍 :shipit:

@DanHeidinga
Copy link
Member Author

Can one of the committers approve / merge this? @charliegracie @pshipton others?

@pshipton pshipton merged commit 54876e9 into eclipse-openj9:master Sep 18, 2017
@DanHeidinga DanHeidinga deleted the djh/travis branch September 18, 2017 19:45
SueChaplain added a commit to SueChaplain/openj9 that referenced this pull request Nov 1, 2017
Add markdown file that covers the build of OpenjDK9
with OpenJ9 on other platforms as well as some
extra pointers for new users of Docker.

The AIX install section includes a reference to the yum_install_aix-ppc64.txt
file, which contains a list of SW pre-reqs for AIX 64-bit. This
text file can be used as a recipe, in a similar way to a Dockerfile.

[skip-ci]

Issue raised on the Eclipse/openj9-website repo, number eclipse-openj9#25

issue eclipse-openj9#25 (github.com/eclipse/openj9-website)

Signed-off-by: Sue Chaplain <sue_chaplain@uk.ibm.com>
jonoommen pushed a commit to jonoommen/openj9 that referenced this pull request Nov 2, 2017
Add markdown file that covers the build of OpenjDK9
with OpenJ9 on other platforms as well as some
extra pointers for new users of Docker.

The AIX install section includes a reference to the yum_install_aix-ppc64.txt
file, which contains a list of SW pre-reqs for AIX 64-bit. This
text file can be used as a recipe, in a similar way to a Dockerfile.

[skip-ci]

Issue raised on the Eclipse/openj9-website repo, number eclipse-openj9#25

issue eclipse-openj9#25 (github.com/eclipse/openj9-website)

Signed-off-by: Sue Chaplain <sue_chaplain@uk.ibm.com>
SueChaplain added a commit to SueChaplain/openj9 that referenced this pull request Nov 20, 2017
Add markdown file for building OpenJDK8 with
OpenJ9 on Linux and AIX systems.

Add the yum text file that we reference in the
AIX build instructions.

Issue at github.com/eclipse-openj9/openj9-website/issues/25

[ci skip]

Issue eclipse-openj9#25

Signed-off-by: Sue Chaplain <sue_chaplain@uk.ibm.com>
AlenBadel pushed a commit to AlenBadel/openj9 that referenced this pull request Nov 22, 2017
Add markdown file for building OpenJDK8 with
OpenJ9 on Linux and AIX systems.

Add the yum text file that we reference in the
AIX build instructions.

Issue at github.com/eclipse-openj9/openj9-website/issues/25

[ci skip]

Issue eclipse-openj9#25

Signed-off-by: Sue Chaplain <sue_chaplain@uk.ibm.com>
JamesKingdon pushed a commit to JamesKingdon/openj9 that referenced this pull request Apr 18, 2019
Add markdown file that covers the build of OpenjDK9
with OpenJ9 on other platforms as well as some
extra pointers for new users of Docker.

The AIX install section includes a reference to the yum_install_aix-ppc64.txt
file, which contains a list of SW pre-reqs for AIX 64-bit. This
text file can be used as a recipe, in a similar way to a Dockerfile.

[skip-ci]

Issue raised on the Eclipse/openj9-website repo, number eclipse-openj9#25

issue eclipse-openj9#25 (github.com/eclipse/openj9-website)

Signed-off-by: Sue Chaplain <sue_chaplain@uk.ibm.com>
tajila pushed a commit to tajila/openj9 that referenced this pull request Jul 11, 2019
* Updated the writing options

* Added base functionality for teardown

* Teardown is now called on JVM shutdown

* Changed where teardown is called, also fixed a crashing issue when not using our feature

* Missed macro

* Removed unused variables and code cleanup

* Removed function from internalFunctions array

Signed-off-by: Brady Jessup <brady.jessup@ibm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants