Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

graalvm: rc8->rc15 #59371

Merged
merged 1 commit into from May 9, 2019

Conversation

Projects
None yet
5 participants
@hlolli
Copy link
Contributor

commented Apr 12, 2019

Motivation for this change

graal is constantly changeing, and the mx build tool is becoming more important and complex. It took me a week to get this working. Few things off the bat, I use openjdk instead of oraclejdk, reason is simply that the newest openjdk works with graalvm-1.0.0rc15 but oraclejdk does not. So that means I should change the license to GPL-2.0?

To build graalvm, I used heavily patched mx I named it mx-internal. This patched mx file is probably so patched that it's not unlikely that its intended default behavior is affected (building into home directories could be useful for example), so the vanilla mx command should work like documented.

Things done

Bumped mx, jvmci8 and graalvm. Patched mx and graalvm when needed. Added graaljs and intend to add fastr, pythongraal and more polyglot packages. Bumped all the jar dependencies by hand, this all took quite some time, kudos to @volth for their initial work. Like with previous graal nix-expression, this does not produce Sulong's lli binary, I'm now looking into how to get that working.

  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Assured whether relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@NeQuissimus

This comment has been minimized.

Copy link
Member

commented Apr 12, 2019

Nice! This looks good to me, I currently cannot test this though. Thank you for the effort. I would change the license.

mkdir -p ${out}/${name}
cp -rf ${fetchgit { preferLocalBuild = false;
deepClone = true;
leaveDotGit = true;

This comment has been minimized.

Copy link
@volth

volth Apr 12, 2019

Contributor

leaveDotGit = true; is not fully deterministic, the sha256 on another machine might vary.
that's why here I created fake hg repos to fool mx

This comment has been minimized.

Copy link
@hlolli

hlolli Apr 12, 2019

Author Contributor

Ah good catch, I had it there before I totally patched the revision check mechanism from mx out, I'll remove this, shouldn't affect anything.

This comment has been minimized.

Copy link
@hlolli

hlolli Apr 14, 2019

Author Contributor

Fixed

@hlolli hlolli force-pushed the hlolli:graal-bump branch from 0401e2b to 860d4d9 Apr 14, 2019

@hlolli

This comment has been minimized.

Copy link
Contributor Author

commented Apr 14, 2019

So I've changed the license and the review from @volth, still a good reason to look over this again. I've simplified the install process. There was a folder with a super long name starting with graalvm sitting in mxbuild directory that has the directory setup all correct, plus it includes all the binaries (native-image, lli, js etc). I made an attempt to add fastr, truffleruby and pythongraal polyglots, but for now only succeded with python. Trying to compile fastr is out of my league, as it's using java via truffle-ffi to make a sharedlibfile (this case libR.so) and run it. I've run out of time trying to solve it, maybe truffleruby is easier, but I really need to do other stuff for the next 1 1/2 week. I'll paste here below the commands that I didn't include in this derivation, but were needed to get the build process further. (probably a memo to my future self, but please anyone reading this, have a go at this)

    # # TRUFFLERUBY # #
    # patch truffleruby home-dir path
    substituteInPlace ${out}/truffleruby/src/processor/java/org/truffleruby/processor/BuildInformationProcessor.java \
      --replace "trufflerubyHome = findHome();" "trufflerubyHome = new File(\"${out}/truffleruby\");"

    # patch shebangs in truffleruby
    chmod +x ${out}/truffleruby/bin/*
    chmod +x ${out}/truffleruby/tool/{*.rb,*.sh}
    patchShebangs ${out}/truffleruby/bin/
    patchShebangs ${out}/truffleruby/tool/

    # # FASTR # #
    # patch some buggy path in fastr
    substituteInPlace ${out}/fastr/com.oracle.truffle.r.parser.processor/src/com/oracle/truffle/r/parser/processor/GenerateRParserProcessor.java \
      --replace 'join(parserSrcDir, "R.g").getAbsolutePath()' \
                "\"${out}/fastr/com.oracle.truffle.r.parser/src/com/oracle/truffle/r/parser/R.g\""

    # copy R over to the fastr libdownloads
    mkdir -p ${out}/fastr/libdownloads
    cp ${R} ${out}/fastr/libdownloads/R-3.5.1.tar.gz

    # BUG: https://github.com/oracle/fastr/issues/69
    substituteInPlace ${out}/fastr/com.oracle.truffle.r.parser/src/com/oracle/truffle/r/parser/R.g \
      --replace "'α'..'ω'|'Α'..'Ω'|" ""

    # Let's make the R compilation print the stdout/stderr and patch R sources with same patch as the R derivation
    substituteInPlace ${out}/fastr/com.oracle.truffle.r.native/gnur/Makefile.gnur \
      --replace "> gnur_configure.log 2>&1" "" \
      --replace "> gnur_make.log 2>&1" ""
  
    substituteInPlace ${out}/fastr/com.oracle.truffle.r.native/gnur/Makefile.libs \
      --replace "mx -p" "PKG_LDFLAGS_OVERRIDE='-L${pcre.out}/lib -L${gfortran.cc.lib}/lib -L${zlib}/lib' mx-internal -p" \

    substituteInPlace ${out}/fastr/mx.fastr/mx_copylib.py \
      --replace "not os.path.exists(os.path.join(target_dir, f))" \
                "not os.path.exists(os.path.join(target_dir, f)) and not target.endswith('libgcc_s.so')"

    # make R.sh use mx-internal and patch its shebangs
    substituteInPlace ${out}/fastr/com.oracle.truffle.r.native/run/R.sh \
      --replace '$mx' 'mx-internal'
    chmod +x ${out}/fastr/com.oracle.truffle.r.native/run/R.sh
    patchShebangs ${out}/fastr/com.oracle.truffle.r.native/run

@hlolli hlolli force-pushed the hlolli:graal-bump branch from 860d4d9 to 93795a9 Apr 14, 2019

@GrahamcOfBorg GrahamcOfBorg requested a review from NeQuissimus Apr 14, 2019

@NeQuissimus NeQuissimus merged commit 3cf07ef into NixOS:master May 9, 2019

11 checks passed

grahamcofborg-eval ^.^!
Details
grahamcofborg-eval-check-maintainers matching changed paths to changed attrs...
Details
grahamcofborg-eval-check-meta config.nix: checkMeta = true
Details
grahamcofborg-eval-darwin nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A darwin-tested
Details
grahamcofborg-eval-nixos-manual nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./nixos/release.nix -A manual
Details
grahamcofborg-eval-nixos-options nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./nixos/release.nix -A options
Details
grahamcofborg-eval-nixpkgs-manual nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A manual
Details
grahamcofborg-eval-nixpkgs-tarball nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A tarball
Details
grahamcofborg-eval-nixpkgs-unstable-jobset nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A unstable
Details
grahamcofborg-eval-package-list nix-env -qa --json --file .
Details
grahamcofborg-eval-package-list-no-aliases nix-env -qa --json --file . --arg config { allowAliases = false; }
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.