Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot compile with openjdk17 and gcc[89] with Intel opencl. #145

Closed
yazun opened this issue Sep 17, 2021 · 17 comments
Closed

Cannot compile with openjdk17 and gcc[89] with Intel opencl. #145

yazun opened this issue Sep 17, 2021 · 17 comments
Assignees
Labels
enhancement New feature or request question Further information is requested

Comments

@yazun
Copy link
Contributor

yazun commented Sep 17, 2021

Describe the bug
Trying to compile on Centos 7 with opencl support on devtoolset-8 and devtoolset-9 :

How To Reproduce

 make jdk-11-plus BACKEND=opencl
...
INFO] --- maven-jar-plugin:3.1.2:jar (default-jar) @ tornado-benchmarks ---
-- Looking for CL_VERSION_2_2 - found
-- Found OpenCL: /usr/lib64/libOpenCL.so (found version "2.2")
[INFO] Building jar: /d00/local/tornado/tornadovm/benchmarks/target/tornado-benchmarks-0.11-dev-c61f388.jar
-- Found JNI: /d00/gaia/jdk17/lib/libjawt.so
-- Configuring done
-- Generating done
-- Build files have been written to: /d00/local/tornado/tornadovm/drivers/opencl-jni/target/linux-amd64-release/cmake
[INFO]
[INFO] --- cmake-maven-plugin:3.7.0-b2:compile (cmake-compile) @ tornado-drivers-opencl-jni ---
[INFO] Configured to use native CMake
gmake[1]: Entering directory '/d00/local/tornado/tornadovm/drivers/opencl-jni/target/linux-amd64-release/cmake'
/d00/local/tornado/cmake-3.10.1/bin/cmake -H/d00/local/tornado/tornadovm/drivers/opencl-jni/target/linux-amd64-release/sources -B/d00/local/tornado/tornadovm/drivers/opencl-jni/target/linux-amd64-release/cmake --check-build-system CMakeFiles/Makefile.cmake 0
/d00/local/tornado/cmake-3.10.1/bin/cmake -E cmake_progress_start /d00/local/tornado/tornadovm/drivers/opencl-jni/target/linux-amd64-release/cmake/CMakeFiles /d00/local/tornado/tornadovm/drivers/opencl-jni/target/linux-amd64-release/cmake/CMakeFiles/progress.marks
/opt/rh/devtoolset-9/root/usr/bin/gmake -f CMakeFiles/Makefile2 all
gmake[2]: Entering directory '/d00/local/tornado/tornadovm/drivers/opencl-jni/target/linux-amd64-release/cmake'
/opt/rh/devtoolset-9/root/usr/bin/gmake -f CMakeFiles/tornado-opencl.dir/build.make CMakeFiles/tornado-opencl.dir/depend
gmake[3]: Entering directory '/d00/local/tornado/tornadovm/drivers/opencl-jni/target/linux-amd64-release/cmake'
cd /d00/local/tornado/tornadovm/drivers/opencl-jni/target/linux-amd64-release/cmake && /d00/local/tornado/cmake-3.10.1/bin/cmake -E cmake_depends "Unix Makefiles" /d00/local/tornado/tornadovm/drivers/opencl-jni/target/linux-amd64-release/sources /d00/local/tornado/tornadovm/drivers/opencl-jni/target/linux-amd64-release/sources /d00/local/tornado/tornadovm/drivers/opencl-jni/target/linux-amd64-release/cmake /d00/local/tornado/tornadovm/drivers/opencl-jni/target/linux-amd64-release/cmake /d00/local/tornado/tornadovm/drivers/opencl-jni/target/linux-amd64-release/cmake/CMakeFiles/tornado-opencl.dir/DependInfo.cmake --color=
Scanning dependencies of target tornado-opencl
gmake[3]: Leaving directory '/d00/local/tornado/tornadovm/drivers/opencl-jni/target/linux-amd64-release/cmake'
/opt/rh/devtoolset-9/root/usr/bin/gmake -f CMakeFiles/tornado-opencl.dir/build.make CMakeFiles/tornado-opencl.dir/build
gmake[3]: Entering directory '/d00/local/tornado/tornadovm/drivers/opencl-jni/target/linux-amd64-release/cmake'
[  9%] Building CXX object CMakeFiles/tornado-opencl.dir/source/OCLCommandQueue.cpp.o
/opt/rh/devtoolset-9/root/usr/bin/c++  -Dtornado_opencl_EXPORTS -I/d00/local/tornado/tornadovm/drivers/opencl-jni/target/linux-amd64-release/sources/source -I/d00/local/tornado/tornadovm/drivers/opencl-jni/target/linux-amd64-release/sources/headers/opencl/3.0/include -I/d00/gaia/jdk17/include -I/d00/gaia/jdk17/include/linux  -O3 -fpermissive -export-dynamic -fPIC  -O3 -DNDEBUG -fPIC   -o CMakeFiles/tornado-opencl.dir/source/OCLCommandQueue.cpp.o -c /d00/local/tornado/tornadovm/drivers/opencl-jni/target/linux-amd64-release/sources/source/OCLCommandQueue.cpp
/d00/local/tornado/tornadovm/drivers/opencl-jni/target/linux-amd64-release/sources/source/OCLCommandQueue.cpp: In function ‘void Java_uk_ac_manchester_tornado_drivers_opencl_OCLCommandQueue_clEnqueueWaitForEvents(JNIEnv*, jclass, jlong, jlongArray)’:
/d00/local/tornado/tornadovm/drivers/opencl-jni/target/linux-amd64-release/sources/source/OCLCommandQueue.cpp:145:97: warning: ‘cl_int clEnqueueWaitForEvents(cl_command_queue, cl_uint, _cl_event* const*)’ is deprecated [-Wdeprecated-declarations]
  145 |     cl_int status = clEnqueueWaitForEvents((cl_command_queue) queue_id, len, (cl_event *) events);
      |                                                                                                 ^
In file included from /d00/local/tornado/tornadovm/drivers/opencl-jni/target/linux-amd64-release/sources/source/OCLCommandQueue.cpp:30:
/d00/local/tornado/tornadovm/drivers/opencl-jni/target/linux-amd64-release/sources/headers/opencl/3.0/include/CL/cl.h:1891:1: note: declared here
 1891 | clEnqueueWaitForEvents(cl_command_queue  command_queue,
      | ^~~~~~~~~~~~~~~~~~~~~~
/d00/local/tornado/tornadovm/drivers/opencl-jni/target/linux-amd64-release/sources/source/OCLCommandQueue.cpp:145:97: warning: ‘cl_int clEnqueueWaitForEvents(cl_command_queue, cl_uint, _cl_event* const*)’ is deprecated [-Wdeprecated-declarations]
  145 |     cl_int status = clEnqueueWaitForEvents((cl_command_queue) queue_id, len, (cl_event *) events);
      |                                                                                                 ^
In file included from /d00/local/tornado/tornadovm/drivers/opencl-jni/target/linux-amd64-release/sources/source/OCLCommandQueue.cpp:30:
/d00/local/tornado/tornadovm/drivers/opencl-jni/target/linux-amd64-release/sources/headers/opencl/3.0/include/CL/cl.h:1891:1: note: declared here
 1891 | clEnqueueWaitForEvents(cl_command_queue  command_queue,
      | ^~~~~~~~~~~~~~~~~~~~~~
[ 18%] Building CXX object CMakeFiles/tornado-opencl.dir/source/OCLContext.cpp.o
/opt/rh/devtoolset-9/root/usr/bin/c++  -Dtornado_opencl_EXPORTS -I/d00/local/tornado/tornadovm/drivers/opencl-jni/target/linux-amd64-release/sources/source -I/d00/local/tornado/tornadovm/drivers/opencl-jni/target/linux-amd64-release/sources/headers/opencl/3.0/include -I/d00/gaia/jdk17/include -I/d00/gaia/jdk17/include/linux  -O3 -fpermissive -export-dynamic -fPIC  -O3 -DNDEBUG -fPIC   -o CMakeFiles/tornado-opencl.dir/source/OCLContext.cpp.o -c /d00/local/tornado/tornadovm/drivers/opencl-jni/target/linux-amd64-release/sources/source/OCLContext.cpp
/d00/local/tornado/tornadovm/drivers/opencl-jni/target/linux-amd64-release/sources/source/OCLContext.cpp:95:7: warning: extra tokens at end of #endif directive [-Wendif-labels]
   95 | #endif()
      |       ^
[ 27%] Building CXX object CMakeFiles/tornado-opencl.dir/source/OCLDevice.cpp.o
/opt/rh/devtoolset-9/root/usr/bin/c++  -Dtornado_opencl_EXPORTS -I/d00/local/tornado/tornadovm/drivers/opencl-jni/target/linux-amd64-release/sources/source -I/d00/local/tornado/tornadovm/drivers/opencl-jni/target/linux-amd64-release/sources/headers/opencl/3.0/include -I/d00/gaia/jdk17/include -I/d00/gaia/jdk17/include/linux  -O3 -fpermissive -export-dynamic -fPIC  -O3 -DNDEBUG -fPIC   -o CMakeFiles/tornado-opencl.dir/source/OCLDevice.cpp.o -c /d00/local/tornado/tornadovm/drivers/opencl-jni/target/linux-amd64-release/sources/source/OCLDevice.cpp
[ 36%] Building CXX object CMakeFiles/tornado-opencl.dir/source/OCLEvent.cpp.o
/opt/rh/devtoolset-9/root/usr/bin/c++  -Dtornado_opencl_EXPORTS -I/d00/local/tornado/tornadovm/drivers/opencl-jni/target/linux-amd64-release/sources/source -I/d00/local/tornado/tornadovm/drivers/opencl-jni/target/linux-amd64-release/sources/headers/opencl/3.0/include -I/d00/gaia/jdk17/include -I/d00/gaia/jdk17/include/linux  -O3 -fpermissive -export-dynamic -fPIC  -O3 -DNDEBUG -fPIC   -o CMakeFiles/tornado-opencl.dir/source/OCLEvent.cpp.o -c /d00/local/tornado/tornadovm/drivers/opencl-jni/target/linux-amd64-release/sources/source/OCLEvent.cpp
[INFO] -------------------------------------------------------------
[WARNING] COMPILATION WARNING :
[INFO] -------------------------------------------------------------
[WARNING] [options] system modules path not set in conjunction with -source 11
[WARNING] [options] --add-opens has no effect at compile time
[WARNING] using incubating module(s): jdk.incubator.foreign,jdk.incubator.vector
[WARNING] [options] module name in --add-exports option not found: tornado.annotation
[WARNING] [options] module name in --add-exports option not found: tornado.drivers.ptx
[WARNING] using incubating module(s): jdk.incubator.foreign,jdk.incubator.vector
[WARNING] using incubating module(s): jdk.incubator.foreign,jdk.incubator.vector
[WARNING] /d00/local/tornado/tornadovm/drivers/opencl/src/main/java/uk/ac/manchester/tornado/drivers/opencl/graal/phases/TornadoTaskSpecialisation.java:[95,18] [deprecation] isAccessible() in AccessibleObject has been deprecated
[INFO] 8 warnings
[INFO] -------------------------------------------------------------
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /d00/local/tornado/tornadovm/drivers/opencl/src/main/java/uk/ac/manchester/tornado/drivers/opencl/graal/OCLArchitecture.java:[45,7] error: OCLArchitecture is not abstract and does not override abstract method getFeatures() in Architecture
[INFO] 1 error
[INFO] -------------------------------------------------------------

...

INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for tornado 0.11-dev:
[INFO]
[INFO] tornado ............................................ SUCCESS [  1.021 s]
[INFO] tornado-api ........................................ SUCCESS [  3.393 s]
[INFO] tornado-runtime .................................... SUCCESS [  2.587 s]
[INFO] tornado-matrices ................................... SUCCESS [  0.985 s]
[INFO] tornado-drivers .................................... SUCCESS [  0.685 s]
[INFO] tornado-drivers-common ............................. SUCCESS [  0.851 s]
[INFO] tornado-drivers-opencl ............................. FAILURE [  2.008 s]
[INFO] tornado-drivers-opencl-jni ......................... SUCCESS [  5.210 s]
[INFO] tornado-examples ................................... SUCCESS [  2.870 s]
[INFO] tornado-benchmarks ................................. SUCCESS [  7.747 s]
[INFO] tornado-unittests .................................. SUCCESS [  3.850 s]
[INFO] tornado-annotation ................................. SUCCESS [  0.819 s]
[INFO] tornado-assembly ................................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  13.291 s (Wall Clock)
[INFO] Finished at: 2021-09-17T18:22:18+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.1:compile (default-compile) on project tornado-drivers-opencl: Compilation failure
[ERROR] /d00/local/tornado/tornadovm/drivers/opencl/src/main/java/uk/ac/manchester/tornado/drivers/opencl/graal/OCLArchitecture.java:[45,7] error: OCLArchitecture is not abstract and does not override abstract method getFeatures() in Architecture
[ERROR]

Expected behavior
Successful compilation.

Computing system setup (please complete the following information):

  • OS: centos7
  • OpenCL Version : 2.2 reported by the build process
  • TornadoVM commit id c61f388

Additional context

@jjfumero
Copy link
Member

Hi @yazun . OpenJDK 17 is not supported yet. Quite likely, the error you are getting is because the Graal version shipped in OpenJDK 17 has minor changes that we need to address and support.

Currently, we can run OpenJDK 8, 11 and 16 and Graal 21.2.0.

These are the backends and the configurations supported:
https://github.com/beehive-lab/TornadoVM/blob/master/assembly/src/docs/1_INSTALL.md#1-installation

@jjfumero jjfumero self-assigned this Sep 17, 2021
@jjfumero jjfumero added the question Further information is requested label Sep 17, 2021
@yazun
Copy link
Contributor Author

yazun commented Sep 17, 2021

Hello @jjfumero,
Was not Graal dropped from OpenJDK JDK-8268272 ?
Would it mean it would not be possible to use TornadoVM with it? Or you mean TornadoVM would have to upgrade Graal to v17?
Thanks for clarification.

@jjfumero
Copy link
Member

Yes, you are right.
What I meant to say is that TornadoVM needs to sync with JVMCI. We do this in order to use a specific GraalVM version as well as using the shipped JVMCI for each JDK. In fact, one of the errors I see from your logs is a change in JVMCI.

It is in our plans to upgrade to JDK17, meanwhile, if possible, you can give it a try with older versions, such as OpenJDK 16, or 11.

Hope this helps.

@yazun
Copy link
Contributor Author

yazun commented Sep 20, 2021

We just finished a greater effort to move to JDK17 so we would be happy to try TornadoVM once the updated to latest JVMCI.
(just for the info, we consider TornadoVM for astronomy, Variability Studies in ESA Gaia project )
Thanks

@jjfumero
Copy link
Member

Thank you for your interest. I will leave this thread open. Once we support JDK17 we will post it here.

@yazun
Copy link
Contributor Author

yazun commented Sep 21, 2021

Thank you, appreciated!

@jjfumero
Copy link
Member

jjfumero commented Nov 25, 2021

Hi @yazun ,
TornadoVM now supports JDK 17 and GraalVM 21.3.0. It is upstream in the develop branch:

https://github.com/beehive-lab/TornadoVM/commits/develop

$ tornado -version
WARNING: Using incubator modules: jdk.incubator.foreign, jdk.incubator.vector
openjdk version "17.0.1" 2021-10-19
OpenJDK Runtime Environment (build 17.0.1+12-39)
OpenJDK 64-Bit Server VM (build 17.0.1+12-39, mixed mode)
$ tornado --version
version=0.13-dev
branch=develop
commit=d61859b

Please, let us know if this works for you, and feel free to open new issues if you encounter new problems or want to add your feedback.

Juan

@jjfumero jjfumero added this to To do in TornadoVM v0.13 via automation Nov 25, 2021
@jjfumero jjfumero added the enhancement New feature or request label Nov 25, 2021
@yazun
Copy link
Contributor Author

yazun commented Nov 25, 2021

That's excellent news!
We will try in coming weeks - we will assess gains on our cluster with Intel GPUs then CUDA.
Thanks a lot!

TornadoVM v0.13 automation moved this from To do to Done Nov 25, 2021
@yazun
Copy link
Contributor Author

yazun commented Nov 25, 2021

A minor annoyance:
We had a trouble to compile the branch without cloning repo fresh though, looked like make clean did not do its job fully. Cloning fresh helped and it compiled successfully in the end.

Log of the failing compoile without fresh clone, after the make clean:

[root@ecogcn134 tornadovm]# git checkout develop
Already on 'develop'
[root@ecogcn134 tornadovm]# git pull
Already up-to-date.
[root@ecogcn134 tornadovm]# cat etc/sources.env
#!/bin/bash
export JAVA_HOME=/d00/gaia/jdk-17  ## This path is produced in Step 1
export PATH=$PWD/bin:$PATH    ## This directory will be automatically generated during Tornado compilation
export TORNADO_SDK=$PWD/bin/sdk   ## This directory will be automatically generated during Tornado compilation
export CMAKE_ROOT=/d00/local/tornado/cmake-3.10.1           ## or <path/to/cmake/cmake-3.10.2> (see step 4)
[root@ecogcn134 tornadovm]# make jdk-11-plus BACKEND=opencl
./bin/compile.sh jdk-11-plus opencl
JAVA_HOME    : /d00/gaia/jdk-17
JVM_VERSION  : 17
[INFO] Scanning for projects...
[WARNING]
[WARNING] Some problems were encountered while building the effective model for tornado:tornado-runtime:jar:0.13-dev
[WARNING] The expression ${version} is deprecated. Please use ${project.version} instead.
[WARNING]
[WARNING] Some problems were encountered while building the effective model for tornado:tornado-matrices:jar:0.13-dev
[WARNING] The expression ${version} is deprecated. Please use ${project.version} instead.
[WARNING]
[WARNING] Some problems were encountered while building the effective model for tornado:tornado-drivers-common:jar:0.13-dev
[WARNING] The expression ${version} is deprecated. Please use ${project.version} instead.
[WARNING]
[WARNING] Some problems were encountered while building the effective model for tornado:tornado-drivers-opencl:jar:0.13-dev
[WARNING] The expression ${version} is deprecated. Please use ${project.version} instead.
[WARNING]
[WARNING] Some problems were encountered while building the effective model for tornado:tornado-drivers-opencl-jni:jar:0.13-dev
[WARNING] The expression ${version} is deprecated. Please use ${project.version} instead.
[WARNING]
[WARNING] Some problems were encountered while building the effective model for tornado:tornado-drivers-spirv:jar:0.13-dev
[WARNING] The expression ${version} is deprecated. Please use ${project.version} instead.
[WARNING]
[WARNING] Some problems were encountered while building the effective model for tornado:tornado-drivers-spirv-levelzero-jni:jar:0.13-dev
[WARNING] The expression ${version} is deprecated. Please use ${project.version} instead.
[WARNING]
[WARNING] Some problems were encountered while building the effective model for tornado:tornado-drivers-ptx:jar:0.13-dev
[WARNING] The expression ${version} is deprecated. Please use ${project.version} instead.
[WARNING]
[WARNING] Some problems were encountered while building the effective model for tornado:tornado-drivers-ptx-jni:jar:0.13-dev
[WARNING] The expression ${version} is deprecated. Please use ${project.version} instead.
[WARNING]
[WARNING] Some problems were encountered while building the effective model for tornado:tornado-drivers:pom:0.13-dev
[WARNING] The expression ${version} is deprecated. Please use ${project.version} instead.
[WARNING]
[WARNING] Some problems were encountered while building the effective model for tornado:tornado-examples:jar:0.13-dev
[WARNING] The expression ${version} is deprecated. Please use ${project.version} instead.
[WARNING]
[WARNING] Some problems were encountered while building the effective model for tornado:tornado-benchmarks:jar:0.13-dev
[WARNING] The expression ${version} is deprecated. Please use ${project.version} instead.
[WARNING]
[WARNING] Some problems were encountered while building the effective model for tornado:tornado-unittests:jar:0.13-dev
[WARNING] The expression ${version} is deprecated. Please use ${project.version} instead.
[WARNING]
[WARNING] Some problems were encountered while building the effective model for tornado:tornado-assembly:pom:0.13-dev
[WARNING] The expression ${version} is deprecated. Please use ${project.version} instead.
[WARNING]
[WARNING] Some problems were encountered while building the effective model for tornado:tornado-api:jar:0.13-dev
[WARNING] The expression ${version} is deprecated. Please use ${project.version} instead.
[WARNING]
[WARNING] Some problems were encountered while building the effective model for tornado:tornado-annotation:jar:0.13-dev
[WARNING] The expression ${version} is deprecated. Please use ${project.version} instead.
[WARNING]
[WARNING] Some problems were encountered while building the effective model for tornado:tornado:pom:0.13-dev
[WARNING] The expression ${version} is deprecated. Please use ${project.version} instead.
[WARNING]
[WARNING] It is highly recommended to fix these problems because they threaten the stability of your build.
[WARNING]
[WARNING] For this reason, future Maven versions might no longer support building such malformed projects.
[WARNING]
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] tornado                                                            [pom]
[INFO] tornado-api                                                        [jar]
[INFO] tornado-runtime                                                    [jar]
[INFO] tornado-matrices                                                   [jar]
[INFO] tornado-drivers                                                    [pom]
[INFO] tornado-drivers-common                                             [jar]
[INFO] tornado-drivers-opencl                                             [jar]
[INFO] tornado-drivers-opencl-jni                                         [jar]
[INFO] tornado-drivers-spirv                                              [jar]
[INFO] tornado-drivers-spirv-levelzero-jni                                [jar]
[INFO] tornado-drivers-ptx                                                [jar]
[INFO] tornado-drivers-ptx-jni                                            [jar]
[INFO] tornado-examples                                                   [jar]
[INFO] tornado-benchmarks                                                 [jar]
[INFO] tornado-unittests                                                  [jar]
[INFO] tornado-annotation                                                 [jar]
[INFO] tornado-assembly                                                   [pom]
[INFO]
[INFO] --------------------------< tornado:tornado >---------------------------
[INFO] Building tornado 0.13-dev                                         [1/17]
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ tornado ---
[INFO] Deleting /d00/local/tornado/tornadovm/target
[INFO]
[INFO] ------------------------< tornado:tornado-api >-------------------------
[INFO] Building tornado-api 0.13-dev                                     [2/17]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ tornado-api ---
[INFO] Deleting /d00/local/tornado/tornadovm/tornado-api/target
[INFO]
[INFO] ----------------------< tornado:tornado-runtime >-----------------------
[INFO] Building tornado-runtime 0.13-dev                                 [3/17]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ tornado-runtime ---
[INFO]
[INFO] ----------------------< tornado:tornado-matrices >----------------------
[INFO] Building tornado-matrices 0.13-dev                                [4/17]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ tornado-matrices ---
[INFO]
[INFO] ----------------------< tornado:tornado-drivers >-----------------------
[INFO] Building tornado-drivers 0.13-dev                                 [5/17]
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ tornado-drivers ---
[INFO]
[INFO] -------------------< tornado:tornado-drivers-common >-------------------
[INFO] Building tornado-drivers-common 0.13-dev                          [6/17]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ tornado-drivers-common ---
[INFO]
[INFO] -------------------< tornado:tornado-drivers-opencl >-------------------
[INFO] Building tornado-drivers-opencl 0.13-dev                          [7/17]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ tornado-drivers-opencl ---
[INFO]
[INFO] -----------------< tornado:tornado-drivers-opencl-jni >-----------------
[INFO] Building tornado-drivers-opencl-jni 0.13-dev                      [8/17]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ tornado-drivers-opencl-jni ---
[INFO]
[INFO] -------------------< tornado:tornado-drivers-spirv >--------------------
[INFO] Building tornado-drivers-spirv 0.13-dev                           [9/17]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ tornado-drivers-spirv ---
[INFO]
[INFO] ------------< tornado:tornado-drivers-spirv-levelzero-jni >-------------
[INFO] Building tornado-drivers-spirv-levelzero-jni 0.13-dev            [10/17]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ tornado-drivers-spirv-levelzero-jni ---
[INFO]
[INFO] --------------------< tornado:tornado-drivers-ptx >---------------------
[INFO] Building tornado-drivers-ptx 0.13-dev                            [11/17]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ tornado-drivers-ptx ---
[INFO]
[INFO] ------------------< tornado:tornado-drivers-ptx-jni >-------------------
[INFO] Building tornado-drivers-ptx-jni 0.13-dev                        [12/17]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ tornado-drivers-ptx-jni ---
[INFO]
[INFO] ----------------------< tornado:tornado-examples >----------------------
[INFO] Building tornado-examples 0.13-dev                               [13/17]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ tornado-examples ---
[INFO]
[INFO] ---------------------< tornado:tornado-benchmarks >---------------------
[INFO] Building tornado-benchmarks 0.13-dev                             [14/17]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ tornado-benchmarks ---
[INFO]
[INFO] ---------------------< tornado:tornado-unittests >----------------------
[INFO] Building tornado-unittests 0.13-dev                              [15/17]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ tornado-unittests ---
[INFO]
[INFO] ---------------------< tornado:tornado-annotation >---------------------
[INFO] Building tornado-annotation 0.13-dev                             [16/17]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ tornado-annotation ---
[INFO]
[INFO] ----------------------< tornado:tornado-assembly >----------------------
[INFO] Building tornado-assembly 0.13-dev                               [17/17]
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] --- maven-clean-plugin:3.0.0:clean (default-clean) @ tornado-assembly ---
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for tornado 0.13-dev:
[INFO]
[INFO] tornado ............................................ SUCCESS [  0.095 s]
[INFO] tornado-api ........................................ SUCCESS [  0.004 s]
[INFO] tornado-runtime .................................... SUCCESS [  0.003 s]
[INFO] tornado-matrices ................................... SUCCESS [  0.003 s]
[INFO] tornado-drivers .................................... SUCCESS [  0.003 s]
[INFO] tornado-drivers-common ............................. SUCCESS [  0.002 s]
[INFO] tornado-drivers-opencl ............................. SUCCESS [  0.002 s]
[INFO] tornado-drivers-opencl-jni ......................... SUCCESS [  0.003 s]
[INFO] tornado-drivers-spirv .............................. SUCCESS [  0.002 s]
[INFO] tornado-drivers-spirv-levelzero-jni ................ SUCCESS [  0.003 s]
[INFO] tornado-drivers-ptx ................................ SUCCESS [  0.002 s]
[INFO] tornado-drivers-ptx-jni ............................ SUCCESS [  0.002 s]
[INFO] tornado-examples ................................... SUCCESS [  0.003 s]
[INFO] tornado-benchmarks ................................. SUCCESS [  0.002 s]
[INFO] tornado-unittests .................................. SUCCESS [  0.003 s]
[INFO] tornado-annotation ................................. SUCCESS [  0.002 s]
[INFO] tornado-assembly ................................... SUCCESS [  0.111 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  0.441 s
[INFO] Finished at: 2021-11-25T11:38:45+01:00
[INFO] ------------------------------------------------------------------------
mvn -T1.5C -Dcmake.root.dir=/d00/local/tornado/cmake-3.10.1 -Pjdk-11-plus,opencl-backend  install
[INFO] Scanning for projects...
[WARNING]
[WARNING] Some problems were encountered while building the effective model for tornado:tornado-runtime:jar:0.13-dev
[WARNING] The expression ${version} is deprecated. Please use ${project.version} instead.
[WARNING]
[WARNING] Some problems were encountered while building the effective model for tornado:tornado-matrices:jar:0.13-dev
[WARNING] The expression ${version} is deprecated. Please use ${project.version} instead.
[WARNING]
[WARNING] Some problems were encountered while building the effective model for tornado:tornado-drivers-common:jar:0.13-dev
[WARNING] The expression ${version} is deprecated. Please use ${project.version} instead.
[WARNING]
[WARNING] Some problems were encountered while building the effective model for tornado:tornado-drivers-opencl:jar:0.13-dev
[WARNING] The expression ${version} is deprecated. Please use ${project.version} instead.
[WARNING]
[WARNING] Some problems were encountered while building the effective model for tornado:tornado-drivers-opencl-jni:jar:0.13-dev
[WARNING] The expression ${version} is deprecated. Please use ${project.version} instead.
[WARNING]
[WARNING] Some problems were encountered while building the effective model for tornado:tornado-drivers:pom:0.13-dev
[WARNING] The expression ${version} is deprecated. Please use ${project.version} instead.
[WARNING]
[WARNING] Some problems were encountered while building the effective model for tornado:tornado-examples:jar:0.13-dev
[WARNING] The expression ${version} is deprecated. Please use ${project.version} instead.
[WARNING]
[WARNING] Some problems were encountered while building the effective model for tornado:tornado-benchmarks:jar:0.13-dev
[WARNING] The expression ${version} is deprecated. Please use ${project.version} instead.
[WARNING]
[WARNING] Some problems were encountered while building the effective model for tornado:tornado-unittests:jar:0.13-dev
[WARNING] The expression ${version} is deprecated. Please use ${project.version} instead.
[WARNING]
[WARNING] Some problems were encountered while building the effective model for tornado:tornado-assembly:pom:0.13-dev
[WARNING] The expression ${version} is deprecated. Please use ${project.version} instead.
[WARNING]
[WARNING] Some problems were encountered while building the effective model for tornado:tornado-api:jar:0.13-dev
[WARNING] The expression ${version} is deprecated. Please use ${project.version} instead.
[WARNING]
[WARNING] Some problems were encountered while building the effective model for tornado:tornado-annotation:jar:0.13-dev
[WARNING] The expression ${version} is deprecated. Please use ${project.version} instead.
[WARNING]
[WARNING] Some problems were encountered while building the effective model for tornado:tornado:pom:0.13-dev
[WARNING] The expression ${version} is deprecated. Please use ${project.version} instead.
[WARNING]
[WARNING] It is highly recommended to fix these problems because they threaten the stability of your build.
[WARNING]
[WARNING] For this reason, future Maven versions might no longer support building such malformed projects.
[WARNING]
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] tornado                                                            [pom]
[INFO] tornado-api                                                        [jar]
[INFO] tornado-runtime                                                    [jar]
[INFO] tornado-matrices                                                   [jar]
[INFO] tornado-drivers                                                    [pom]
[INFO] tornado-drivers-common                                             [jar]
[INFO] tornado-drivers-opencl                                             [jar]
[INFO] tornado-drivers-opencl-jni                                         [jar]
[INFO] tornado-examples                                                   [jar]
[INFO] tornado-benchmarks                                                 [jar]
[INFO] tornado-unittests                                                  [jar]
[INFO] tornado-annotation                                                 [jar]
[INFO] tornado-assembly                                                   [pom]
[INFO]
[INFO] Using the MultiThreadedBuilder implementation with a thread count of 12
[INFO]
[INFO] --------------------------< tornado:tornado >---------------------------
[INFO] Building tornado 0.13-dev                                         [1/13]
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] --- buildnumber-maven-plugin:1.4:create (default) @ tornado ---
[INFO] ShortRevision tag detected. The value is '7'.
[INFO] Executing: /bin/sh -c cd '/d00/local/tornado/tornadovm' && 'git' 'rev-parse' '--verify' '--short=7' 'HEAD'
[INFO] Working directory: /d00/local/tornado/tornadovm
[INFO] Storing buildNumber: d61859b at timestamp: 1637836727001
[INFO] Storing buildScmBranch: develop
[INFO]
[INFO] --- maven-shade-plugin:3.2.1:shade (default) @ tornado ---
[INFO] Including org.openjdk.jmh:jmh-core:jar:1.29 in the shaded jar.
[INFO] Including net.sf.jopt-simple:jopt-simple:jar:4.6 in the shaded jar.
[INFO] Including org.apache.commons:commons-math3:jar:3.2 in the shaded jar.
[INFO] Including org.apache.logging.log4j:log4j-api:jar:2.13.2 in the shaded jar.
[INFO] Including org.apache.logging.log4j:log4j-core:jar:2.13.2 in the shaded jar.
[INFO] Replacing /d00/local/tornado/tornadovm/target/jmhbenchmarks.pom with /d00/local/tornado/tornadovm/target/tornado-0.13-dev-shaded.pom
[INFO]
[INFO] --- maven-install-plugin:2.4:install (default-install) @ tornado ---
[INFO] Installing /d00/local/tornado/tornadovm/pom.xml to /root/.m2/repository/tornado/tornado/0.13-dev/tornado-0.13-dev.pom
[INFO]
[INFO] ------------------------< tornado:tornado-api >-------------------------
[INFO] Building tornado-api 0.13-dev                                     [2/13]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- buildnumber-maven-plugin:1.4:create (default) @ tornado-api ---
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ tornado-api ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /d00/local/tornado/tornadovm/tornado-api/src/main/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) @ tornado-api ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 132 source files to /d00/local/tornado/tornadovm/tornado-api/target/classes
[INFO] -------------------------------------------------------------
[WARNING] COMPILATION WARNING :
[INFO] -------------------------------------------------------------
[WARNING] [options] system modules path not set in conjunction with -source 11
[WARNING] [options] --add-opens has no effect at compile time
[INFO] 2 warnings
[INFO] -------------------------------------------------------------
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] error: duplicate module on upgrade module path
[INFO] 1 error
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for tornado 0.13-dev:
[INFO]
[INFO] tornado ............................................ SUCCESS [  1.096 s]
[INFO] tornado-api ........................................ FAILURE [  0.995 s]
[INFO] tornado-runtime .................................... SKIPPED
[INFO] tornado-matrices ................................... SKIPPED
[INFO] tornado-drivers .................................... SKIPPED
[INFO] tornado-drivers-common ............................. SKIPPED
[INFO] tornado-drivers-opencl ............................. SKIPPED
[INFO] tornado-drivers-opencl-jni ......................... SKIPPED
[INFO] tornado-examples ................................... SKIPPED
[INFO] tornado-benchmarks ................................. SKIPPED
[INFO] tornado-unittests .................................. SKIPPED
[INFO] tornado-annotation ................................. SKIPPED
[INFO] tornado-assembly ................................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  2.281 s (Wall Clock)
[INFO] Finished at: 2021-11-25T11:38:48+01:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.1:compile (default-compile) on project tornado-api: Compilation failure
[ERROR] error: duplicate module on upgrade module path
[ERROR]
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <args> -rf :tornado-api

 Compilation failed

The version with mvn -X can be found here: https://pastebin.com/1kyh2jLf

@yazun
Copy link
Contributor Author

yazun commented Nov 25, 2021

After the successful compilation we get error trying to run examples:

tornado -m tornado.examples/uk.ac.manchester.tornado.examples.compute.MatrixMultiplication2D 512
Unrecognized VM option 'UseParallelOldGC'
Did you mean '(+/-)UseParallelGC'? Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.

@yazun
Copy link
Contributor Author

yazun commented Nov 25, 2021

@jjfumero Let me know if you'd prefer a new issue to be created for this one.

@yazun
Copy link
Contributor Author

yazun commented Nov 25, 2021

root@ecogcn134 tornadovm_fresh]# JAVA_CMD=${JAVA_HOME}/bin/java
[root@ecogcn134 tornadovm_fresh]# JAVA_VERSION_OUTPUT=$("$JAVA_CMD" -version 2>&1)
[root@ecogcn134 tornadovm_fresh]# JAVA_VERSION=$(echo "$JAVA_VERSION_OUTPUT" | awk -F[\"\.] -v OFS=. 'NR==1{print $2,$3}')
[root@ecogcn134 tornadovm_fresh]# echo $JAVA_VERSION
17. 2021-09-14

Looks like openjdk added timestamp to the version string without the minor version.

@jjfumero
Copy link
Member

Thanks for the report. I am trying to reproduce the error but it seems you are using an older version. Are you using the latest develop?

$ git clone https://github.com/beehive-lab/TornadoVM/
$ cd TornadoVM && git checkout develop
$ export PATH=$PWD/bin/bin:$PATH    ## This directory will be automatically generated during Tornado compilation
$ export TORNADO_SDK=$PWD/bin/sdk   ## This directory will be automatically generated during Tornado compilation
$ export CMAKE_ROOT=/usr 
$ export JAVA_HOME=/home/juan/jvms/graalvm-ce-java17-21.3.0   ## USE YOUR PATH HERE
$ make 

We changed the default GC to -XX:+UseParallelGC

$ tornado -m tornado.examples/uk.ac.manchester.tornado.examples.compute.MatrixMultiplication2D 512
WARNING: Using incubator modules: jdk.incubator.foreign, jdk.incubator.vector
Computing MxM of 512x512
[TornadoVM] Warning: The loop bounds will be configured by the GridScheduler. Check the grid by using the flag --threadInfo.
	Single Threaded CPU Execution: 1.04 GFlops, Total time = 257 ms
	TornadoVM Execution on GPU (Accelerated): 268.44 GFlops, Total Time = 1 ms
	Speedup: 257.0x
	Verification true

$  tornado -version
WARNING: Using incubator modules: jdk.incubator.foreign, jdk.incubator.vector
openjdk version "17.0.1" 2021-10-19
OpenJDK Runtime Environment GraalVM CE 21.3.0 (build 17.0.1+12-jvmci-21.3-b05)
OpenJDK 64-Bit Server VM GraalVM CE 21.3.0 (build 17.0.1+12-jvmci-21.3-b05, mixed mode)

Let me know if this works. Otherwise please, open a new issue with the bug report.

@jjfumero jjfumero reopened this Nov 25, 2021
TornadoVM v0.13 automation moved this from Done to In progress Nov 25, 2021
@yazun
Copy link
Contributor Author

yazun commented Nov 26, 2021

I can confirm that version parsing was the problem - when kludging it with removal of minor version JDK checks at lines 121 and 285:

121:JAVA_VERSION=$(echo "$JAVA_VERSION_OUTPUT" | awk -F[\"\.] -v OFS=. 'NR==1{print $2}')
285: if [ "$JAVA_VERSION" = "16.0" ] || [ "$JAVA_VERSION" = "17" ]; then

it worked!

tornado -Dtornado.enable.nativeFunctions=true -m tornado.benchmarks/uk.ac.manchester.tornado.benchmarks.BenchmarkRunner dft 8192 4096 --iterations 1
WARNING: Using incubator modules: jdk.incubator.foreign, jdk.incubator.vector
bm=dft-15-8192    , id=java-reference      , average=3.907213e+09, median=3.858058e+09, firstIteration=4.221748e+09, best=3.852115e+09
bm=dft-15-8192    , device=0:0  , average=6.537091e+08, median=6.212930e+08, firstIteration=6.964311e+08, best=5.920942e+08, speedupAvg=5.9770, speedupMedian=6.2097, speedupFirstIteration=6.0620, CV=-0.0000%, deviceName= [Intel(R) CPU Runtime for OpenCL(TM) Applications] -- Intel(R) Xeon(R) CPU E3-1585L v5 @ 3.00GHz
bm=dft-15-8192    , device=0:1  , average=1.005646e+07, median=9.893467e+06, firstIteration=1.149101e+07, best=9.294369e+06, speedupAvg=388.5274, speedupMedian=389.9602, speedupFirstIteration=367.3957, CV=-0.0000%, deviceName= [Intel(R) OpenCL HD Graphics] -- Intel(R) Iris(TM) Pro Graphics P580 [0x193a]

We have probably the first released JDK17 installed with no minor versions yet.

@jjfumero
Copy link
Member

Great. I will close this issue then. Feel free to open new issues if you encounter new problems.

TornadoVM v0.13 automation moved this from In progress to Done Nov 26, 2021
@yazun
Copy link
Contributor Author

yazun commented Nov 26, 2021

As a side note the github maven repo you provide still does not have ver 0.13 jars. Can we use 0.12 jars to use with 0.13 distribution?

@jjfumero
Copy link
Member

Yes, that's right. Our maven repos only ship the releases. What you propose will also work because we rarely change the API. We haven't scheduled any changes for the next release regarding the API, so it will keep as it is for now.

What we do internally is to use 0.13-dev (current tag as today) in our pom files and then we update accordingly with each release.

Hope this helps

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request question Further information is requested
Projects
No open projects
Development

No branches or pull requests

2 participants