NPB in HermitCore #104

bguala · 2018-10-12T20:55:50Z

Hello

I need run the NAS parallel benchmarks (NPB) in HermitCore. Is possible this?.
NPBs are implement in Fortran, the build process require run a makefile, called make.def, this file contains compilation params.
make.def generates an executable based in classes of complexity (S, A, W). Exists three classes of benchmark: BT, SP and LU, they solves diferential ecuations systems. When I run the command "make benchmark class" with the class BT, for example, it generates an executable named BT-MZ.S.x. This executable, can it run in HermitCore?. Can it build with the build process of HermitCore?.
Is there some way for solve this problem?

stlankes · 2018-10-12T22:22:44Z

I already test NPB with an older version of HermitCore and it works! In make.def you have to set F77 to x86_64-hermit-gfortran and CC to x86_64-hermit-gcc. The other flags are similar to the gcc configuration, if you want to compile NPB on Linux. Please inform me, if you have any problems to build NPB.

bguala · 2018-10-13T18:53:15Z

I can't build the NPBs in HermitCore.

1) First Configuration

a) I copied the directory NPB3.3.1-MZ in libhermit/usr. NPB3.3.1-MZ has the follow structure:

NPB3.3.1-MZ-MPI
NPB3.3.1-MZ-OMP = this is the version i want to test in HermitCore.
NPB3.3.1-MZ-SER
Changes.log
README

b) I included the CmakeLists.txt file in NPB3.3.1-MZ. This file contains the follow sentences:

cmake_minimum_required(VERSION 3.7)
include(../../cmake/HermitCore-Application.cmake)

install_local_targets(extra/NPB3.3.1-MZ)

c) I modified the make.def file in libhermit/usr/NPB3.3.1-MZ/NPB3.3-MZ-OMP/config, with the follow sentences:

F77 = x86_64-hermit-gfortran
CC = x86_64-hermit-gcc

d) I added the follow sentence in the libhermit/CmakeLists.txt file:

#Test and benchmarks
build_external(NPB3.3.1-MZ ${HERMIT_ROOT}/usr/NPB3.3.1-MZ hermit)

e) Finally, I ran the build process of HermitCore:

sudo cmake ..  => OK
sudo make       => ERROR
[80%] Performing configure step for 'NPB3.3.1-MZ'
Not searching for unused variables given on the command line
     --The CXX compiler identification is unknown
Cmake Error in CmakeLists.txt
No CMAKE_CXX_COMPILER could be found

2) Second Configuration

a) I only modified the CmakeLists.txt file in libhermit/usr/NPB3.3.1-MZ. Now, this file contains the follow sentences:

project(hermit-NPB Fortran)
project(hermit-NPB CXX)

b) I ran the build process:

sudo cmake .. OK
sudo make ERROR
[ 80%] Performing configure step for 'NPB3.3.1-MZ'
Not searching for unused variables given on the command line.
-- The CXX compiler identification is unknown
CMake Error at /home/bguala/Escritorio/HermitCore/libhermit/cmake/HermitCore.cmake:72 (enable_language):
  No CMAKE_CXX_COMPILER could be found.

CMake Error at /home/bguala/Escritorio/HermitCore/libhermit/cmake/HermitCore.cmake:72 (enable_language):
  No CMAKE_Go_COMPILER could be found.

Am I forget something?

stlankes · 2018-10-13T20:42:06Z

Hm, personal I would not use NPB3.3.1-MZ. I prefer NPB3.3.1, which also supports OpenMP.

In libhermit/usr/NPB3.3.1/CmakeLists.txt, I would add a command to call the external tool make. As far as I know, you have to use add_custom_target. It could look like this:

add_custom_target(
   NPB3.3.1
   COMMAND make
   WORKING_DIRECTORY full_path to where Makefile is located
)

bguala · 2018-10-15T23:49:57Z

The NPB3.3.1-MZ doesn't work. Therefore, I tested NPB3.3.1 with

add_custom_target(
   NPB3.3.1
   COMMAND make
   WORKING_DIRECTORY /home/bguala/Escritorio/HermitCore/libhermit/usr/NPB3.3.1/NPB3.3.1-OMP
)

in libhermit/usr/NPB3.3.1/CmakeLists.txt, but this sentence generates the follow error message:

CMake Error at /home/bguala/Escritorio/HermitCore/libhermit/cmake/HermitCore-Utils.cmake:128 (install):
  install TARGETS given target "NPB3.3.1" which is not an executable,
  library, or module.
Call Stack (most recent call first):
  CMakeLists.txt:13 (install_local_targets)

when I run the command "sudo make". The sintax of add_custom_target is

add_custom_target(Name [ALL] [command1 [args1...]]
                  [COMMAND command2 [args2...] ...]
                  [DEPENDS depend depend depend ... ]
                  [BYPRODUCTS [files...]]
                  [WORKING_DIRECTORY dir]
                  [COMMENT comment]
                  [VERBATIM] [USES_TERMINAL]
                  [SOURCES src1 [src2...]])

I also tested diferent variants of this sentence such as:

add_custom_target(NPB3.3.1 make bt CLASS=S)

but, it doesn't work.

stlankes · 2018-10-16T05:16:58Z

If I understand it correctly, the application will be built but not installed. The makefile doesn't have an install rule. Maybe you should start with an empty rule to test, if this is the problem. Please add the end following lines to NPB's makefile. The last line contains only a TAB.

install:

bguala · 2018-10-16T19:56:31Z

I added install: in libhermit/usr/NPB3.3.1/CmakeLists.txt, but when I run sudo make I see

CMake Error at /home/bguala/Escritorio/HermitCore/libhermit/cmake/HermitCore-Utils.cmake:128 (install):
  install TARGETS given target "NPB3.3.1" which is not an executable,
  library, or module.
Call Stack (most recent call first):
  CMakeLists.txt:14 (install_local_targets)

Build NPB implies to execute make BT CLASS=S, for example. Where does this sentence include?. make BT CLASS=S saves the executable bt.S.x in libhermit/usr/NPB3.3.1/NPB3.3.1-OMP/bin. Therefore, Should I delete the sentence install_local_targets(extra/NPB3.3.1) in libhermit/usr/NPB3.3.1/CmakeLists.txt?.

stlankes · 2018-10-18T08:28:14Z

Hm, difficult to understand. Could send me your configuration files. I will test it on my system,

bguala · 2018-10-19T02:38:00Z

Yes, of course. These are my files:

CMakeLists.txt contains the sentence build_external.

CMakeLists.txt contains add_custom_target.

stlankes · 2019-01-07T06:17:19Z

Excuse me, I forgot this request. Do you still need support?

bguala · 2019-01-09T23:27:19Z

Yes, I do. The last thing I tried was to compile the benchmarks separately, but it did not work.

stlankes · 2019-01-10T22:08:00Z

Ok, I will do it tomorrow

stlankes · 2019-01-15T23:19:23Z

Please test the version at https://gigamove.rz.rwth-aachen.de/d/id/JtGNq5LWradWdY

It isn't fully integrated in our toolchain. But it works on our systems. Does it help?

bguala · 2019-01-28T01:03:11Z

I don't know. I'm traying...

bguala · 2019-12-01T02:30:31Z

Hello.

I am traying to build NPB in HermitCore again. This time the build process begins, but it ends when performing configure step for xray.

The current hierarchy of project is:

HermitCore/libhermit/usr/CMakeLists.txt: this file contains the sentence:

build_external(BPB3.3-OMP ${HERMIT_ROOT}/usr/NPB3.3-OMP hermit)

HermitCore/libhermit/usr/NPB3.3-OMP/CMakeLists: this file contains the code:

project(hermit-NPB Fortran)

add_custom_target(
BPB3.3-OMP
COMMAND make
WORKING DIRECTORY /Desktop/HermitCore/libhermit/usr/NPB3.3-OMP
)

install_local_targets(extra/NPB3.3-OMP)

NPB3.3-OMP also contains a makefile.

What is xray for??. A simple profiler for Native Client.

Can it be resolved by removing benchamarks???.

stlankes · 2019-12-03T22:38:14Z

Hm, xray is a simple profiler. You can remove it. Comment out these lines. Does it work for you?

bguala · 2019-12-25T22:22:08Z

I commented both lines (build_external and add_dependences) in libhermit/CmakeLists.txt, but It didn't work, now I see the error message munmap_chunk() invalid pointer:

I executed the command make in libhermit/usr. The folders arch and CmakeFiles are located inside the usr, they contains all .obj files linked in libhermit.a. The error message happends in libhermit/usr/CMakeFiles/hermit-bootstraps.dir

$(CMAKE_COMMAND) -P CMakeFiles/hermit-bootstrap.dir/cmake_clean_target.cmake
$(CMAKE_COMMAND) -E cmake_link_script CMakeFiles/hermit-bootstrap.dir/link.txt --verbose=$(VERBOSE)
/opt/hermit/bin/x86_64-hermit-objcopy --rename-section .bss=.kbss --rename-section .text=.ktext --rename-section .data=.kdata /home/bguala/Escritorio/HermitCore/libhermit/usr/libhermit.a
/usr/bin/cmake -E make_directory /home/bguala/Escritorio/HermitCore/libhermit/build/local_prefix//opt/hermit/x86_64-hermit/lib
/usr/bin/cmake -E copy_if_different /home/bguala/Escritorio/HermitCore/libhermit/usr/libhermit.a /home/bguala/Escritorio/HermitCore/libhermit/build/local_prefix//opt/hermit/x86_64-hermit/lib/
/usr/bin/cmake -E make_directory /home/bguala/Escritorio/HermitCore/libhermit/build/local_prefix//opt/hermit/x86_64-hermit/include/hermit
/usr/bin/cmake -E copy_if_different /home/bguala/Escritorio/HermitCore/libhermit/usr/include/hermit/.h /home/bguala/Escritorio/HermitCore/libhermit/build/local_prefix//opt/hermit/x86_64-hermit/include/hermit/
/usr/bin/cmake -E copy_if_different /home/bguala/Escritorio/HermitCore/libhermit/usr/include/hermit/.asm /home/bguala/Escritorio/HermitCore/libhermit/build/local_prefix//opt/hermit/x86_64-hermit/include/hermit/

However, if I try to execute the command make inside libhermit/build, the error message is different:

I don't understand, why is the error message different?.

stlankes · 2020-01-01T16:06:24Z

Do you change some other parts of the toolchain? I don't understand the error message munmap_chunk() invalid pointer.

I build everything from scratch and was able to build NPB 3.3.1 (openmp version, class C) with following config file "make.def" (see below). Does it work for you?

#---------------------------------------------------------------------------
#
#                SITE- AND/OR PLATFORM-SPECIFIC DEFINITIONS. 
#
#---------------------------------------------------------------------------

#---------------------------------------------------------------------------
# Items in this file will need to be changed for each platform.
#---------------------------------------------------------------------------

#---------------------------------------------------------------------------
# Parallel Fortran:
#
# For CG, EP, FT, MG, LU, SP, BT and UA, which are in Fortran, the following 
# must be defined:
#
# F77        - Fortran compiler
# FFLAGS     - Fortran compilation arguments
# F_INC      - any -I arguments required for compiling Fortran 
# FLINK      - Fortran linker
# FLINKFLAGS - Fortran linker arguments
# F_LIB      - any -L and -l arguments required for linking Fortran 
# 
# compilations are done with $(F77) $(F_INC) $(FFLAGS) or
#                            $(F77) $(FFLAGS)
# linking is done with       $(FLINK) $(F_LIB) $(FLINKFLAGS)
#---------------------------------------------------------------------------

#---------------------------------------------------------------------------
# This is the fortran compiler used for Fortran programs
#---------------------------------------------------------------------------
F77 = x86_64-hermit-gfortran
# This links fortran programs; usually the same as ${F77}
FLINK	= $(F77)

#---------------------------------------------------------------------------
# These macros are passed to the linker 
#---------------------------------------------------------------------------
F_LIB  =

#---------------------------------------------------------------------------
# These macros are passed to the compiler 
#---------------------------------------------------------------------------
F_INC =

#---------------------------------------------------------------------------
# Global *compile time* flags for Fortran programs
#---------------------------------------------------------------------------
FFLAGS	= -O2 -fopenmp

#---------------------------------------------------------------------------
# Global *link time* flags. Flags for increasing maximum executable 
# size usually go here. 
#---------------------------------------------------------------------------
FLINKFLAGS = -O2 -fopenmp


#---------------------------------------------------------------------------
# Parallel C:
#
# For IS and DC, which are in C, the following must be defined:
#
# CC         - C compiler 
# CFLAGS     - C compilation arguments
# C_INC      - any -I arguments required for compiling C 
# CLINK      - C linker
# CLINKFLAGS - C linker flags
# C_LIB      - any -L and -l arguments required for linking C 
#
# compilations are done with $(CC) $(C_INC) $(CFLAGS) or
#                            $(CC) $(CFLAGS)
# linking is done with       $(CLINK) $(C_LIB) $(CLINKFLAGS)
#---------------------------------------------------------------------------

#---------------------------------------------------------------------------
# This is the C compiler used for C programs
#---------------------------------------------------------------------------
CC = x86_64-hermit-gcc
# This links C programs; usually the same as ${CC}
CLINK	= $(CC)

#---------------------------------------------------------------------------
# These macros are passed to the linker 
#---------------------------------------------------------------------------
C_LIB  = -lm

#---------------------------------------------------------------------------
# These macros are passed to the compiler 
#---------------------------------------------------------------------------
C_INC =

#---------------------------------------------------------------------------
# Global *compile time* flags for C programs
# DC inspects the following flags (preceded by "-D"):
#
# IN_CORE - computes all views and checksums in main memory (if there is 
# enough memory)
#
# VIEW_FILE_OUTPUT - forces DC to write the generated views to disk
#
# OPTIMIZATION - turns on some nonstandard DC optimizations
#
# _FILE_OFFSET_BITS=64 
# _LARGEFILE64_SOURCE - are standard compiler flags which allow to work with 
# files larger than 2GB.
#---------------------------------------------------------------------------
CFLAGS	= -O2 -fopenmp

#---------------------------------------------------------------------------
# Global *link time* flags. Flags for increasing maximum executable 
# size usually go here. 
#---------------------------------------------------------------------------
CLINKFLAGS = -O2 -fopenmp


#---------------------------------------------------------------------------
# Utilities C:
#
# This is the C compiler used to compile C utilities.  Flags required by 
# this compiler go here also; typically there are few flags required; hence 
# there are no separate macros provided for such flags.
#---------------------------------------------------------------------------
UCC	= gcc


#---------------------------------------------------------------------------
# Destination of executables, relative to subdirs of the main directory. . 
#---------------------------------------------------------------------------
BINDIR	= ../bin


#---------------------------------------------------------------------------
# The variable RAND controls which random number generator 
# is used. It is described in detail in README.install. 
# Use "randi8" unless there is a reason to use another one. 
# Other allowed values are "randi8_safe", "randdp" and "randdpvec"
#---------------------------------------------------------------------------
RAND   = randi8
# The following is highly reliable but may be slow:
# RAND   = randdp


#---------------------------------------------------------------------------
# The variable WTIME is the name of the wtime source code module in the
# common directory.  
# For most machines,       use wtime.c
# For SGI power challenge: use wtime_sgi64.c
#---------------------------------------------------------------------------
WTIME  = wtime.c


#---------------------------------------------------------------------------
# Enable if either Cray (not Cray-X1) or IBM: 
# (no such flag for most machines: see common/wtime.h)
# This is used by the C compiler to pass the machine name to common/wtime.h,
# where the C/Fortran binding interface format is determined
#---------------------------------------------------------------------------
# MACHINE	=	-DCRAY
# MACHINE	=	-DIBM

bguala · 2020-01-02T23:45:37Z

I updated the HermitCore toolchain (gcc-hermit, libhermit) and I downloaded the project from repository https://github.com/hermitcore/libhermit to start from scratch, but it didn't work. This is my current setup:

In libhermit/CMakeLists.txt I added the following sentence:

build_external(NPB3.3-OMP ${HERMIT_ROOT}/usr/NPB3.3.1 hermit)

Then, in libhermit/usr I added the folder NPB3.3.1. It contains the folder NPB3.3-OMP and the file CMakeLists.txt. This file contains:

cmake_minimum_required(VERSION 3.7)

include(../../cmake/HermitCore-Application.cmake)

project(HermitCore C Fortran)

add_custom_target(

NPB3.3.1
COMMAND make
WORKING_DIRECTORY /Escritorio/HermitCore/libhermit/usr/NPB3.3.1/NPB3.3-OMP/Makefile

)

When I run the command make in libhermit/build, I see the following error message:

There is no rule to build install target, however the file Makefile located in libhermit/usr/NPB3.3.1/NPB3.3-OMP contains the install target empty:

install:

My config file make.def is the same as yours. The build process fails in the following sentences:

cd /home/bguala/Escritorio/HermitCore/libhermit/build/NPB3.3-OMP-prefix/src/NPB3.3-OMP-build && /usr/bin/cmake --build /home/bguala/Escritorio/HermitCore/libhermit/build/NPB3.3-OMP-prefix/src/NPB3.3-OMP-build --target install -- DESTDIR=/home/bguala/Escritorio/HermitCore/libhermit/build/local_prefix
cd /home/bguala/Escritorio/HermitCore/libhermit/build/NPB3.3-OMP-prefix/src/NPB3.3-OMP-build && /usr/bin/cmake -E touch /home/bguala/Escritorio/HermitCore/libhermit/build/NPB3.3-OMP-prefix/src/NPB3.3-OMP-stamp/NPB3.3-OMP-install

located in build.make inside the folder libhermit/build/CMakeLists/NPB3.3-OMP.dir. I tried to use install: ../bin, but it didn't work.

Without the install_local_target(extra/npb) sentence, where build programs are saved??.

Your setup is the same???.

bguala · 2020-02-03T01:49:49Z

After doing several tests, I was able to advance the compilation of the NPB. Now, when I run the command make no errors ocur.

However, I don't understand where the benchmarks that were compiled were. They are not in the bin or extra folder.
I should have these benchmarks somewhere.

The CMakeLists.txt file in NPB-3.3-OMP contains the install_local_targets(extra/bin) statement, but apparently the build process ignores this order.

I need to add something in some makefile???.

stlankes · 2020-02-08T20:08:39Z

Hm, the cmake file is included in the main file like the OpenMP benchmarks (https://github.com/hermitcore/libhermit/blob/master/CMakeLists.txt#L177)?

bguala · 2020-02-10T01:32:38Z

Yes, this is my setup in the main CMakeLists:

This is my setup in CMakeLists located inside NPB3.3.1:

If I change the install_local_targets statement:

The build process fail!:

I never understood where they will go to stop if the benchmarks are compiled ??. In the directory you specify install_local_targets ???. In the bin directory???, If indeed left in bin, you can run in HermitCore???:

bguala · 2020-12-08T23:33:49Z

Hi Stefan

I'm writing to ask you some questions about HermitCore behavior that I don't understand. I am executing a matrix multiplication algorithm on this platform and I am also measuring its execution time with the time command. I did several tests increasing the size of the input until at one point I observe that the execution time stabilizes at approximately 40, 24, 14, 7 y 3 seconds beyond number of threads. The result of each test is reflected in the following table:

Cfg = CPU x Number of Threads

The output in each run is correct. Please could you explain the causes of this behavior to me? Why is the execution time stable if the input size and number of threads increases? How is thread scheduling done in the library operating system? In all cases I am using a virtual machine with 1GB of memory and 4 cores, how is the distribution of threads done in the most extreme case: Is it 16 threads?.

Thank you very much in avance.

Greetings.

MJChku · 2021-11-04T02:24:33Z

Want to add a general question, how does openmp work in your kernel?

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

NPB in HermitCore #104

NPB in HermitCore #104

bguala commented Oct 12, 2018 •

edited

Loading

stlankes commented Oct 12, 2018

bguala commented Oct 13, 2018 •

edited

Loading

stlankes commented Oct 13, 2018

bguala commented Oct 15, 2018

stlankes commented Oct 16, 2018

bguala commented Oct 16, 2018

stlankes commented Oct 18, 2018

bguala commented Oct 19, 2018

stlankes commented Jan 7, 2019

bguala commented Jan 9, 2019

stlankes commented Jan 10, 2019

stlankes commented Jan 15, 2019

bguala commented Jan 28, 2019

bguala commented Dec 1, 2019

stlankes commented Dec 3, 2019

bguala commented Dec 25, 2019

stlankes commented Jan 1, 2020

bguala commented Jan 2, 2020

bguala commented Feb 3, 2020

stlankes commented Feb 8, 2020

bguala commented Feb 10, 2020

bguala commented Dec 8, 2020

MJChku commented Nov 4, 2021

NPB in HermitCore #104

NPB in HermitCore #104

Comments

bguala commented Oct 12, 2018 • edited Loading

stlankes commented Oct 12, 2018

bguala commented Oct 13, 2018 • edited Loading

1) First Configuration

2) Second Configuration

stlankes commented Oct 13, 2018

bguala commented Oct 15, 2018

stlankes commented Oct 16, 2018

bguala commented Oct 16, 2018

stlankes commented Oct 18, 2018

bguala commented Oct 19, 2018

stlankes commented Jan 7, 2019

bguala commented Jan 9, 2019

stlankes commented Jan 10, 2019

stlankes commented Jan 15, 2019

bguala commented Jan 28, 2019

bguala commented Dec 1, 2019

stlankes commented Dec 3, 2019

bguala commented Dec 25, 2019

stlankes commented Jan 1, 2020

bguala commented Jan 2, 2020

bguala commented Feb 3, 2020

stlankes commented Feb 8, 2020

bguala commented Feb 10, 2020

bguala commented Dec 8, 2020

MJChku commented Nov 4, 2021

bguala commented Oct 12, 2018 •

edited

Loading

bguala commented Oct 13, 2018 •

edited

Loading