Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Apptainer file for arm64 Linux is incorrect #2120

Closed
edp8489 opened this issue Nov 27, 2023 · 0 comments · Fixed by #2121
Closed

Apptainer file for arm64 Linux is incorrect #2120

edp8489 opened this issue Nov 27, 2023 · 0 comments · Fixed by #2121

Comments

@edp8489
Copy link
Contributor

edp8489 commented Nov 27, 2023

Describe the bug

The Apptainer definition file for arm64 Linux is (1) improperly defined and (2) inefficient. As defined, it fails during the call to starter/build.sh because it can't find the specified compilers in the PATH. Issues and solutions are described below.

1. Missing dependencies

  • which is provided as a Bash alias in the Rocky image and doesn't behave as expected when called from the build scripts.

  • python is not installed, either in the base Rocky image or with the remaining dependencies. By default it is used in the build scripts and not disabled.

2. ARM compilers

  • ARM compiler suite and performance libraries are huge downloads (~2 GB). Given the frequent (sometimes 2-3x per week) updates of OpenRadioss, this is a drain on build time.
  • Additionally, the container definition does not clean up the downloaded and expanded files after installation, leaving a large (~4 GB) footprint in the image that is not necessary.

Example build error

...

+ cd /opt/OpenRadioss/starter
+ ./build_script.sh -arch=linuxa64 -nt=80
 
 Build OpenRadioss Starter 
 --------------------------
 Build Arguments :
 arch =                 :  linuxa64
 precision =            :  dp
 debug =                :  0
 static_link =          :  0
 
 Executable name        :  starter_linuxa64
 
 #threads for Makefile :  80
 
 
./build_script.sh: line 251: which: command not found
./build_script.sh: line 252: which: command not found
./build_script.sh: line 253: which: command not found
./build_script.sh: line 254: which: command not found

...

-- Build files have been written to: /opt/OpenRadioss/engine/cbuild_engine_linuxa64
[  0%] Generating CMakeFiles/includes_engine_linuxa64/engine_message_description.inc
[  0%] Generating CMakeFiles/includes_engine_linuxa64/__foo.inc, CMakeFiles/includes_engine_linuxa64/build_info.inc
make[2]: python: No such file or directory
make[2]: *** [CMakeFiles/engine_linuxa64.dir/build.make:74: CMakeFiles/includes_engine_linuxa64/engine_message_description.inc] Error 127
make[2]: *** Waiting for unfinished jobs....
make[2]: python: No such file or directory
make[2]: *** [CMakeFiles/engine_linuxa64.dir/build.make:78: CMakeFiles/includes_engine_linuxa64/__foo.inc] Error 127
make[1]: *** [CMakeFiles/Makefile2:82: CMakeFiles/engine_linuxa64.dir/all] Error 2
make: *** [Makefile:136: all] Error 2

Solution

1. Fix dependency and build issues

  • which and python3 added to list of packages installed via final dnf command.

  • A symlink pointing /usr/local/bin/python --> /usr/local/bin/python3 is added.

2. Clean up definition files

  • A separate definition file arm_buildEnv.def is provided to download and install all prerequisites into a base image. This will only occur once.
    • The following options are added to the wget command:
      • --show-progress so user doesn't think progress has stalled when downloading the Arm compiler suite.
      • -c --read-timeout=15 to restart download after 15 seconds of inactivity and continue from the partially-downloaded file.
    • rm ... commands are added to clean up the Arm compiler and performance library intermediate files (*.tar downloads and unpacked *.sh installers) left behind after installation is complete.
  • openradioss_arm.def definition file is now bootstrapped from the local build environment image (arm_buildEnv.sif) and ONLY downloads the OpenRadioss source. Libraries are static-linked so binaries can be copied to an archive and distributed externally.
  • A build script build_arm.sh is provided that will first check if the build environment image exists and create it if necessary.
  • Definition files are moved to Apptainer\arm\ directory.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant