Reframe on Isambard bootstrapping error related with urllib package version

[kaanolgu]

I have installed the ReFrame with Spack and trying to run it initially but it gives me the following error message :

[br-kolgu@login-01 reframe-4.0.4-aywne5pbfdemxvulmokilxvtpjzu6oo4]$ ./bootstrap.sh
==> python3 -m ensurepip --root /home/br-kolgu/excalibur-tests/spack/isambard-cascadelake/compute-node/opt/cray-rhel8-broadwell/gcc-9.2.0/reframe-4.0.4-aywne5pbfdemxvulmokilxvtpjzu6oo4/external/ --default-pip
Requirement already satisfied: setuptools in /usr/lib/python3.6/site-packages
Requirement already satisfied: pip in /usr/lib/python3.6/site-packages
==> python3 -m pip install --no-cache-dir -q --upgrade pip --target=external/
==> python3 -m pip install --no-cache-dir -q -r requirements.txt --target=external/ --upgrade
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
botocore 1.18.15 requires urllib3<1.26,>=1.20, but you have urllib3 1.26.15 which is incompatible.

[teojgo]

@kaanolgu can you try the bootstrapping with --pip-opts '--ignore-installed' ?

[kaanolgu]

Hi @teojgo it gives me the same error message

[teojgo]

@kaanolgu can you try with /bootstrap.sh --pip-opts '--no-deps' ?

[kaanolgu]

It works now but gives me the error message when I try to run the basic example

[br-kolgu@login-01 reframe-4.0.4-aywne5pbfdemxvulmokilxvtpjzu6oo4]$ ./bootstrap.sh --pip-opts '--no-deps'
==> python3 -m ensurepip --root /home/br-kolgu/excalibur-tests/spack/isambard-cascadelake/compute-node/opt/cray-rhel8-broadwell/gcc-9.2.0/reframe-4.0.4-aywne5pbfdemxvulmokilxvtpjzu6oo4/external/ --default-pip
Requirement already satisfied: setuptools in /usr/lib/python3.6/site-packages
Requirement already satisfied: pip in /usr/lib/python3.6/site-packages
==> python3 -m pip install --no-cache-dir -q --upgrade pip --target=external/
==> python3 -m pip install --no-cache-dir -q -r requirements.txt --target=external/ --upgrade --no-deps
[br-kolgu@login-01 reframe-4.0.4-aywne5pbfdemxvulmokilxvtpjzu6oo4]$ ./bin/reframe -c tutorials/basics/hello/hello1.py -r
[ReFrame Setup]
  version:           4.0.4
  command:           './bin/reframe -c tutorials/basics/hello/hello1.py -r'
  launched by:       br-kolgu@login-01
  working directory: '/lustre/home/br-kolgu/excalibur-tests/spack/isambard-cascadelake/compute-node/opt/cray-rhel8-broadwell/gcc-9.2.0/reframe-4.0.4-aywne5pbfdemxvulmokilxvtpjzu6oo4'
  settings files:    '<builtin>'
  check search path: '/lustre/home/br-kolgu/excalibur-tests/spack/isambard-cascadelake/compute-node/opt/cray-rhel8-broadwell/gcc-9.2.0/reframe-4.0.4-aywne5pbfdemxvulmokilxvtpjzu6oo4/tutorials/basics/hello/hello1.py'
  stage directory:   '/lustre/home/br-kolgu/excalibur-tests/spack/isambard-cascadelake/compute-node/opt/cray-rhel8-broadwell/gcc-9.2.0/reframe-4.0.4-aywne5pbfdemxvulmokilxvtpjzu6oo4/stage'
  output directory:  '/lustre/home/br-kolgu/excalibur-tests/spack/isambard-cascadelake/compute-node/opt/cray-rhel8-broadwell/gcc-9.2.0/reframe-4.0.4-aywne5pbfdemxvulmokilxvtpjzu6oo4/output'
  log files:         '/tmp/rfm-tiqtk4ls.log'

[==========] Running 1 check(s)
[==========] Started on Wed Apr 12 12:43:42 2023 

[----------] start processing checks
[ RUN      ] HelloTest /2b3e4546 @generic:default+builtin
[     FAIL ] (1/1) HelloTest /2b3e4546 @generic:default+builtin
==> test failed during 'compile_wait': test staged in '/lustre/home/br-kolgu/excalibur-tests/spack/isambard-cascadelake/compute-node/opt/cray-rhel8-broadwell/gcc-9.2.0/reframe-4.0.4-aywne5pbfdemxvulmokilxvtpjzu6oo4/stage/generic/default/builtin/HelloTest'
[----------] all spawned checks have finished

[  FAILED  ] Ran 1/1 test case(s) from 1 check(s) (1 failure(s), 0 skipped)
[==========] Finished on Wed Apr 12 12:43:43 2023 

===========================================================================================================================================================================================
SUMMARY OF FAILURES
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
FAILURE INFO for HelloTest 
  * Expanded name: HelloTest
  * Description: 
  * System partition: generic:default
  * Environment: builtin
  * Stage directory: /lustre/home/br-kolgu/excalibur-tests/spack/isambard-cascadelake/compute-node/opt/cray-rhel8-broadwell/gcc-9.2.0/reframe-4.0.4-aywne5pbfdemxvulmokilxvtpjzu6oo4/stage/generic/default/builtin/HelloTest
  * Node list: 
  * Job type: local (id=None)
  * Dependencies (conceptual): []
  * Dependencies (actual): []
  * Maintainers: []
  * Failing phase: compile_wait
  * Rerun with '-n /2b3e4546 -p builtin --system generic:default -r'
  * Reason: build error: stdout: 'rfm_build.out', stderr: 'rfm_build.err'
--- rfm_build.err (first 10 lines) ---
ld.lld: error: /opt/cray/pe/libsci/20.03.1/CRAY/8.5/x86_64/lib/libsci_cray_mpi.so: undefined reference to MPI_Bcast [--no-allow-shlib-undefined]
ld.lld: error: /opt/cray/pe/libsci/20.03.1/CRAY/8.5/x86_64/lib/libsci_cray_mpi.so: undefined reference to MPI_Op_create [--no-allow-shlib-undefined]
ld.lld: error: /opt/cray/pe/libsci/20.03.1/CRAY/8.5/x86_64/lib/libsci_cray_mpi.so: undefined reference to MPI_Pack_size [--no-allow-shlib-undefined]
ld.lld: error: /opt/cray/pe/libsci/20.03.1/CRAY/8.5/x86_64/lib/libsci_cray_mpi.so: undefined reference to MPI_Send [--no-allow-shlib-undefined]
ld.lld: error: /opt/cray/pe/libsci/20.03.1/CRAY/8.5/x86_64/lib/libsci_cray_mpi.so: undefined reference to MPI_Type_struct [--no-allow-shlib-undefined]
ld.lld: error: /opt/cray/pe/libsci/20.03.1/CRAY/8.5/x86_64/lib/libsci_cray_mpi.so: undefined reference to MPI_Isend [--no-allow-shlib-undefined]
ld.lld: error: /opt/cray/pe/libsci/20.03.1/CRAY/8.5/x86_64/lib/libsci_cray_mpi.so: undefined reference to MPI_Unpack [--no-allow-shlib-undefined]
ld.lld: error: /opt/cray/pe/libsci/20.03.1/CRAY/8.5/x86_64/lib/libsci_cray_mpi.so: undefined reference to MPI_Cancel [--no-allow-shlib-undefined]
ld.lld: error: /opt/cray/pe/libsci/20.03.1/CRAY/8.5/x86_64/lib/libsci_cray_mpi.so: undefined reference to MPI_Initialized [--no-allow-shlib-undefined]
ld.lld: error: /opt/cray/pe/libsci/20.03.1/CRAY/8.5/x86_64/lib/libsci_cray_mpi.so: undefined reference to MPI_Type_free [--no-allow-shlib-undefined]
--- rfm_build.err --- 
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Run report saved in '/home/br-kolgu/.reframe/reports/run-report-3.json'
Log file(s) saved in '/tmp/rfm-tiqtk4ls.log'
[br-kolgu@login-01 reframe-4.0.4-aywne5pbfdemxvulmokilxvtpjzu6oo4]$ 

[teojgo]

@kaanolgu that's different. I suggest to close this and you can join the Slack channel so that it's easier to debug it.

[kaanolgu]

@teojgo I was planning to ask there but the link is broken for slack channel

Application error
An error occurred in the application and your page could not be served. If you are the application owner, check your logs for details. You can do this from the Heroku CLI with the command

[teojgo]

@teojgo I was planning to ask there but the link is broken for slack channel

Application error An error occurred in the application and your page could not be served. If you are the application owner, check your logs for details. You can do this from the Heroku CLI with the command

Ok, I will check with the other ReFrame developers and come back to you.

[vkarak]

Yes, we just recently realised that the Slack link was broken! This is fixed in #2862. Please use the link in that PR to join!

[vkarak]

As for the error here, when you install reframe through spack, you should not run the ./bootstrap.sh script. All the dependencies are installed through Spack, so the reframe command should just work.