{devel}[system] ReFrame v3.3 #11787

teojgo · 2020-11-30T10:25:48Z

boegel · 2020-11-30T16:15:58Z

@boegelbot please test @ generoso

boegelbot · 2020-11-30T16:20:33Z

@boegel: Request for testing this PR well received on generoso

PR test command 'EB_PR=11787 EB_ARGS= /apps/slurm/default/bin/sbatch --job-name test_PR_11787 --ntasks=4 ~/boegelbot/eb_from_pr_upload_generoso.sh' executed!

exit code: 0
output:

Submitted batch job 11578

Test results coming soon (I hope)...

- notification for comment with ID 735885925 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

boegelbot · 2020-11-30T16:21:59Z

Test report by @boegelbot
FAILED
Build succeeded for 0 out of 1 (1 easyconfigs in total)
generoso-c2-s-1 - Linux centos linux 8.2.2004, x86_64, Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz (haswell), Python 3.6.8
See https://gist.github.com/1a079cb94c7c300db41fa829d7bede48 for a full test report.

teojgo · 2020-11-30T16:26:04Z

@boegel let me try adding also a newer setuptools that supports finding the namespace packages.

boegel · 2020-11-30T16:36:31Z

@teojgo Please make sure this easyconfig file works for you (you can test with eb --from-pr 11787).

Note that EasyBuild won't be happy if pip is pulling in stuff from PyPI during the installation...

teojgo · 2020-11-30T16:38:41Z

@teojgo Please make sure this easyconfig file works for you (you can test with eb --from-pr 11787).

Note that EasyBuild won't be happy if pip is pulling in stuff from PyPI during the installation...

Yeah, I have checked before pushing.

boegel · 2020-12-01T09:18:06Z

@boegelbot please test @ generoso

boegel · 2020-12-01T09:19:17Z

easybuild/easyconfigs/r/ReFrame/ReFrame-3.3.eb

+]
+
+postinstallcmds = [
+    "cd %(builddir)s/reframe/reframe-%(version)s && ./bootstrap.sh +pygelf && "


@teojgo Please make this two separate commands (easier to debug if something goes wrong), and add a comment to explain what the bootstrap script does, and why it's used).

boegelbot · 2020-12-01T09:20:28Z

@boegel: Request for testing this PR well received on generoso

PR test command 'EB_PR=11787 EB_ARGS= /apps/slurm/default/bin/sbatch --job-name test_PR_11787 --ntasks=4 ~/boegelbot/eb_from_pr_upload_generoso.sh' executed!

exit code: 0
output:

Submitted batch job 12170

Test results coming soon (I hope)...

- notification for comment with ID 736335824 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

boegelbot · 2020-12-01T09:22:11Z

Test report by @boegelbot
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in total)
generoso-c1-s-1 - Linux centos linux 8.2.2004, x86_64, Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz (haswell), Python 3.6.8
See https://gist.github.com/5ce8595f29479d93e8e7e3e354523f4b for a full test report.

easybuild/easyconfigs/r/ReFrame/ReFrame-3.3.eb

boegel · 2020-12-01T16:54:39Z

@boegelbot please test @ generoso

boegelbot · 2020-12-01T16:55:46Z

@boegel: Request for testing this PR well received on generoso

PR test command 'EB_PR=11787 EB_ARGS= /apps/slurm/default/bin/sbatch --job-name test_PR_11787 --ntasks=4 ~/boegelbot/eb_from_pr_upload_generoso.sh' executed!

exit code: 0
output:

Submitted batch job 12174

Test results coming soon (I hope)...

- notification for comment with ID 736679892 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

boegelbot · 2020-12-01T16:59:09Z

Test report by @boegelbot
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in total)
generoso-c1-s-1 - Linux centos linux 8.2.2004, x86_64, Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz (haswell), Python 3.6.8
See https://gist.github.com/53ed85385ff5b80cde7ac3e31f2799f2 for a full test report.

boegel · 2020-12-05T14:00:07Z

Test report by @boegel
FAILED
Build succeeded for 0 out of 1 (1 easyconfigs in total)
boegel.ugent.be - Darwin UNKNOWN UNKNOWN, x86_64, Intel(R) Core(TM) i5-7360U CPU @ 2.30GHz, Python 3.9.0
See https://gist.github.com/7cab0668b20d788ced5c45c19f7449a3 for a full test report.

boegel · 2020-12-05T14:28:48Z

Test report by @boegel
FAILED
Build succeeded for 0 out of 1 (1 easyconfigs in total)
node3575.doduo.os - Linux RHEL 8.2, x86_64, AMD EPYC 7552 48-Core Processor (zen2), Python 3.6.8
See https://gist.github.com/a0354927d653d24aecfb2981c61be01a for a full test report.

boegel · 2020-12-05T14:30:04Z

easybuild/easyconfigs/r/ReFrame/ReFrame-3.3.eb

+    # Use bootstrap script to install ReFrame dependencies under `external`.
+    # ReFrame adds `external` to its own python path before executing, without
+    # affecting the python path of the system
+    './bootstrap.sh +docs +pygelf', 'cp -r external %(installdir)s',


@teojgo Should this be done via preinstallopts in the reframe extension above, to avoid the installation problems popping up in the test reports (the installation of reframe fails because jsonschema is not available)?

boegel · 2020-12-08T22:03:40Z

Test report by @boegel
FAILED
Build succeeded for 0 out of 1 (1 easyconfigs in total)
node3504.doduo.os - Linux RHEL 8.2, x86_64, AMD EPYC 7552 48-Core Processor (zen2), Python 3.6.8
See https://gist.github.com/80f9268788c9754e25a7baef3348f7ee for a full test report.

boegel · 2020-12-08T22:03:56Z

Test report by @boegel
FAILED
Build succeeded for 0 out of 1 (1 easyconfigs in total)
Kenneths-MacBook-Pro.local - Darwin UNKNOWN UNKNOWN, x86_64, Intel(R) Core(TM) i5-7360U CPU @ 2.30GHz, Python 3.9.0
See https://gist.github.com/cf09a5b0fd20643ce6a8767c5b6a8a42 for a full test report.

verdurin · 2021-01-04T12:15:35Z

@boegel are you happy with the current state of this one?

teojgo · 2021-01-04T12:37:46Z

Although the dependencies are installed by the bootstrap script, since the setup.py contains the corresponding dependencies, the installation fails in certain cases. Do you thing I should add everything to the exts_list?

boegel · 2021-01-05T10:31:13Z

@teojgo It looks like the bootstrap script isn't installed everything that is needed?

This seems to occur when installing the docs part:

ModuleNotFoundError: No module named 'jsonschema'

teojgo · 2021-01-05T10:37:20Z

@teojgo It looks like the bootstrap script isn't installed everything that is needed?

This seems to occur when installing the docs part:
ModuleNotFoundError: No module named 'jsonschema'

It does install jsonschema under the external directory which is added to the PYTHONPATH before making the docs. It seems though that because of the setup.py which lists jsonschema as a required module, it fails to find it.

…otstrap script installs required Python packages there

boegel · 2021-01-05T19:48:47Z

@teojgo During the installation of the docs it's actually OK, but the external subdirectory is not being added to $PYTHONPATH outside of the bootstrap.sh script.

I've fixed this in teojgo#2, and then the installation works fine...

add 'external' subdirectory to $PYTHONPATH for ReFrame v3.3, since bootstrap script installs required Python packages there

vkarak · 2021-01-07T11:13:39Z

@teojgo It looks like the bootstrap script isn't installed everything that is needed?

This seems to occur when installing the docs part:
ModuleNotFoundError: No module named 'jsonschema'

I can't understand how this is happening. ReFrame sets the sys.path internally, I don't know why it misses that. @teojgo any ideas?

teojgo · 2021-01-07T11:32:44Z

@teojgo It looks like the bootstrap script isn't installed everything that is needed?
This seems to occur when installing the docs part:
ModuleNotFoundError: No module named 'jsonschema'
I can't understand how this is happening. ReFrame sets the sys.path internally, I don't know why it misses that. @teojgo any ideas?

On the system I was testing jsonschema was installed system-wide so I could not reproduce the problem. While the bootstrap script and reframe take into account the external directory, when trying to install reframe using the setup.py script, jsonschema was not found. That's why what @boegel added makes the setup.py script aware of the packages installed using the bootstrap script.

boegel · 2021-01-09T09:39:24Z

@boegelbot please test @ generoso

boegelbot · 2021-01-09T09:40:48Z

@boegel: Request for testing this PR well received on generoso

PR test command 'EB_PR=11787 EB_ARGS= /apps/slurm/default/bin/sbatch --job-name test_PR_11787 --ntasks=4 ~/boegelbot/eb_from_pr_upload_generoso.sh' executed!

exit code: 0
output:

Submitted batch job 12401

Test results coming soon (I hope)...

- notification for comment with ID 757123617 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

boegelbot · 2021-01-09T09:43:24Z

Test report by @boegelbot
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in total)
generoso-x-3 - Linux centos linux 8.2.2004, x86_64, Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz (haswell), Python 3.6.8
See https://gist.github.com/871958cd640cad2235bd39e3f375056d for a full test report.

boegel · 2021-01-09T10:36:58Z

I also tried just adding the external subdir to $PYTHONPATH when doing the installation, but that's not good enough because then the pip check that we run as a part of the sanity check fails with:

reframe-hpc 3.3 requires jsonschema, which is not installed.

boegel · 2021-01-09T10:39:09Z

Test report by @boegel
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in total)
node2602.swalot.os - Linux centos linux 7.9.2009, x86_64, Intel(R) Xeon(R) CPU E5-2660 v3 @ 2.60GHz (haswell), Python 3.6.8
See https://gist.github.com/40ec32f9138ba58397c3f6532e5e7998 for a full test report.

boegel · 2021-01-09T10:41:52Z

Test report by @boegel
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in total)
node3106.skitty.os - Linux centos linux 7.9.2009, x86_64, Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz, Python 3.6.8
See https://gist.github.com/d91d2d658bb52fee3936322252977447 for a full test report.

boegel · 2021-01-09T11:07:08Z

Going in, thanks @teojgo!

vkarak · 2021-01-12T08:02:53Z

Thanks @boegel and @teojgo !

Add easyconfig for ReFrame 3.3

1fcc25b

boegel changed the title ~~Add easyconfig for ReFrame 3.3~~ {devel}[system] ReFrame v3.3 Nov 30, 2020

boegel added the update label Nov 30, 2020

boegel added this to the 4.x milestone Nov 30, 2020

Add 'setuptools' to the extensions list

fe54297

boegel requested changes Dec 1, 2020

View reviewed changes

Add comment regarding the bootstrap script

71b5a19

vkarak reviewed Dec 1, 2020

View reviewed changes

easybuild/easyconfigs/r/ReFrame/ReFrame-3.3.eb Outdated Show resolved Hide resolved

vkarak reviewed Dec 1, 2020

View reviewed changes

easybuild/easyconfigs/r/ReFrame/ReFrame-3.3.eb Outdated Show resolved Hide resolved

Generate the man pages

8ce55fe

boegel requested changes Dec 5, 2020

View reviewed changes

boegel modified the milestones: 4.x, 4.3.2 (next release) Dec 6, 2020

Execute bootstrap script in preinstallopts of ReFrame

dea5de2

boegel modified the milestones: 4.3.2 (next release), 4.4.0 Dec 8, 2020

add 'external' subdirectory to $PYTHONPATH for ReFrame v3.3, since bo…

f590559

…otstrap script installs required Python packages there

boegel mentioned this pull request Jan 5, 2021

add 'external' subdirectory to $PYTHONPATH for ReFrame v3.3, since bootstrap script installs required Python packages there teojgo/easybuild-easyconfigs#2

Merged

Merge pull request #2 from boegel/reframe/3.3

80be26c

add 'external' subdirectory to $PYTHONPATH for ReFrame v3.3, since bootstrap script installs required Python packages there

boegel approved these changes Jan 9, 2021

View reviewed changes

boegel merged commit dca6b85 into easybuilders:develop Jan 9, 2021

teojgo mentioned this pull request Jan 18, 2021

Update official EB recipe to use the ./bootstrap.sh script reframe-hpc/reframe#1615

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

{devel}[system] ReFrame v3.3 #11787

{devel}[system] ReFrame v3.3 #11787

teojgo commented Nov 30, 2020

boegel commented Nov 30, 2020

boegelbot commented Nov 30, 2020

boegelbot commented Nov 30, 2020

teojgo commented Nov 30, 2020 •

edited

boegel commented Nov 30, 2020

teojgo commented Nov 30, 2020

boegel commented Dec 1, 2020

boegel Dec 1, 2020

boegelbot commented Dec 1, 2020

boegelbot commented Dec 1, 2020

boegel commented Dec 1, 2020

boegelbot commented Dec 1, 2020

boegelbot commented Dec 1, 2020

boegel commented Dec 5, 2020

boegel commented Dec 5, 2020

boegel Dec 5, 2020

boegel commented Dec 8, 2020

boegel commented Dec 8, 2020

verdurin commented Jan 4, 2021

teojgo commented Jan 4, 2021

boegel commented Jan 5, 2021

teojgo commented Jan 5, 2021

boegel commented Jan 5, 2021

vkarak commented Jan 7, 2021

teojgo commented Jan 7, 2021

boegel commented Jan 9, 2021

boegelbot commented Jan 9, 2021

boegelbot commented Jan 9, 2021

boegel commented Jan 9, 2021

boegel commented Jan 9, 2021

boegel commented Jan 9, 2021

boegel commented Jan 9, 2021

vkarak commented Jan 12, 2021

{devel}[system] ReFrame v3.3 #11787

{devel}[system] ReFrame v3.3 #11787

Conversation

teojgo commented Nov 30, 2020

boegel commented Nov 30, 2020

boegelbot commented Nov 30, 2020

boegelbot commented Nov 30, 2020

teojgo commented Nov 30, 2020 • edited

boegel commented Nov 30, 2020

teojgo commented Nov 30, 2020

boegel commented Dec 1, 2020

boegel Dec 1, 2020

Choose a reason for hiding this comment

boegelbot commented Dec 1, 2020

boegelbot commented Dec 1, 2020

boegel commented Dec 1, 2020

boegelbot commented Dec 1, 2020

boegelbot commented Dec 1, 2020

boegel commented Dec 5, 2020

boegel commented Dec 5, 2020

boegel Dec 5, 2020

Choose a reason for hiding this comment

boegel commented Dec 8, 2020

boegel commented Dec 8, 2020

verdurin commented Jan 4, 2021

teojgo commented Jan 4, 2021

boegel commented Jan 5, 2021

teojgo commented Jan 5, 2021

boegel commented Jan 5, 2021

vkarak commented Jan 7, 2021

teojgo commented Jan 7, 2021

boegel commented Jan 9, 2021

boegelbot commented Jan 9, 2021

boegelbot commented Jan 9, 2021

boegel commented Jan 9, 2021

boegel commented Jan 9, 2021

boegel commented Jan 9, 2021

boegel commented Jan 9, 2021

vkarak commented Jan 12, 2021

teojgo commented Nov 30, 2020 •

edited