Improve the server launch scripts to handle launches on PBS #4088
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
1RyanK
approved these changes
Feb 13, 2025
e-kayrakli
added a commit
to chapel-lang/chapel
that referenced
this pull request
Feb 15, 2025
... and, while there, also improve debugging/maintenance capabilities. Commits in this PR should be squashed. - adds `log` function to be able to print things from Arkouda's `sub_test` - use `ARKOUDA_SERVER_LAUNCH_PREFIX` to wire in `chpl_launchcmd` in the server launch command. This will be supported by Bears-R-Us/arkouda#4088 - Add the following environment variables to disable parts of Arkouda testing. This has been very valuable during development, where I could simply make the Jenkins use an existing Arkouda build to reduce the turnaround time significantly: - `CHPL_TEST_ARKOUDA_SKIP_RM_CLONE` - `CHPL_TEST_ARKOUDA_SKIP_CHAPEL_PY` - `CHPL_TEST_ARKOUDA_SKIP_BUILD` - `CHPL_TEST_ARKOUDA_SKIP_MAKE_CHECK` - `CHPL_TEST_ARKOUDA_SKIP_UNIT_TESTS` (this will be used by the new performance testing to avoid unit testing in those configs) - While there, this also adds a `-nobuild` flag to `nightly` to avoid rebuilding Chapel in each run. Obviously, this is for maintenance only. - In total, this PR adds 4 jobs: - `util/cron/test-perf.hpe-apollo-hdr.arkouda.bash` - `util/cron/test-perf.hpe-apollo-hdr.arkouda.release.bash` - `util/cron/test.hpe-apollo-hdr.arkouda.bash` - `util/cron/test.hpe-apollo-hdr.arkouda.release.bash` Corresponding Jenkins config PR: https://github.hpe.com/hpe/hpc-chapel-ci-config/pull/1361 [Reviewed by @ShreyasKhandekar]
e-kayrakli
force-pushed
the
server-util-pbs
branch
from
2dd6a61 to
9325ff0
Compare
ajpotts
approved these changes
Feb 24, 2025
jaketrookman
approved these changes
Feb 24, 2025
Contributor
jaketrookman
left a comment
jaketrookman
left a comment
There was a problem hiding this comment.
Looks good to be merged once the flake8 issue is fixed
auto-merge was automatically disabled
February 24, 2025 19:46
Head branch was pushed to by a user without write access
Signed-off-by: Engin Kayraklioglu <e-kayrakli@users.noreply.github.com>
e-kayrakli
force-pushed
the
server-util-pbs
branch
from
5f6399f to
8584aa4
Compare
ajpotts
approved these changes
Feb 26, 2025
e-kayrakli
added a commit
to chapel-lang/chapel
that referenced
this pull request
Mar 1, 2025
Bears-R-Us/arkouda#4088 was merged in couple of days ago. Now we can start testing the `master` Arkouda branch in the new configs. This PR removes specific fork/branch from testing scripts. [Trivial, not reviewed]
github-merge-queue bot
pushed a commit
that referenced
this pull request
Mar 6, 2025
#4088 added the capability dump the server's output into a log file during testing. I realized that you can also do `--logChannel=LogChannel.FILE` to achieve the same thing through the server itself. That's more maintainable and robust. This PR reverts that change from the PR above and uses the flag above instead. Signed-off-by: Engin Kayraklioglu <e-kayrakli@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR improves the server launch scripts in order to be able to run on PBS systems in nightly testing. It includes two changes
ARKOUDA_SERVER_LAUNCH_PREFIXenvironment can be used to "inject" things before./arkouda_server: this will allow us to usechpl_launchcmdtesting script and pass some arguments to it.-nlis passed: Space-separated things has to be passed as seperate items in the list when constructing the command.chpl_launchcmdis sensitive to hownlargument is used, and was getting tripped up if that argument is not passed properly.I manually tested this on a PBS system. Next week, I am hoping to wire nightly testing configs and test through that before merging this.