Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Nexus: Pass PYTHONPATH recorded at cmake step to nxs-test. #3935

Merged
merged 1 commit into from
Apr 7, 2022

Conversation

ye-luo
Copy link
Contributor

@ye-luo ye-luo commented Apr 7, 2022

Proposed changes

I can reproduce nxs-test issue of sulfur on my workstation.
Both python interpreter and PYTHONPATH must be properly propagated to all the cmake to python and python internal calls.
Previous issues

  1. nxs-test --pythonpath wipes out necessary PYTHONPATH
  2. All nexus internal calls to python scripts doesn't enforce parent python interpreter. There might be more fixes needed. I only found those covered by ctest.

With this PR, at the CMake step, recorded python environment will be used for ctest. Even the environment is changed after cmake, ctest runs in the recorded environment. Updating test environment requires rerun cmake.

What type(s) of changes does this code introduce?

  • Bugfix

Does this introduce a breaking change?

  • No

What systems has this change been tested on?

epyc-server

Checklist

  • Yes. This PR is up to date with current the current state of 'develop'

@ye-luo ye-luo requested review from prckent and jtkrogel April 7, 2022 18:54
Copy link
Contributor

@prckent prckent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quick gcc build on sulfur shows that this fixes the persistent problems. i.e. Cures about 12 x number of builds failures on cdash. Thanks Ye. Note that I only ran the PR and I did not search for any "missed" updates in nexus/our cmake.

Output of ctest -L deterministic below.

Before:

99% tests passed, 12 tests failed out of 1139

Label Time Summary:
QMCPACK                     = 269.25 sec*proc (62 tests)
QMCPACK-checking-results    =  52.83 sec*proc (893 tests)
converter                   =  14.54 sec*proc (20 tests)
deterministic               = 532.33 sec*proc (1139 tests)
nexus                       =  60.45 sec*proc (70 tests)
quality_unknown             = 507.86 sec*proc (1104 tests)
unit                        =  36.48 sec*proc (45 tests)

Total Test time (real) =  38.93 sec

The following tests FAILED:
        2206 - ntest_nexus_nxs_sim (Failed)
        2207 - ntest_nexus_qmca (Failed)
        2208 - ntest_nexus_qmc_fit (Failed)
        2209 - ntest_nexus_qdens (Failed)
        2211 - ntest_nexus_example_gamess_H2O (Failed)
        2212 - ntest_nexus_example_pwscf_relax_Ge_T (Failed)
        2213 - ntest_nexus_example_qmcpack_H2O (Failed)
        2214 - ntest_nexus_example_qmcpack_LiH (Failed)
        2215 - ntest_nexus_example_qmcpack_c20 (Failed)
        2216 - ntest_nexus_example_qmcpack_diamond (Failed)
        2217 - ntest_nexus_example_qmcpack_graphene (Failed)
        2218 - ntest_nexus_example_qmcpack_oxygen_dimer (Failed)
Errors while running CTest

After:

100% tests passed, 0 tests failed out of 1139

Label Time Summary:
QMCPACK                     = 271.16 sec*proc (62 tests)
QMCPACK-checking-results    =  53.31 sec*proc (893 tests)
converter                   =  14.81 sec*proc (20 tests)
deterministic               = 556.09 sec*proc (1139 tests)
nexus                       =  79.94 sec*proc (70 tests)
quality_unknown             = 531.27 sec*proc (1104 tests)
unit                        =  37.02 sec*proc (45 tests)

Total Test time (real) =  43.39 sec

@prckent prckent changed the title Pass PYTHONPATH recorded at cmake step to nxs-test. Nexus: Pass PYTHONPATH recorded at cmake step to nxs-test. Apr 7, 2022
@ye-luo
Copy link
Contributor Author

ye-luo commented Apr 7, 2022

Test this please

@ye-luo ye-luo enabled auto-merge April 7, 2022 20:41
@ye-luo ye-luo merged commit 6edddf6 into QMCPACK:develop Apr 7, 2022
@ye-luo ye-luo deleted the fix-test-nexus branch April 24, 2022 00:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants