Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Three bench_test failures #322

Open
swryan opened this issue Jun 12, 2024 · 4 comments
Open

Three bench_test failures #322

swryan opened this issue Jun 12, 2024 · 4 comments
Labels
bug Something isn't working

Comments

@swryan
Copy link
Contributor

swryan commented Jun 12, 2024 •
edited
Loading

Description

There are 3 bench_test failures when running on the cluster:

The following tests failed:
test_FLOPS_based_sizing_N3CC.py:ProblemPhaseTestCase.bench_test_sizing_N3CC
test_NPSS_builder.py:AviaryNPSSTestCase.bench_test_aviary_NPSS
test_cmd_entry_points.py:CommandEntryPointsTestCases.bench_test_SNOPT_cmd


Passed:  11
Failed:  3
Skipped: 0

Example

testflo -vs aviary/interface/test/test_cmd_entry_points.py:CommandEntryPointsTestCases.bench_test_SNOPT_cmd --testmatch=bench_test*

 aviary run_mission aviary/models/test_aircraft/aircraft_for_bench_GwGm.csv --optimizer SNOPT --max_iter 1
Traceback (most recent call last):
  File "/home/swryan/miniconda3/envs/aviary/bin/aviary", line 8, in <module>
    sys.exit(aviary_cmd())
             ^^^^^^^^^^^^
  File "/home/swryan/dev/Aviary/aviary/interface/cmd_entry_points.py", line 122, in aviary_cmd
    options.executor(options, user_args)
  File "/home/swryan/dev/Aviary/aviary/interface/methods_for_level1.py", line 203, in _exec_level1
    prob = run_level_1(
           ^^^^^^^^^^^^
  File "/home/swryan/dev/Aviary/aviary/interface/methods_for_level1.py", line 135, in run_level_1
    prob = run_aviary(input_deck, phase_info, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/swryan/dev/Aviary/aviary/interface/methods_for_level1.py", line 70, in run_aviary
    prob.load_inputs(aircraft_filename, phase_info, verbosity=verbosity)
  File "/home/swryan/dev/Aviary/aviary/interface/methods_for_level2.py", line 306, in load_inputs
    phase_info, _ = phase_info_parameterization(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnboundLocalError: cannot access local variable 'phase_info_parameterization' where it is not associated with a value

testflo -vs aviary/validation_cases/benchmark_tests/test_FLOPS_based_sizing_N3CC.py --testmatch=bench_test*

aviary/validation_cases/benchmark_tests/test_FLOPS_based_sizing_N3CC.py:ProblemPhaseTestCase.bench_test_sizing_N3CC  ... FAIL (00:00:3.72, 284 MB)

[ ... a bunch of warnings deleted ... ]

Traceback (most recent call last):
  File "/home/swryan/miniconda3/envs/aviary/lib/python3.12/site-packages/openmdao/solvers/linear/direct.py", line 320, in _linearize
    self._lu = scipy.sparse.linalg.splu(matrix)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/swryan/miniconda3/envs/aviary/lib/python3.12/site-packages/scipy/sparse/linalg/_dsolve/linsolve.py", line 437, in splu
    return _superlu.gstrf(N, A.nnz, A.data, indices, indptr,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Factor is exactly singular

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/swryan/dev/Aviary/aviary/validation_cases/benchmark_tests/test_FLOPS_based_sizing_N3CC.py", line 445, in bench_test_sizing_N3CC
    prob = run_trajectory(sim=False)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/swryan/dev/Aviary/aviary/validation_cases/benchmark_tests/test_FLOPS_based_sizing_N3CC.py", line 427, in run_trajectory
    dm.run_problem(prob, simulate=sim, make_plots=False, simulate_kwargs={
  File "/home/swryan/miniconda3/envs/aviary/lib/python3.12/site-packages/dymos/run_problem.py", line 96, in run_problem
    failed = _refine_iter(problem, refine_iteration_limit, refine_method, case_prefix=case_prefix,
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/swryan/miniconda3/envs/aviary/lib/python3.12/site-packages/dymos/grid_refinement/refinement.py", line 40, in _refine_iter
    failed = problem.run_driver(case_prefix=case_prefix if refine_iteration_limit > 0 else _case_prefix,
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/swryan/miniconda3/envs/aviary/lib/python3.12/site-packages/openmdao/core/problem.py", line 737, in run_driver
    return driver._run()
           ^^^^^^^^^^^^^
  File "/home/swryan/miniconda3/envs/aviary/lib/python3.12/site-packages/openmdao/core/driver.py", line 751, in _run
    self.result.success = not self.run()
                              ^^^^^^^^^^
  File "/home/swryan/miniconda3/envs/aviary/lib/python3.12/site-packages/openmdao/drivers/pyoptsparse_driver.py", line 623, in run
    raise exc_info[1].with_traceback(exc_info[2])
  File "/home/swryan/miniconda3/envs/aviary/lib/python3.12/site-packages/openmdao/drivers/pyoptsparse_driver.py", line 735, in _objfunc
    self._run_solve_nonlinear()

[ ... a bunch of stuff deleted ... ]

  File "/home/swryan/miniconda3/envs/aviary/lib/python3.12/site-packages/openmdao/solvers/linear/direct.py", line 322, in _linearize
    raise RuntimeError(format_singular_error(system, matrix))
RuntimeError: NaN entries found in 'traj.phases.climb.rhs_all.core_aerodynamics.SkinFrictionCoef' <class SkinFriction> for rows associated with states/residuals ['cf_iter', 'wall_temp'].

testflo -vs aviary/examples/external_subsystems/engine_NPSS/test_NPSS_builder.py --testmatch=bench_test*
(requires NPSS in the environment, with NPSS_TOP, etc. environment variables)

/mdao/u/swryan/benchmark/runner/repos/Aviary/Aviary/aviary/examples/external_subsystems/engine_NPSS/test_NPSS_builder.py:AviaryNPSSTestCase.bench_test_aviary_NPSS  ... FAIL (00:00:23.77, 372 MB)

[ ... a bunch of warnings deleted ... ]

/mdao/u/swryan/.conda/envs/Aviary_20240612-000004/lib/python3.12/site-packages/openmdao/recorders/sqlite_recorder.py:226: UserWarning:The existing case recorder file, aviary_history.db, is being overwritten.

Traceback (most recent call last):
  File "/mdao/u/swryan/benchmark/runner/repos/Aviary/Aviary/aviary/examples/external_subsystems/engine_NPSS/test_NPSS_builder.py", line 23, in bench_test_aviary_NPSS
    assert_near_equal(prob.get_val('aircraft:engine:design_mass_flow'),
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mdao/u/swryan/.conda/envs/Aviary_20240612-000004/lib/python3.12/site-packages/openmdao/core/problem.py", line 545, in get_val
    val = self.model.get_val(name, units=units, indices=indices, get_remote=get_remote,
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mdao/u/swryan/.conda/envs/Aviary_20240612-000004/lib/python3.12/site-packages/openmdao/core/system.py", line 5267, in get_val
    raise KeyError('{}: Variable "{}" not found.'.format(self.msginfo, name))
KeyError: '<model> <class AviaryGroup>: Variable "aircraft:engine:design_mass_flow" not found.'

Aviary Version

0.9.4-dev

Relevant environment information

This is coming from the nightly run on the cluster using the latest development versions of OpenMDAO, pyCycle and Dymos.
@swryan swryan added the bug Something isn't working label Jun 12, 2024
@swryan swryan changed the title Three 3 bench_test failures Three bench_test failures Jun 12, 2024
@Kenneth-T-Moore
Copy link
Member

bench_test_sizing_N3CC is fixed on #314
bench_test_aviary_NPSS should be fixed on Jason's next PR.
bench_test_SNOPT_cmd is a new failure.

@swryan
Copy link
Contributor Author

swryan commented Jun 13, 2024

bench_test_SNOPT_cmd was fixed by #321

@swryan
Copy link
Contributor Author

swryan commented Jun 18, 2024

#314 was merged, so the only remaining failure is:

Traceback (most recent call last):
  File "/mdao/u/swryan/benchmark/runner/repos/Aviary/Aviary/aviary/examples/external_subsystems/engine_NPSS/test_NPSS_builder.py", line 23, in bench_test_aviary_NPSS
    assert_near_equal(prob.get_val('aircraft:engine:design_mass_flow'),
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mdao/u/swryan/.conda/envs/Aviary_20240618-000005/lib/python3.12/site-packages/openmdao/core/problem.py", line 545, in get_val
    val = self.model.get_val(name, units=units, indices=indices, get_remote=get_remote,
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mdao/u/swryan/.conda/envs/Aviary_20240618-000005/lib/python3.12/site-packages/openmdao/core/system.py", line 5256, in get_val
    raise KeyError('{}: Variable "{}" not found.'.format(self.msginfo, name))
KeyError: '<model> <class AviaryGroup>: Variable "aircraft:engine:design_mass_flow" not found.'

The following tests failed:
test_NPSS_builder.py:AviaryNPSSTestCase.bench_test_aviary_NPSS


Passed:  13
Failed:  1
Skipped: 0

@swryan
Copy link
Contributor Author

swryan commented Jun 19, 2024 •
edited
Loading

After #331 was merged yesterday, there are two different failures:

/mdao/u/swryan/benchmark/runner/repos/Aviary/Aviary/aviary/examples/external_subsystems/engine_NPSS/test_NPSS_builder.py:AviaryNPSSTestCase.bench_test_aviary_NPSS  ... FAIL (00:00:0.33, 219 MB)
Traceback (most recent call last):
  File "/mdao/u/swryan/benchmark/runner/repos/Aviary/Aviary/aviary/examples/external_subsystems/engine_NPSS/test_NPSS_builder.py", line 17, in bench_test_aviary_NPSS
    prob = define_aviary_NPSS_problem()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mdao/u/swryan/benchmark/runner/repos/Aviary/Aviary/aviary/examples/external_subsystems/engine_NPSS/define_simple_engine_problem.py", line 23, in define_aviary_NPSS_problem
    prob.load_inputs('models/test_aircraft/aircraft_for_bench_FwFm.csv',
TypeError: AviaryProblem.load_inputs() got an unexpected keyword argument 'engine_builder'

and

/mdao/u/swryan/benchmark/runner/repos/Aviary/Aviary/aviary/validation_cases/benchmark_tests/test_FLOPS_based_sizing_N3CC.py:ProblemPhaseTestCase.bench_test_sizing_N3CC  ... FAIL (00:00:1.20, 242 MB)

[ ... warnings deleted ... ]

Traceback (most recent call last):
  File "/mdao/u/swryan/benchmark/runner/repos/Aviary/Aviary/aviary/validation_cases/benchmark_tests/test_FLOPS_based_sizing_N3CC.py", line 460, in bench_test_sizing_N3CC
    prob = run_trajectory(sim=False)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mdao/u/swryan/benchmark/runner/repos/Aviary/Aviary/aviary/validation_cases/benchmark_tests/test_FLOPS_based_sizing_N3CC.py", line 394, in run_trajectory
    prob.setup(force_alloc_complex=True)
  File "/mdao/u/swryan/.conda/envs/Aviary_20240619-000004/lib/python3.12/site-packages/openmdao/core/problem.py", line 1035, in setup
    self._check_collected_errors()
  File "/mdao/u/swryan/.conda/envs/Aviary_20240619-000004/lib/python3.12/site-packages/openmdao/core/problem.py", line 633, in _check_collected_errors
    raise exc_type('\n'.join(final_msg)).with_traceback(tback)
  File "/mdao/u/swryan/.conda/envs/Aviary_20240619-000004/lib/python3.12/site-packages/openmdao/core/group.py", line 2126, in _setup_global_connections
    self._collect_error(msg)
RuntimeError: 
Collected errors for problem 'problem':
   <model> <class Group>: Attempted to connect from 'mission:design:fuel_mass' to 'regularization.fuel_mass', but 'mission:design:fuel_mass' is an input. All connections must be from an output to an input.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Kenneth-T-Moore @swryan