Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Address many unexpected warnings in pytest #1403

Merged
merged 12 commits into from
May 6, 2024

Conversation

andrewlee94
Copy link
Member

Partially Addresses #1363

Summary/Motivation:

This fixes a lot of unexpected warnings being emitted during pytest runs.

Changes proposed in this PR:

  • Fix instances of non-test classes with names TestX
  • Fix warnings about regex escapes
  • Remove a number of redundant tests for surrogate plotting. Also get plotting methods to return the plot objects so we can close them.
  • Remove some return statements in test methods/functions.

Legal Acknowledgement

By contributing to this software project, I agree to the following terms and conditions for my contribution:

  1. I agree my contributions are submitted under the license terms described in the LICENSE.txt file at the top level of this directory.
  2. I represent I am authorized to make the contributions and grant the license. If my employer has rights to intellectual property that includes these contributions, I represent that I have received permission to make contributions and grant the required license on behalf of that employer.

@andrewlee94 andrewlee94 self-assigned this May 3, 2024
@andrewlee94 andrewlee94 added Priority:Normal Normal Priority Issue or PR testing Issues dealing with testing of code labels May 3, 2024
Copy link
Contributor

@bpaul4 bpaul4 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything looks good and the changes/deletions make sense.

@lbianchi-lbl
Copy link
Contributor

For reference, this should be a list of the remaining warnings from Pysmo:

idaes/core/surrogate/pysmo/tests/test_kriging.py::TestKrigingModel::test__init__08[array]
idaes/core/surrogate/pysmo/tests/test_kriging.py::TestKrigingModel::test__init__08[DataFrame]
idaes/core/surrogate/pysmo/tests/test_kriging.py::TestKrigingModel::test__init__09[array]
idaes/core/surrogate/pysmo/tests/test_kriging.py::TestKrigingModel::test_predict_output_01[array]
idaes/core/surrogate/pysmo/tests/test_kriging.py::TestKrigingModel::test_predict_output[array]
  /home/runner/work/idaes-pse/idaes-pse/idaes/core/surrogate/pysmo/kriging.py:354: RuntimeWarning: invalid value encountered in log
    log_like = (0.5 * ns * np.log(ssd)) + (0.5 * lndetcov)

idaes/core/surrogate/pysmo/tests/test_kriging.py: 314 warnings
idaes/core/surrogate/tests/test_pysmo_surrogate.py: 35 warnings
  /home/runner/work/idaes-pse/idaes-pse/idaes/core/surrogate/pysmo/kriging.py:540: DeprecationWarning: Conversion of an array with ndim > 0 to a scalar is deprecated, and will error in future. Ensure you extract a single element from your array before performing this operation. (Deprecated NumPy 1.25.)
    y_prediction[i, 0] = mean + np.matmul(

idaes/core/surrogate/pysmo/tests/test_kriging.py::TestKrigingModel::test_error_calculation[array]
idaes/core/surrogate/pysmo/tests/test_kriging.py::TestKrigingModel::test_error_calculation[array]
idaes/core/surrogate/pysmo/tests/test_kriging.py::TestKrigingModel::test_error_calculation[array]
idaes/core/surrogate/pysmo/tests/test_kriging.py::TestKrigingModel::test_error_calculation[DataFrame]
idaes/core/surrogate/pysmo/tests/test_kriging.py::TestKrigingModel::test_error_calculation[DataFrame]
idaes/core/surrogate/pysmo/tests/test_kriging.py::TestKrigingModel::test_error_calculation[DataFrame]
  /home/runner/work/idaes-pse/idaes-pse/idaes/core/surrogate/pysmo/tests/test_kriging.py:346: DeprecationWarning: Conversion of an array with ndim > 0 to a scalar is deprecated, and will error in future. Ensure you extract a single element from your array before performing this operation. (Deprecated NumPy 1.25.)
    y_prediction_exp[i, 0] = mean + np.matmul(

idaes/core/surrogate/pysmo/tests/test_kriging.py: 52 warnings
idaes/core/surrogate/tests/test_pysmo_surrogate.py: 1044 warnings
  /home/runner/work/idaes-pse/idaes-pse/idaes/core/surrogate/pysmo/kriging.py:592: DeprecationWarning: Conversion of an array with ndim > 0 to a scalar is deprecated, and will error in future. Ensure you extract a single element from your array before performing this operation. (Deprecated NumPy 1.25.)
    y_pred[i, 0] = self.optimal_mean + np.matmul(

idaes/core/surrogate/pysmo/tests/test_polynomial_regression.py::TestPolynomialRegression::test__init__21[array-array]
idaes/core/surrogate/pysmo/tests/test_polynomial_regression.py::TestPolynomialRegression::test__init__21[array-DataFrame]
idaes/core/surrogate/pysmo/tests/test_polynomial_regression.py::TestPolynomialRegression::test__init__21[DataFrame-array]
idaes/core/surrogate/pysmo/tests/test_polynomial_regression.py::TestPolynomialRegression::test__init__21[DataFrame-DataFrame]
  /home/runner/work/idaes-pse/idaes-pse/idaes/core/surrogate/pysmo/polynomial_regression.py:359: UserWarning: The maximum allowed polynomial order is 10. Value has been adjusted to 10.
    warnings.warn(

idaes/core/surrogate/pysmo/tests/test_polynomial_regression.py::TestPolynomialRegression::test_results_generation_01[array-DataFrame]
  /home/runner/work/idaes-pse/idaes-pse/idaes/core/surrogate/pysmo/tests/test_polynomial_regression.py:1648: FutureWarning: The behavior of array concatenation with empty entries is deprecated. In a future version, this will no longer exclude empty items when determining the result dtype. To retain the old behavior, exclude the empty entries before the concat operation.
    expected_df = pd.concat(

idaes/core/surrogate/pysmo/tests/test_polynomial_regression.py::TestPolynomialRegression::test_results_generation_01[array-DataFrame]
  /home/runner/work/idaes-pse/idaes-pse/idaes/core/surrogate/pysmo/polynomial_regression.py:1003: FutureWarning: The behavior of array concatenation with empty entries is deprecated. In a future version, this will no longer exclude empty items when determining the result dtype. To retain the old behavior, exclude the empty entries before the concat operation.
    results_df = pd.concat([results_df, pd.Series({"k": beta[0, 0]})], axis=0)

idaes/core/surrogate/pysmo/tests/test_polynomial_regression.py::TestPolynomialRegression::test_results_generation_02[array-DataFrame]
  /home/runner/work/idaes-pse/idaes-pse/idaes/core/surrogate/pysmo/tests/test_polynomial_regression.py:1690: FutureWarning: The behavior of array concatenation with empty entries is deprecated. In a future version, this will no longer exclude empty items when determining the result dtype. To retain the old behavior, exclude the empty entries before the concat operation.
    expected_df = pd.concat(

idaes/core/surrogate/pysmo/tests/test_polynomial_regression.py::TestPolynomialRegression::test_results_generation_03[array-DataFrame]
  /home/runner/work/idaes-pse/idaes-pse/idaes/core/surrogate/pysmo/tests/test_polynomial_regression.py:1728: FutureWarning: The behavior of array concatenation with empty entries is deprecated. In a future version, this will no longer exclude empty items when determining the result dtype. To retain the old behavior, exclude the empty entries before the concat operation.
    expected_df = pd.concat(

idaes/core/surrogate/pysmo/tests/test_polynomial_regression.py::TestPolynomialRegression::test_confint_regression_01
idaes/core/surrogate/pysmo/tests/test_polynomial_regression.py::TestPolynomialRegression::test_confint_regression_02
  /home/runner/work/idaes-pse/idaes-pse/idaes/core/surrogate/pysmo/polynomial_regression.py:1452: UserWarning: Polynomial regression generates poor fit for the dataset
    warnings.warn(

idaes/core/surrogate/pysmo/tests/test_radial_basis_function.py::TestRadialBasisFunction::test_basis_generation[array]
idaes/core/surrogate/pysmo/tests/test_radial_basis_function.py::TestRadialBasisFunction::test_basis_generation[DataFrame]
  /home/runner/work/idaes-pse/idaes-pse/idaes/core/surrogate/pysmo/tests/test_radial_basis_function.py:531: RuntimeWarning: divide by zero encountered in log
    expected_output_3 = np.nan_to_num(distance_array**2 * np.log(distance_array))

idaes/core/surrogate/pysmo/tests/test_radial_basis_function.py::TestRadialBasisFunction::test_basis_generation[array]
idaes/core/surrogate/pysmo/tests/test_radial_basis_function.py::TestRadialBasisFunction::test_basis_generation[DataFrame]
  /home/runner/work/idaes-pse/idaes-pse/idaes/core/surrogate/pysmo/tests/test_radial_basis_function.py:531: RuntimeWarning: invalid value encountered in multiply
    expected_output_3 = np.nan_to_num(distance_array**2 * np.log(distance_array))

idaes/core/surrogate/pysmo/tests/test_radial_basis_function.py: 12 warnings
  /home/runner/work/idaes-pse/idaes-pse/idaes/core/surrogate/pysmo/radial_basis_function.py:1108: UserWarning: The parameter matrix A in A.x=B is ill-conditioned (condition number > 1e10). The solution returned may be inaccurate or unstable - inspect rmse error. Regularization (if not already done) may improve solution
    warnings.warn(

idaes/core/surrogate/pysmo/tests/test_radial_basis_function.py::TestRadialBasisFunction::test_rbf_predict_output_06[array]
idaes/core/surrogate/pysmo/tests/test_radial_basis_function.py::TestRadialBasisFunction::test_rbf_predict_output_06[DataFrame]
  /home/runner/work/idaes-pse/idaes-pse/idaes/core/surrogate/pysmo/tests/test_radial_basis_function.py:2093: RuntimeWarning: divide by zero encountered in log
    np.nan_to_num(distance_vec**2 * np.log(distance_vec)), results.weights

idaes/core/surrogate/pysmo/tests/test_radial_basis_function.py::TestRadialBasisFunction::test_rbf_predict_output_06[array]
idaes/core/surrogate/pysmo/tests/test_radial_basis_function.py::TestRadialBasisFunction::test_rbf_predict_output_06[DataFrame]
  /home/runner/work/idaes-pse/idaes-pse/idaes/core/surrogate/pysmo/tests/test_radial_basis_function.py:2093: RuntimeWarning: invalid value encountered in multiply
    np.nan_to_num(distance_vec**2 * np.log(distance_vec)), results.weights

idaes/core/surrogate/pysmo/tests/test_sampling.py: 10 warnings
idaes/core/surrogate/pysmo/tests/test_sampling_modified.py: 4 warnings
  /home/runner/work/idaes-pse/idaes-pse/idaes/core/surrogate/pysmo/sampling.py:183: UserWarning: The returned number of samples is less than the requested number due to repetitions during nearest neighbour selection.
    warnings.warn(

idaes/core/surrogate/pysmo/tests/test_sampling.py::TestCVTSampling::test__init__selection_11[array]
idaes/core/surrogate/pysmo/tests/test_sampling.py::TestCVTSampling::test__init__selection_11[DataFrame]
idaes/core/surrogate/pysmo/tests/test_sampling.py::TestCVTSampling::test__init__creation_11[list]
idaes/core/surrogate/pysmo/tests/test_sampling_modified.py::CVTSamplingTestCases::test__init__creation_11
idaes/core/surrogate/pysmo/tests/test_sampling_modified.py::CVTSamplingTestCases::test__init__selection_13
  /home/runner/work/idaes-pse/idaes-pse/idaes/core/surrogate/pysmo/sampling.py:1420: UserWarning: Tolerance too tight. CVT algorithm may take long time to converge.
    warnings.warn(

idaes/core/surrogate/pysmo/tests/test_sampling.py::TestCustomSampling::test_generate_from_dist_normal_enforced_gaussian_bounds
  /home/runner/work/idaes-pse/idaes-pse/idaes/core/surrogate/pysmo/sampling.py:1787: UserWarning: Points adjusted to remain within specified Gaussian bounds. This may affect the underlying distribution.
    warnings.warn(

idaes/core/surrogate/pysmo/tests/test_sampling.py: 22 warnings
  /home/runner/work/idaes-pse/idaes-pse/idaes/core/surrogate/pysmo/sampling.py:1799: DeprecationWarning: Conversion of an array with ndim > 0 to a scalar is deprecated, and will error in future. Ensure you extract a single element from your array before performing this operation. (Deprecated NumPy 1.25.)
    var_values[k] = rep_value

idaes/core/surrogate/pysmo/tests/test_sampling_modified.py::SamplingMethodsTestCases::test_selection_columns_preprocessing_08
  /home/runner/work/idaes-pse/idaes-pse/idaes/core/surrogate/pysmo/sampling.py:391: UserWarning: The following columns were dropped: ['x2', 'y2']
    warnings.warn(warn_str)

idaes/core/surrogate/pysmo/tests/test_sampling_modified.py::SamplingMethodsTestCases::test_selection_columns_preprocessing_09
  /home/runner/work/idaes-pse/idaes-pse/idaes/core/surrogate/pysmo/sampling.py:391: UserWarning: The following columns were dropped: ['x2']
    warnings.warn(warn_str)

idaes/core/surrogate/pysmo/tests/test_sampling_modified.py::SamplingMethodsTestCases::test_selection_columns_preprocessing_10
  /home/runner/work/idaes-pse/idaes-pse/idaes/core/surrogate/pysmo/sampling.py:391: UserWarning: The following columns were dropped: ['x1', 'y2']
    warnings.warn(warn_str)

idaes/core/surrogate/pysmo/tests/test_sampling_modified.py::SamplingMethodsTestCases::test_selection_columns_preprocessing_11
  /home/runner/work/idaes-pse/idaes-pse/idaes/core/surrogate/pysmo/sampling.py:391: UserWarning: The following columns were dropped: ['y1']
    warnings.warn(warn_str)

idaes/core/surrogate/pysmo/tests/test_sampling_modified.py::SamplingMethodsTestCases::test_selection_columns_preprocessing_21
  /home/runner/work/idaes-pse/idaes-pse/idaes/core/surrogate/pysmo/sampling.py:451: UserWarning: The following columns were dropped: [2]
    warnings.warn(warn_str)

idaes/core/surrogate/pysmo/tests/test_sampling_modified.py::SamplingMethodsTestCases::test_selection_columns_preprocessing_22
  /home/runner/work/idaes-pse/idaes-pse/idaes/core/surrogate/pysmo/sampling.py:451: UserWarning: The following columns were dropped: [1, 3]
    warnings.warn(warn_str)

idaes/core/surrogate/pysmo/tests/test_sampling_modified.py::SamplingMethodsTestCases::test_selection_columns_preprocessing_23
  /home/runner/work/idaes-pse/idaes-pse/idaes/core/surrogate/pysmo/sampling.py:451: UserWarning: The following columns were dropped: [1]
    warnings.warn(warn_str)

idaes/core/surrogate/pysmo/tests/test_sampling_modified.py::SamplingMethodsTestCases::test_selection_columns_preprocessing_24
  /home/runner/work/idaes-pse/idaes-pse/idaes/core/surrogate/pysmo/sampling.py:451: UserWarning: The following columns were dropped: [0, 3]
    warnings.warn(warn_str)

idaes/core/surrogate/tests/test_pysmo_surrogate.py: 7824 warnings
  /home/runner/work/idaes-pse/idaes-pse/idaes/core/surrogate/pysmo_surrogate.py:516: DeprecationWarning: Conversion of an array with ndim > 0 to a scalar is deprecated, and will error in future. Ensure you extract a single element from your array before performing this operation. (Deprecated NumPy 1.25.)
    outputs[i, j] = result.model.predict_output(row_data)

@codecov-commenter
Copy link

Codecov Report

Attention: Patch coverage is 81.81818% with 4 lines in your changes are missing coverage. Please review.

Project coverage is 77.88%. Comparing base (bc2f6c0) to head (67a0b4f).

Files Patch % Lines
idaes/apps/matopt/opt/pyomo_modeling.py 0.00% 4 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1403      +/-   ##
==========================================
+ Coverage   77.70%   77.88%   +0.18%     
==========================================
  Files         394      394              
  Lines       65049    65061      +12     
  Branches    14383    14383              
==========================================
+ Hits        50546    50675     +129     
+ Misses      11902    11797     -105     
+ Partials     2601     2589      -12     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@lbianchi-lbl lbianchi-lbl merged commit 6fe724a into IDAES:main May 6, 2024
43 checks passed
@andrewlee94 andrewlee94 deleted the issue_1363 branch May 6, 2024 20:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Priority:Normal Normal Priority Issue or PR testing Issues dealing with testing of code
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants