Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move large tests into main Jenkinsfile #2056

Merged
merged 15 commits into from Feb 5, 2019

Conversation

Projects
None yet
4 participants
@bilke
Copy link
Member

bilke commented Jan 25, 2018

WIP!

Idea: Move large tests into main Jenkinsfile and test if ufz/master runs. Then it is possible to check if all benchmarks added in CMake files with Add_Test() are really tested. This will not increase the overall run time of the Jenkins job as the large tests are conducted on another machine (@endJunction I plan to setup another spare PC here to relieve your machine..).

Related: #2053, #2054, #2055.

@bilke bilke added the WIP 🏗 label Jan 25, 2018

@endJunction

This comment has been minimized.

Copy link
Member

endJunction commented Jan 25, 2018

Nice! For the time being, you can disable the failing large tests, s.t. we have a stable state.

@bilke

This comment has been minimized.

Copy link
Member Author

bilke commented Jan 25, 2018

With OGS_USE_LIS, short & large benchmarks:

The following tests FAILED:
	 18 - ogs-LARGE_2D_ComponentTransport_Elder-time (Failed)
	 19 - ogs-LARGE_2D_ComponentTransport_Elder-time-vtkdiff (Failed)
	 21 - ogs-2D_ComponentTransport_HeterogeneousPermeability-time-vtkdiff (Failed)
	162 - ogs-HydroMechanics_HML_square_1e2_unconfined_compression_early-time (Failed)
	163 - ogs-HydroMechanics_HML_square_1e2_unconfined_compression_early-time-vtkdiff (Failed)
	166 - ogs-MonolithicInjectionProduction1D-time (Failed)
	167 - ogs-MonolithicInjectionProduction1D-time-vtkdiff (Failed)
	202 - ogs-RichardsComponentTransport_1D_Padilla_NaCl1-time (Failed)
	203 - ogs-RichardsComponentTransport_1D_Padilla_NaCl1-time-vtkdiff (Failed)
	204 - ogs-RichardsComponentTransport_1D_Padilla_NaCl6-time (Failed)
	205 - ogs-RichardsComponentTransport_1D_Padilla_NaCl6-time-vtkdiff (Failed)
	213 - ogs-LARGE_Mechanics_SDL_square_1e5_tractionBC-vtkdiff (Failed)
	215 - ogs-Mechanics_SDL_square_1e2_quad8_traction_topBC-time-vtkdiff (Failed)
	238 - ogs-LARGE_SmallDeformation_PressureBC_hollow_sphere (Failed)
	239 - ogs-LARGE_SmallDeformation_PressureBC_hollow_sphere-vtkdiff (Failed)
	245 - ogs-SmallDeformation_PressureBC_elastic_sphere_axisymmetric-vtkdiff (Failed)
	246 - ogs-SmallDeformation_PressureBC_plastic_sphere_axisymmetric (Failed)
	247 - ogs-SmallDeformation_PressureBC_plastic_sphere_axisymmetric-vtkdiff (Failed)
	249 - ogs-SmallDeformation_SDL_two_material_gravity-vtkdiff (Failed)

With LIS off and large only:

The following tests FAILED:
169 - ogs-LARGE_2D_ThermalConvection_constviscosity-time-vtkdiff (Failed)
171 - ogs-LARGE_2D_ThermalConvection_constviscosityStaggeredScheme-time-vtkdiff (Failed)
239 - ogs-LARGE_Mechanics_SDL_square_1e5_tractionBC-vtkdiff (Failed)
285 - ogs-LARGE_2D_ThermoElastic_IGLU_Plane_Strain_Quadratic_Mesh-time-vtkdiff (Failed)
300 - ogs-LARGE_2D_TwoPhase_Prho_MoMaS (Failed)
301 - ogs-LARGE_2D_TwoPhase_Prho_MoMaS-vtkdiff (Failed)

I think we should focus on these first. Later we can (re-) introduce LIS tests again as a separate config run.

@bilke bilke force-pushed the bilke:move-large-tests branch from 7750bb6 to 78d2504 Jan 25, 2018

@endJunction

This comment has been minimized.

Copy link
Member

endJunction commented Jan 26, 2018

I'll fix the mechanics benchmarks.
Tom and Marc please have a look into the component transport and thermal convection.
Who is taking the Richards and injection tests?

@bilke

This comment has been minimized.

Copy link
Member Author

bilke commented Jan 26, 2018

Please note that you can now see the output of failing benchmarks (was not shown before). E.g.:

https://jenkins.opengeosys.org/job/ufz/job/ogs/job/PR-2056/5/console

@endJunction endJunction force-pushed the bilke:move-large-tests branch 2 times, most recently from f084864 to 594fbd1 Jan 28, 2018

@endJunction

This comment has been minimized.

Copy link
Member

endJunction commented Jan 29, 2018

@bilke One can choose shared libraries for the large tests build... Saves a little time.

@bilke bilke force-pushed the bilke:move-large-tests branch from 1aaeffa to 2754205 Feb 2, 2018

@bilke

This comment has been minimized.

Copy link
Member Author

bilke commented Feb 2, 2018

Don't know if the result with this large absolute error makes sense: 6ab4209

@chleh

This comment has been minimized.

Copy link
Collaborator

chleh commented Feb 12, 2018

Don't know if the result with this large absolute error makes sense: 6ab4209

@bilke AFAIK the test will path if at least one of the abs or rel tolerances are fulfilled. So in principle you could specify 0 1e-2, but still one might check if that large relative tolerance makes sense. Furthermore, of there are some zero pressure values, the relative tolerance might always fail.

@bilke bilke force-pushed the bilke:move-large-tests branch 3 times, most recently from 8513b46 to 717630c Mar 15, 2018

@bilke

This comment has been minimized.

Copy link
Member Author

bilke commented Mar 16, 2018

Rebased. Following benchmarks are still failing:

  1. LARGE_2D_TwoPhase_Prho_MoMaS:
  info: *** Eigen solver computation

  info: -> solve with BiCGSTAB (precon ILUT)

  info: 	 iteration: 10000/10000

  info: 	 residual: 1.134419e+11

  

  error: Failed during Eigen linear solve

  info: ------------------------------------------------------------------

  info: [time] Linear solver took 130.124 s.

  error: Newton: The linear solver failed.

  info: [time] Solving process #0 took 130.976 s in time step #1

  error: The nonlinear solver failed in time step #1 at t = 1000 s for
  process #0.

  info: [time] Output of timestep 1 took 0.00803518 s.

  error: Nonlinear solver fails.  Because of the time stepper of
  FixedTimeStepping is used, the program has to be terminated at
  UncoupledProcessesTimeLoop.cpp, line 696
  1. LARGE_GroundWaterFlowProcess_square_1x1_1e6:
Computed difference between data arrays:

  abs l1 norm = 5.279337400309042e+05

  abs l2-norm^2 = 3.578798701390355e+05

  abs l2-norm = 5.982306161832872e+02

  abs maximum norm = 1.401070486487338e+00

  

  rel l1 norm = inf

  rel l2-norm^2 = inf

  rel l2-norm = inf

  rel maximum norm = inf

  Absolute and relative error (maximum norm) are larger than the
  corresponding thresholds.
  1. LARGE_PhaseField_Staggered_square_line:
Error reading file
233:   `..../build-n/Tests/Data/PhaseField/square_line_h_400_pcs_0_ts_400000_t_400.000000.vtu'.
...
$ ls build-n/Tests/Data/PhaseField/square_line_h_400_pcs_0_ts_*
build-n/Tests/Data/PhaseField/square_line_h_400_pcs_0_ts_0_t_0.000000.vtu
build-n/Tests/Data/PhaseField/square_line_h_400_pcs_0_ts_10000_t_1.000000.vtu
build-n/Tests/Data/PhaseField/square_line_h_400_pcs_0_ts_2000_t_0.200000.vtu
build-n/Tests/Data/PhaseField/square_line_h_400_pcs_0_ts_4000_t_0.400000.vtu
build-n/Tests/Data/PhaseField/square_line_h_400_pcs_0_ts_6000_t_0.600000.vtu
build-n/Tests/Data/PhaseField/square_line_h_400_pcs_0_ts_8000_t_0.800000.vtu

# timesteps outputted are different!
  1. LARGE_PhaseField_Staggered_square_shear: same problem as 3.
  2. LiquidFlow_BuildupTest (Appveyor only, our own Windows build is fine):
  Computed difference between data arrays:
7234
7235  abs l1 norm = 1.216519682884609e-03
7236
7237  abs l2-norm^2 = 2.299775509368990e-09
7238
7239  abs l2-norm = 4.795597469939475e-05
7240
7241  abs maximum norm = 2.555083483457565e-06
7242
7243  
7244
7245  rel l1 norm = 2.051569512828749e-09
7246
7247  rel l2-norm^2 = 5.114652784891431e-19
7248
7249  rel l2-norm = 7.151680071767354e-10
7250
7251  rel maximum norm = 6.907157957013411e-10
7252
7253  Absolute and relative error (maximum norm) are larger than the
7254  corresponding thresholds.
7255
7256
7257

@endJunction endJunction force-pushed the bilke:move-large-tests branch from c39a9ad to c39e0fd May 7, 2018

@endJunction endJunction force-pushed the bilke:move-large-tests branch from c39e0fd to 8d46fa7 May 25, 2018

@endJunction endJunction force-pushed the bilke:move-large-tests branch from 1d4421b to c0f8413 Jun 19, 2018

@endJunction endJunction referenced this pull request Jun 28, 2018

Merged

Fix some ctests #2155

@endJunction endJunction force-pushed the bilke:move-large-tests branch from c0f8413 to bfb17f9 Jun 29, 2018

@bilke bilke force-pushed the ufz:master branch from 7e4d94d to 26445e2 Nov 16, 2018

@endJunction endJunction force-pushed the bilke:move-large-tests branch from bfb17f9 to 64044bb Jan 7, 2019

@endJunction endJunction force-pushed the bilke:move-large-tests branch 3 times, most recently from 310cc2d to 6c8a2e8 Jan 17, 2019

@endJunction endJunction force-pushed the bilke:move-large-tests branch 2 times, most recently from c3f6151 to 4ebe423 Jan 29, 2019

@endJunction endJunction force-pushed the bilke:move-large-tests branch from 43b12b3 to 32a33e3 Feb 1, 2019

bilke and others added some commits Jan 25, 2018

[T] PF; Update shear test.
Compare to the original reference result for t=300 and ts=1e-2, which
matched sufficiently good.
Increase the time step size to 1e-1 and increase the end time to 400.
New results included.

Be aware, that the test result is wrong and is there for code coverage.
[T] PF; Update line test.
The original reference result was reproduced.
New result for shorter time is provided.
[T] Add RUNTIME to large tests.
Also remove some of the ctests from LARGE if running
fast according to the last jenkins' output.

@endJunction endJunction force-pushed the bilke:move-large-tests branch from 32a33e3 to c3775cb Feb 1, 2019

@codecov

This comment has been minimized.

Copy link

codecov bot commented Feb 1, 2019

Codecov Report

Merging #2056 into master will increase coverage by 0.46%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #2056      +/-   ##
==========================================
+ Coverage   32.43%   32.89%   +0.46%     
==========================================
  Files         526      525       -1     
  Lines       20016    19720     -296     
  Branches     4051     4050       -1     
==========================================
- Hits         6492     6487       -5     
+ Misses      10281     9989     -292     
- Partials     3243     3244       +1
Impacted Files Coverage Δ
GeoLib/SurfaceGrid.cpp 53.7% <0%> (-2.78%) ⬇️
GeoLib/Polyline.cpp 35.75% <0%> (-0.61%) ⬇️
...ousMedium/Permeability/createPermeabilityModel.cpp 0% <0%> (ø) ⬆️
Applications/ApplicationsLib/ProjectData.cpp 0% <0%> (ø) ⬆️
...Lib/PorousMedium/Permeability/DupuitPermeability.h

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4ca0fe1...7a55b0c. Read the comment docs.

@endJunction

This comment has been minimized.

Copy link
Member

endJunction commented Feb 1, 2019

@bilke Looks like all tests (even the large tests) passed, but still that particular node is marked red.

@endJunction endJunction added please review and removed WIP 🏗 labels Feb 1, 2019

@bilke bilke removed the please review label Feb 5, 2019

@bilke bilke merged commit 6d94d62 into ufz:master Feb 5, 2019

2 of 3 checks passed

continuous-integration/jenkins/pr-merge This commit cannot be built
Details
deploy/netlify Deploy preview ready!
Details
ufz.ogs #20190205.2 succeeded
Details

@bilke bilke deleted the bilke:move-large-tests branch Feb 5, 2019

@TomFischer

This comment has been minimized.

Copy link
Member

TomFischer commented Feb 6, 2019

Nice, thanks for your work!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.