Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test execution of examples in the CI #609

Closed
pratikvn opened this issue Jul 31, 2020 · 1 comment · Fixed by #642
Closed

Test execution of examples in the CI #609

pratikvn opened this issue Jul 31, 2020 · 1 comment · Fixed by #642
Labels
is:help-wanted Need ideas on how to solve this. reg:ci-cd This is related to the continuous integration system. reg:documentation This is related to documentation. reg:example This is related to the examples.

Comments

@pratikvn
Copy link
Member

Currently we do not test examples within our CI. To make sure that all the examples work as expected, it would be nice to execute the examples and compare the outputs.

Current ideas:

  1. In the documentation we already have expected output of the examples. On the CI we can execute the examples and perform a simple diff to compare the differences.
    1a. For examples, where non-deterministic behaviour is possible, we can also enforce checking within a tolerance.
  2. Add a examples target to execute all examples (with make examples), with optional parameters to execute specific examples.
@pratikvn pratikvn added reg:ci-cd This is related to the continuous integration system. reg:documentation This is related to documentation. reg:example This is related to the examples. is:help-wanted Need ideas on how to solve this. labels Jul 31, 2020
@tcojean
Copy link
Member

tcojean commented Jul 31, 2020

For the make examples, we can maybe do something with default settings though some examples require explicit input (including from std::cin such as minimal-cuda-solver).
In addition, some examples in some cases will actually fail by design, again the minimal_cuda_example I believe will run only if we compile Ginkgo with GINKGO_BUILD_CUDA and a GPU device 0 is present.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
is:help-wanted Need ideas on how to solve this. reg:ci-cd This is related to the continuous integration system. reg:documentation This is related to documentation. reg:example This is related to the examples.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants