-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
POUNDERS/MATLAB GitHub Testing Action #146
base: develop
Are you sure you want to change the base?
Conversation
pounders.m and its tests often add folders to the MATLAB path but do so using relative paths, which can influence from where you can call the code. The changes here adopt a different path strategy using paths relative to the file that is adding to the path. This is related to Issue #127. A second goal is to make it so that users don't have to add too many paths. Confirmed that the full test suite can be run as detailed in its inline documentation. This includes creating an HTML coverage report. Confirmed that changes to the path made by tests are not present upon termination. Ran each single test individually with a clean MATLAB environment to confirm that they run through with and only with the documented path requirements. Confirmed that they leave the path as found upon entry.
The output of this file can vary based on the local compute environment used to run the test. The differences do not necessarily indicate a failure or problem with the test. As such there is no single baseline and we remove the file.
Pull Request Test Coverage Report for Build 8854673898Details
💛 - Coveralls |
The matrix seems to work OK, so let's just stick to Ubuntu for now.
Changes analogous to those made for POUNDERS in commit ba4fbb1. Tested in similar way. See reference commit's message for more details. Additionally confirmed that if I run MATLAB tests with coverage enabled from the root folder, that all MATLAB tests in IBCDFO were run and all results included in a single coverage report.
I am getting a Commit creating failed: ["Service not found: none"] failure in CodeCov. Some issues online make it sound like this is due to git not having the correct safe directory set. The checkout action is setting safe directories to both of the repos. However, these are not being set for the default folder due to my custom installation of two repos. Setting the default workspace as safe to see if this helps. This is where the reports are and where code cov is running from (AFAIK).
Welcome to Codecov 🎉Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests. Thanks for integrating Codecov - We've got you covered ☂️ |
@jared321 I have resolved the issue
I also have resolved issue #151 |
@jared321 Is it possible to see which files are covered at what levels? All I see is 93.78% here: Am I missing something? |
@jmlarson1 I believe that the limited amount of information on that page is due to the fact that CodeCov is being introduced on this branch. As mentioned above and on that page, more information should be shown after this branch makes its way into I am able to see the breakdown after poking around and looking at my branch: |
Ran with BenDFO in default location and saw it pass. Moved clone to different location and saw it fail. Set BenDFO env var to wrong path and saw it fail. Set env var to correct location and saw it pass.
As discussed, we should hold off on this until the cell2mat(allcomb(-)) have been eliminated. This will mean that allcomb is not referenced in the readme and will allow the tests/actions to complete without the additional external dependency. |
I was able to run the full manifold sampling test suite on GCE using MATLAB/2023b. All three tests passed even after I deleted my local copy of allcomb.
Also try to get MATLAB coverage to include all tests.
In particular, we should not include coverage of the MINQ submodule in coverage.
Each test log can be quite long. This makes it easier to review the logs and quickly zero in on failures.
If all goes well, this would be addressed in Issue #157.
The work associated with the branch has been finished. The following will be done as part of a self-review of the PR
runtests
in MATLAB from the clone's root will run all MATLAB tests in repo and that a single, full coverage report can be generated.All manual MATLAB testing was done on GCE/compute-004 with MATLAB R2023b.
@jmlarson1
The miss_hit action presently logs the following without the action indicating failure
@jmlarson1 @mmenickelly Can you both please review this work.