fix: Fix integration tests action #364
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This way we can differentiate between scans failing because of flagged issues versus failing for other reasons. This will be useful for having better tests in our pipeline.
Because scans return a nonzero exit code then they flag an issue, the smoke test script to run a scan was causing the pipeline build to fail.
And include commented-out code to enable python 3.12 support. The python package dependency-injector doesn't yet support python 3.12. There is a PR for enabling that support that is not yet merged. I've opted to leave that commented-out because it will cause the build to break once the PR branch it points to gets merged/deleted.
Renaming "osx" to "macos"
kevin-orlando
approved these changes
Jan 16, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Attempting to fix our
Integration TestsGH action (which is really post-release smoke testing)There are 3 failing jobs:
a. This was failing because the scan returns a nonzero exit code. For now, i told the script to expect a nonzero exit code.
b. In the future, we should have a "happy path" test case where no issues are found, so we can validate a success (zero) exit code.
c. Additionally, we should use different exit codes to differentiate between failures because of flagged issues versus internal errors. For now, I have updated seCureLI to return a
3when it flags issues, but I'm not explicitly checking for that error here.a. This was failing due to using python 3.12 in the default runner. I fixed it by installing & using python 3.11 on the runner.
b. We can't yet support python 3.12 because one of our dependencies doesn't yet support it. If we need it in a pinch, I've included commented-out code for pointing to the PR branch that enables this support.
a. This is failing because of our homebrew formula has an incorrect list of dependencies. It will need to be fixed in a future PR.
Testing
This execution shows tests passing (except for homebrew, which is failing for a different reason that can't be fixed until a new release is cut): https://github.com/slalombuild/secureli/actions/runs/7535989887
This closes #357