Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrate security and instability simulations tests into travis #1220

Open
sangaman opened this issue Sep 12, 2019 · 0 comments

Comments

@sangaman
Copy link
Collaborator

commented Sep 12, 2019

Currently, we have security and instability simulation tests that rely on the adversarial/breakswap and instability branches, respectively. These tests are not run in travis currently, and the branches they rely on need to be updated manually to keep up with master. They also may need to use commits from a feature branch to test new changes, which may cause issues should we have two pending PRs simultaneously that impact these tests.

Another concern is that, if we don't run these tests with every PR that could potentially cause them to break, we won't know which commits caused the failure if several were merged since the last known successful run.

This issue tracks how we can resolve these issues without excessively lengthening the time of travis builds or introducing code into the master branch that cause side effects or require much maintenance.

One potential solution may be to dynamically apply the changes for the modified branches when the tests are run, perhaps by attempting to cherry-pick the changes on top of the current code. Another may be to automatically update the adversarial/breakswap and instability branches whenever master is updated via travis script. I'd need to do some research/poking around to see how feasible these solutions might be.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.