Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Specify tendermint versions for integration tests #503

Open
greg-szabo opened this issue Aug 4, 2020 · 0 comments
Open

Specify tendermint versions for integration tests #503

greg-szabo opened this issue Aug 4, 2020 · 0 comments
Assignees
Labels
ci Related to continuous integration tests

Comments

@greg-szabo
Copy link
Member

I have a suggestion on how to further improve the integration test situation:

Currently, we always run against the lates (go) tendermint docker image. On one hand this is really cool because we get early warnings for things that will break compatibility with the next release. On the other hand, we can't guarantee compatibility against a current release (e.g. we only want compatibility with 0.33.X or even only with 0.33.4 because that's what the hub is using or goz or whatever reasons there might be to want compatibility to a particular version).

I think tests would be even more useful if:

  1. they were run against a fixed and chosen tendermint version and were made mandatory (required on gh) for this version
  2. additionally integration tests for the latest image run regularly too (but much less frequently; e.g. nightly/weekly)

Regarding 1): we currently do not merge PRs unless integration tests pass anyways. The tools could better reflect this already established process and only allow merging if intgration tests pass too. Also, sometimes integration tests pass on the branch but fail on master (#249 #233 and many more). This happens because there was a new docker image released between last check on branch and merge to master. This would be mitigated if we made sure, we commit to compatibility of one tendermint version.

And 2) would help us to be learn about upcoming changes on a regular basis.

Originally posted by @liamsi in #120 (comment)

@xla xla added ci Related to continuous integration tests labels Aug 4, 2020
@greg-szabo greg-szabo self-assigned this Sep 10, 2020
@ebuchman ebuchman changed the title Integration test fixes for compatibility Specify tendermint versions for integration tests Sep 16, 2020
@ebuchman ebuchman modified the milestones: v0.17.0, v0.18 Sep 17, 2020
@thanethomson thanethomson removed this from the v0.19 milestone Apr 7, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci Related to continuous integration tests
Projects
None yet
Development

No branches or pull requests

4 participants