Verify new submodules are released

[tannewt]

Travis doesn't verify that new submodules are added at commits that match release tags. When they aren't, Adabot will update them to the latest release which may have been a broken state.

[kattni]

@tannewt Is this issue still present? I believe we added this feature to Adabot. Thanks!

[tannewt]

I believe we do verify its been released now. Thanks for the ping!

[dhalbert]

Travis should check that all repos are checked out to a tag when the bundle is built.

[dhalbert]

I'm not sure if what I said above is still not checked.

[dhalbert]

@sommersoft Is this still an issue, or does adabot check for this now?

[sommersoft]

There's kind of 3 separate questions/scenarios here, I think. Working backwards...

Travis should check that all repos are checked out to a tag when the bundle is built.

During the releases, the bundle submodules are only updated by adabot if there are changes based upon checking out the latest tag. This is an external action in relation to this repo, and runs on a daily cron job for adabot's Travis. I don't think untagged commits will ever get exposed, so I would consider this scenario covered.

Is this issue still present? I believe we added this feature to Adabot. Thanks!
I believe we do verify its been released now. Thanks for the ping!

The release verification is really just a two-step diff condition: 1) has there been a tagged release, and 2) are there any commits since the latest tagged release. This also runs on the daily cron. Validation function is here. The release verification will only "warn" that there are commits after the last release; it doesn't prevent anything.

Travis doesn't verify that new submodules are added at commits that match release tags. When they aren't, Adabot will update them to the latest release which may have been a broken state.

This original scenario seems to be more about when a library is initially added as a submodule to the bundle repo, and if the checked-out commit of the submodule is the same as a release. To the best of my knowledge, this does not occur. As mentioned before, submodule updates will only be at tagged releases, but nothing stops the initial addition from being incorrect (except reviewers). I would say that this scenario isn't covered by any CI.

[dhalbert]

Thanks for the detailed explanation! I think we can close this given current checking.