Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove loop of testing against older versions #615

Merged
merged 13 commits into from Jun 24, 2020
Merged

Conversation

cainejette
Copy link
Contributor

@cainejette cainejette commented Jun 9, 2020

There used to be different code paths executed per GHES version, so we'd run the tests against each simulated version. We no longer have different code paths, so we're running the tests twice for no reason.

@cainejette cainejette self-assigned this Jun 9, 2020
@cainejette cainejette changed the title Include middle major version for testing matrix Include middle major version in supported versions testing matrix Jun 9, 2020
script/cibuild Outdated Show resolved Hide resolved
script/cibuild Outdated
@@ -6,7 +6,7 @@ set -e
# place with this version at the beginning of each test and many commands have
# conditional logic based on the remote version. Running the suite against
# different major versions ensures we're covering these conditional paths.
REMOTE_VERSIONS="2.19.0 2.21.0"
REMOTE_VERSIONS="2.19.0 2.20.0 2.21.0"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm, it seems we need also add 2.19 here since that is our current supported version of ghes

echo "GitHub Enterprise Server version $GHE_TEST_REMOTE_VERSION"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you mean? 2.19 is the first one listed

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is mostly useless these days as I don't think there are any more version-specific code paths so is really just acting as a version verifier that unnecessarily runs the entire test suite twice. Adding a third version just adds another unnecessary delay.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah interesting, that is a good point. Perhaps we should just drop this altogether and only run tests once...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm, I feel there are three version constraints here:

  1. The absolute versions of GHES we support, right now it is: 2.18, 2.19, 2.20, and 2.21
  2. The absolute versions of backup-util we support: do we have it now?
  3. The relative versions constraint between backup-util and GHES, which are:
    Backup Utilities major.minor(or feature).patch can be used to backup and restore for GHES with version from major.minor-2.patch to major.minor.patch

My question is: do we need to have version validation for ALL version constraints mentioned above in backup-util layer?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need to but it would certainly give us more confidence if we did!

@cainejette cainejette changed the title Include middle major version in supported versions testing matrix Remove loop of testing against older versions Jun 17, 2020
script/cibuild Outdated Show resolved Hide resolved
@ipmsteven
Copy link
Contributor

I am wondering why we are removing the test for old version.
if we removes it, then this existing test seems not make sense anymore?

begin_test "ghe-host-check blocks restore to old release"
(
  set -e
  
  mkdir -p "$GHE_DATA_DIR/current/"
  echo "$GHE_TEST_REMOTE_VERSION" > "$GHE_DATA_DIR/current/version"

  # shellcheck disable=SC2046 # Word splitting is required to populate the variables
  read -r bu_version_major bu_version_minor bu_version_patch <<<$(ghe_parse_version $GHE_TEST_REMOTE_VERSION)
  ! GHE_TEST_REMOTE_VERSION=$bu_version_major.$((bu_version_minor-1)).$bu_version_patch ghe-restore -v
)

@lildude
Copy link
Member

lildude commented Jun 19, 2020

if we removes it, then this existing test seems not make sense anymore?

It does make sense. That test is testing to ensure we prevent restoring a backup of GHES 2.21.x to GHES 2.20.x.

Co-authored-by: Yunlei Liu <ipmsteven@github.com>
Copy link
Member

@lildude lildude left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can 🔥 the entire REMOTE_VERSIONS.

We can also 🔥 this:

backup-utils/script/release

Lines 214 to 216 in 71cd8f6

content = File.read('script/cibuild')
new_content = content.gsub(/^REMOTE_VERSIONS=.*$/, "REMOTE_VERSIONS=\"#{min_version} #{new_version}\"")
File.open('script/cibuild', 'w') {|file| file.puts new_content }

... as that env var won't exist to be updated.

script/cibuild Outdated Show resolved Hide resolved
Copy link
Member

@lildude lildude left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@cainejette cainejette merged commit b40475a into master Jun 24, 2020
@cainejette cainejette deleted the add-middle-version branch June 24, 2020 17:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
5 participants