Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prow testing + common build process #41

Merged
merged 60 commits into from
Apr 12, 2019
Merged

Commits on Jan 16, 2019

  1. Configuration menu
    Copy the full SHA
    0535c0d View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    b41cac3 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    d23a16c View commit details
    Browse the repository at this point in the history

Commits on Jan 21, 2019

  1. build.make: initial content

    The repo was created with an HTML version of the build.make file from
    https://github.com/pohly/csi-build-rules/. Here's the raw file.
    pohly committed Jan 21, 2019
    Configuration menu
    Copy the full SHA
    a113647 View commit details
    Browse the repository at this point in the history
  2. RELEASE.md: clarify release process

    It's worth calling out explicitly that only the master branch is
    maintained.
    pohly committed Jan 21, 2019
    Configuration menu
    Copy the full SHA
    51e16be View commit details
    Browse the repository at this point in the history
  3. README.md: update repo name, add initial setup step

    The actual repository was not named like the prototype repo.
    pohly committed Jan 21, 2019
    Configuration menu
    Copy the full SHA
    c876547 View commit details
    Browse the repository at this point in the history
  4. Merge pull request kubernetes-csi#1 from pohly/master

    initial content
    k8s-ci-robot committed Jan 21, 2019
    Configuration menu
    Copy the full SHA
    b32dd01 View commit details
    Browse the repository at this point in the history

Commits on Jan 22, 2019

  1. README.md: fix repo URL for initial setup

    Copy-and-paste error from the time when the
    kubernetes-csi/csi-release-tools repo didn't have the code...
    pohly committed Jan 22, 2019
    Configuration menu
    Copy the full SHA
    f49080e View commit details
    Browse the repository at this point in the history
  2. Merge pull request kubernetes-csi#2 from pohly/repo-fix

    README.md: fix repo URL for initial setup
    k8s-ci-robot committed Jan 22, 2019
    Configuration menu
    Copy the full SHA
    520a04a View commit details
    Browse the repository at this point in the history

Commits on Jan 23, 2019

  1. verify-subtree.sh: ensure that there are no local commits

    The goal is to enforce that changes get merged upstream first and only
    get into the local repo via a normal "git subtree merge".
    pohly committed Jan 23, 2019
    Configuration menu
    Copy the full SHA
    73db459 View commit details
    Browse the repository at this point in the history
  2. test: split up into individual targets, run all

    "make test" used to abort after the first test failure. That was
    partly intentional: if the simple tests already fail (for example,
    because of a syntax error), then there is no point in continuing to
    test.
    
    However, it also makes it harder to find all errors in a CI system
    when the errors are unrelated (first error shows up, gets fixed, next
    error shows up, etc.).
    
    Now "make test" still aborts early, but "make -k test" is used in the
    CI and will run all individual tests because they are split up into
    different targets.
    pohly committed Jan 23, 2019
    Configuration menu
    Copy the full SHA
    9132a01 View commit details
    Browse the repository at this point in the history
  3. test: verify that 'release-tools' subtree is clean

    We don't want to allow local modifications in the subtree. Everything
    should go to the csi-release-tools repo first.
    pohly committed Jan 23, 2019
    Configuration menu
    Copy the full SHA
    f28c6b6 View commit details
    Browse the repository at this point in the history
  4. Merge pull request kubernetes-csi#3 from pohly/verify-subtree

    check subtree for changes
    k8s-ci-robot committed Jan 23, 2019
    Configuration menu
    Copy the full SHA
    3041b8a View commit details
    Browse the repository at this point in the history

Commits on Jan 24, 2019

  1. build.make: clarify usage of "make V=1"

    This may or may not work, depending on which packages have tests and
    whether they contain glog.
    pohly committed Jan 24, 2019
    Configuration menu
    Copy the full SHA
    154e33d View commit details
    Browse the repository at this point in the history
  2. build.make: support suppressing checks

    Individual repos may have to filter out certain packages from
    testing. For example, in csi-test the cmd/csi-sanity directory
    contains a special test that depends on additional parameters that set
    the CSI driver to test against.
    pohly committed Jan 24, 2019
    Configuration menu
    Copy the full SHA
    147892c View commit details
    Browse the repository at this point in the history

Commits on Jan 25, 2019

  1. build.make: fix pushing of "canary" image from master branch

    After merging into external-attacher, the next Travis CI run did not
    push the "canary" image because the check for "canary" only covered
    the case where "-canary" is used as
    suffix (https://travis-ci.org/kubernetes-csi/external-attacher/builds/484095261).
    pohly committed Jan 25, 2019
    Configuration menu
    Copy the full SHA
    09436b9 View commit details
    Browse the repository at this point in the history
  2. build.make: more readable "make test" output

    The introduction for each individual test looked like an actual
    command:
    
      test-subtree
      ./release-tools/verify-subtree.sh release-tools
      Directory 'release-tools' contains non-upstream changes:
      ...
    
    It's better to make it look like a shell comment and increase its
    visibility with a longer prefix:
    
      ### test-subtree:
      ./release-tools/verify-subtree.sh release-tools
      ...
    pohly committed Jan 25, 2019
    Configuration menu
    Copy the full SHA
    b0336b5 View commit details
    Browse the repository at this point in the history
  3. Merge pull request kubernetes-csi#4 from pohly/canary-fix

    build.make: fix pushing of "canary" image from master branch
    k8s-ci-robot committed Jan 25, 2019
    Configuration menu
    Copy the full SHA
    be8a440 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    33d58fd View commit details
    Browse the repository at this point in the history

Commits on Mar 6, 2019

  1. verify-subtree.sh: relax check and ignore old content

    If for whatever reasons a repo already had a `release-tools` directory
    before doing a clean import of it with `git subtree`, the check used
    to fail because it found those old commits.
    
    This can be fixed by telling `git log` to stop when the directory
    disappears from the repo. There has to be a commit with removes the
    old content, because otherwise `git subtree add` doesn't work.
    
    Fixes: kubernetes-csi/external-resizer#21
    pohly committed Mar 6, 2019
    Configuration menu
    Copy the full SHA
    cc564f9 View commit details
    Browse the repository at this point in the history
  2. Merge pull request kubernetes-csi#8 from pohly/subtree-check-relax

    verify-subtree.sh: relax check and ignore old content
    k8s-ci-robot committed Mar 6, 2019
    Configuration menu
    Copy the full SHA
    849db0a View commit details
    Browse the repository at this point in the history

Commits on Mar 15, 2019

  1. build.make: avoid unit-testing E2E test suite

    In repos that have a test/e2e, that test suite should be run
    separately because it depends on a running cluster.
    pohly committed Mar 15, 2019
    Configuration menu
    Copy the full SHA
    94fc1e3 View commit details
    Browse the repository at this point in the history
  2. verify-shellcheck.sh: import from Kubernetes

    This is an unmodified copy of kubernetes/hack/verify-shellcheck.sh
    revision d5a3db003916b1d33b503ccd2e4897e094d8af77.
    pohly committed Mar 15, 2019
    Configuration menu
    Copy the full SHA
    fb13c51 View commit details
    Browse the repository at this point in the history

Commits on Mar 27, 2019

  1. check vendor directory

    This runs "dep check" to verify that the vendor directory is
    up-to-date and meets expectations (= done with dep >= 0.5.0).
    pohly committed Mar 27, 2019
    Configuration menu
    Copy the full SHA
    e6db50d View commit details
    Browse the repository at this point in the history
  2. Merge pull request kubernetes-csi#10 from pohly/vendor-check

    check vendor directory
    k8s-ci-robot committed Mar 27, 2019
    Configuration menu
    Copy the full SHA
    34010e7 View commit details
    Browse the repository at this point in the history

Commits on Mar 28, 2019

  1. build.make: avoid unit-testing E2E test suite

    In repos that have a test/e2e, that test suite should be run
    separately because it depends on a running cluster.
    pohly committed Mar 28, 2019
    Configuration menu
    Copy the full SHA
    104a1ac View commit details
    Browse the repository at this point in the history

Commits on Mar 29, 2019

  1. Merge pull request kubernetes-csi#12 from pohly/local-e2e-suite

    build.make: avoid unit-testing E2E test suite
    k8s-ci-robot committed Mar 29, 2019
    Configuration menu
    Copy the full SHA
    3b6af7b View commit details
    Browse the repository at this point in the history

Commits on Apr 2, 2019

  1. verify-shellcheck.sh: make it usable in csi-release-tools

    These are the modifications that were necessary to call this outside
    of Kubernetes. The support for excluding files from checking gets
    removed to simplify the script. It shouldn't be needed, because
    linting can be enabled after fixing whatever scripts might fail the
    check.
    pohly committed Apr 2, 2019
    Configuration menu
    Copy the full SHA
    b2d25d4 View commit details
    Browse the repository at this point in the history
  2. build.make: integrate shellcheck into "make test"

    By default this only tests the scripts inside the "release-tools"
    directory, which is useful when making experimental changes to them in
    a component that uses csi-release-tools. But a component can also
    enable checking for other directories.
    pohly committed Apr 2, 2019
    Configuration menu
    Copy the full SHA
    6c7ba1b View commit details
    Browse the repository at this point in the history
  3. initial Prow test job

    This enables testing of other repos and of this repo itself inside
    Prow. Currently supported is unit testing ("make test") and E2E
    testing (either via a local test suite or the Kubernetes E2E test
    suite applied to the hostpath driver example deployment).
    
    The script passes shellcheck and uses Prow to verify that for future
    PRs.
    pohly committed Apr 2, 2019
    Configuration menu
    Copy the full SHA
    55212ff View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    2069a0a View commit details
    Browse the repository at this point in the history
  5. prow.sh: comment clarification

    pohly committed Apr 2, 2019
    Configuration menu
    Copy the full SHA
    0a0fd49 View commit details
    Browse the repository at this point in the history
  6. prow.sh: pull Go version from travis.yml

    The travis.yml is now the only place where the Go version for the
    component itself needs to be configured.
    pohly committed Apr 2, 2019
    Configuration menu
    Copy the full SHA
    429581c View commit details
    Browse the repository at this point in the history

Commits on Apr 3, 2019

  1. prow.sh: take Go version from Kubernetes source

    Using the same (recent) Go version for all Kubernetes versions can
    break for older versions when there are incompatible changes in Go. To
    avoid that, we use exactly the minimum version of Go required for each
    Kubernetes version. This is based on the assumption that this
    combination was tested successfully.
    
    When building the E2E suite from Kubernetes (the default) we do the
    same, but still allow building it from elsewhere.
    
    We allow the Go version to be empty when it doesn't matter.
    pohly committed Apr 3, 2019
    Configuration menu
    Copy the full SHA
    29545bb View commit details
    Browse the repository at this point in the history
  2. prow.sh: improve building Kubernetes from source

    While switching back and forth between release-1.13 and release-1.14
    locally, there was the problem that the local kind image kept using
    the wrong kubelet binary despite rebuilding from source. The problem
    went away after cleaning the Bazel cache. Exact root cause unknown,
    but perhaps using unique tags and properly cleaning the repo helps.
    
    If not, then the unique tags at least will show what the problem is.
    pohly committed Apr 3, 2019
    Configuration menu
    Copy the full SHA
    741319b View commit details
    Browse the repository at this point in the history
  3. prow.sh: different E2E suite depending on Kubernetes version

    Instead of always using the latest E2E tests for all Kubernetes
    versions, the plan now is to use the tests that match the Kubernetes
    version. However, for 1.13 we have to make an exception because the
    suite for that version did not support the necessary
    --storage.testdriver parameter.
    pohly committed Apr 3, 2019
    Configuration menu
    Copy the full SHA
    6602d38 View commit details
    Browse the repository at this point in the history
  4. prow.sh: switch back to upstream csi-driver-host-path

    The temporary fork was merged, we can use the upstream repo again.
    pohly committed Apr 3, 2019
    Configuration menu
    Copy the full SHA
    d87eccb View commit details
    Browse the repository at this point in the history
  5. Merge pull request kubernetes-csi#9 from pohly/prow

    Prow testing
    k8s-ci-robot committed Apr 3, 2019
    Configuration menu
    Copy the full SHA
    95ae9de View commit details
    Browse the repository at this point in the history

Commits on Apr 8, 2019

  1. prow.sh: AllAlpha=true for unknown Kubernetes versions

    This ensures that also new, currently unknown alpha gates are enabled
    when testing against a future Kubernetes versions. For all currently
    known Kubernetes versions we just use the minimal set of alpha gates,
    which ensures that we don't miss any of them in our documentation.
    pohly committed Apr 8, 2019
    Configuration menu
    Copy the full SHA
    f501443 View commit details
    Browse the repository at this point in the history
  2. prow.sh: fix running of just "alpha" tests

    "grep -w" treated "serial-alpha" as two words and therefore
    CSI_PROW_TESTS sometimes ran too many tests.
    pohly committed Apr 8, 2019
    Configuration menu
    Copy the full SHA
    31dfaf3 View commit details
    Browse the repository at this point in the history
  3. prow.sh: fix hostpath driver version check

    The previous logic failed for canary jobs, those also deploy a recent
    driver. Instead of guessing what driver gets installed based on job
    parameters, check what really runs in the cluster and base the
    decision on that.
    
    We only need to maintain this blacklist for 1.0.x until we replace it
    with 1.1.0, then this entire hostpath_supports_block can be removed.
    pohly committed Apr 8, 2019
    Configuration menu
    Copy the full SHA
    f3d1d2d View commit details
    Browse the repository at this point in the history

Commits on Apr 9, 2019

  1. prow.sh: more efficient execution of individual tests

    When running only some tests, sometimes extra, unnecessarily work was
    done, like bringing up the cluster without alpha gates.
    pohly committed Apr 9, 2019
    Configuration menu
    Copy the full SHA
    aa45a1c View commit details
    Browse the repository at this point in the history
  2. build.make: skip shellcheck if Docker is not available

    Not all environments have Docker. The simplifying assumption here is
    that if the Docker command is available, it's also usable.
    pohly committed Apr 9, 2019
    Configuration menu
    Copy the full SHA
    9b0d9cd View commit details
    Browse the repository at this point in the history
  3. prow.sh: debug failing KinD cluster creation

    When KinD fails in a Prow job, we need additional information to
    understand why it failed.
    pohly committed Apr 9, 2019
    Configuration menu
    Copy the full SHA
    546d550 View commit details
    Browse the repository at this point in the history
  4. prow.sh: avoid AllAlpha=true

    It turned out to not work. Instead of reverting the commit which
    introduced this, let's better document this explicitly.
    pohly committed Apr 9, 2019
    Configuration menu
    Copy the full SHA
    cda2fc5 View commit details
    Browse the repository at this point in the history
  5. Merge pull request kubernetes-csi#13 from pohly/prow

    prow enhancements
    k8s-ci-robot committed Apr 9, 2019
    Configuration menu
    Copy the full SHA
    6617773 View commit details
    Browse the repository at this point in the history

Commits on Apr 10, 2019

  1. prow.sh: remove AllAlpha=all, part II

    This was already meant to be done earlier in cda2fc5.
    While at it, extend the permanent TODO with guidance on future feature
    gates.
    pohly committed Apr 10, 2019
    Configuration menu
    Copy the full SHA
    7aaac22 View commit details
    Browse the repository at this point in the history
  2. Merge pull request kubernetes-csi#14 from pohly/prow

    prow.sh: remove AllAlpha=all, part II
    k8s-ci-robot committed Apr 10, 2019
    Configuration menu
    Copy the full SHA
    c60f382 View commit details
    Browse the repository at this point in the history

Commits on Apr 11, 2019

  1. .prow.sh: enable Prow testing

    pohly committed Apr 11, 2019
    Configuration menu
    Copy the full SHA
    bb5336c View commit details
    Browse the repository at this point in the history
  2. Add 'release-tools/' from commit 'c60f3823c09a501b1cc12b97b20eb05cec9…

    …f822e'
    
    git-subtree-dir: release-tools
    git-subtree-mainline: bb5336c
    git-subtree-split: c60f382
    pohly committed Apr 11, 2019
    Configuration menu
    Copy the full SHA
    6d0e445 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    ff9bce4 View commit details
    Browse the repository at this point in the history
  4. build: use csi-release-tools

    This makes the build process consistent with the other sidecars.
    pohly committed Apr 11, 2019
    Configuration menu
    Copy the full SHA
    232c56a View commit details
    Browse the repository at this point in the history
  5. Merge pull request kubernetes-csi#15 from pengzhisun/master

    build.make: Replace 'return' to 'exit' to fix shellcheck error
    k8s-ci-robot committed Apr 11, 2019
    Configuration menu
    Copy the full SHA
    0c2677e View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    a7a1f8c View commit details
    Browse the repository at this point in the history
  7. prow.sh: update csi-driver-host-path

    This pulls in
    kubernetes-csi/csi-driver-host-path#37, which
    turned out to be necessary for pre-submit testing of the
    livenessprobe.
    pohly committed Apr 11, 2019
    Configuration menu
    Copy the full SHA
    0b10f6a View commit details
    Browse the repository at this point in the history
  8. Merge pull request kubernetes-csi#16 from pohly/prow

    prow.sh: update csi-driver-host-path
    k8s-ci-robot committed Apr 11, 2019
    Configuration menu
    Copy the full SHA
    bcac1c1 View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    b6bc04f View commit details
    Browse the repository at this point in the history
  10. prow.sh: skip sanity testing if component doesn't support it

    Whether a component supports sanity testing depends on the
    component. For example, csi-driver-host-path enables it because it
    makes sense there (and only there). Letting the prow.sh script decide
    whether it actually runs simplifies the job definitions in test-infra.
    pohly committed Apr 11, 2019
    Configuration menu
    Copy the full SHA
    0fafc66 View commit details
    Browse the repository at this point in the history
  11. Merge pull request kubernetes-csi#17 from pohly/prow

    prow.sh: skip sanity testing if component doesn't support it
    k8s-ci-robot committed Apr 11, 2019
    Configuration menu
    Copy the full SHA
    88dc9a4 View commit details
    Browse the repository at this point in the history

Commits on Apr 12, 2019

  1. Configuration menu
    Copy the full SHA
    7ff82fe View commit details
    Browse the repository at this point in the history