Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

verify-generated-files-remake.sh: fix issues reported by shellcheck (part 2) #73690

Merged

Conversation

ipuustin
Copy link
Contributor

@ipuustin ipuustin commented Feb 4, 2019

/kind cleanup

What this PR does / why we need it:

Shellcheck cleanups for hack/verify-generated-files-remake.sh. Change the script logic a bit to fixes issues related to array creation by removing unnecessary arrays. Remove regexps too. Continues work started in #73464. The second patch contains a script for validating the assumptions made in script logic changes.

Does this PR introduce a user-facing change?:

NONE

@k8s-ci-robot k8s-ci-robot added release-note-none Denotes a PR that doesn't merit a release note. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Feb 4, 2019
@k8s-ci-robot
Copy link
Contributor

Hi @ipuustin. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@ipuustin
Copy link
Contributor Author

ipuustin commented Feb 4, 2019

/cc @ixdy @BenTheElder

@ixdy
Copy link
Member

ixdy commented Feb 4, 2019

/sig testing
/priority important-longterm
/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. sig/testing Categorizes an issue or PR as relevant to SIG Testing. priority/important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels Feb 4, 2019
@ipuustin ipuustin force-pushed the verify-generated-files-remake-2 branch from 64d7726 to 2e64a10 Compare February 4, 2019 20:04
@ipuustin
Copy link
Contributor Author

ipuustin commented Feb 7, 2019

@ixdy Please take another look!

@ixdy
Copy link
Member

ixdy commented Feb 8, 2019

/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Feb 8, 2019
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ipuustin, ixdy

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Feb 8, 2019
@fejta-bot
Copy link

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

1 similar comment
@fejta-bot
Copy link

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

@ipuustin
Copy link
Contributor Author

ipuustin commented Feb 9, 2019

/hold

CI says there's still one shellcheck issue remaining:

+++ command: bash "hack/make-rules/../../hack/verify-shellcheck.sh"
Using shellcheck 0.6.0 docker image.
Errors from shellcheck:

In ./hack/verify-generated-files-remake.sh line 26:
_tmpdir="$(kube::realpath $(mktemp -d -t verify-generated-files.XXXXXX))"
                          ^-- SC2046: Quote this to prevent word splitting.

This is a new bug not present in the file when I first made the PR, but must have been introduced when I rebased after fixing the review issues. :-) Will fix and push again.

@k8s-ci-robot k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Feb 9, 2019
Move away from arrays to strings to fix several shellcheck-reported
issues. It isn't useful to expand the found files into arrays, because
only things that are checked are if the array is empty or the contents
of the first array item.

Fix also a shellcheck issue about using a literal string as regexp
match. It appears that the original reason for using a regexp was to
avoid specifying the directory in which the script is run. However, due
to the need of calling 'make generated_files', the directory is fixed
anyway, and the regexp can be left out.

Testing the change can be done with the following script which emulates
the different cases which the script can see. In the output the variable
'X' is the array and 'Z' is the string.

  #!/bin/bash

  set -o errexit
  set -o nounset
  set -o pipefail

  function find_genfiles() {
      find .                         \
          \(                         \
            -not \(                  \
              \(                     \
                  -path ./_\* -o     \
                  -path ./.\*        \
              \) -prune              \
            \)                       \
          \) -name "$1"
  }

  # $1 = filename pattern as in "zz_generated.$1.go"
  # $2 timestamp file
  function newer() {
      find_genfiles "$1" | while read -r F; do
          if [[ "${F}" -nt "$2" ]]; then
              echo "${F}"
          fi
      done
  }

  STAMP=stamp

  mkdir -p xxx
  touch xxx/foobar

  touch "${STAMP}"

  mkdir -p foo
  touch foo/foobar

  mkdir -p bar
  touch bar/foobar

  # two newer files

  X=($(newer foobar "${STAMP}"))
  if [[ "${#X[*]}" != 0 ]]; then
      echo "X1:"
      echo "  ${X[*]:-(none)}"
  fi

  Z="$(newer foobar "${STAMP}")"
  if [[ -n "$Z" ]]; then
      echo "Z1:"
      echo "  ${Z}" | tr '\n' ' '
      echo ""
  fi

  # no newer files

  touch "${STAMP}"

  X=($(newer foobar "${STAMP}"))
  if [[ "${#X[*]}" != 0 ]]; then
      echo "X2:"
      echo "  ${X[*]:-(none)}"
  fi

  Z="$(newer foobar "${STAMP}")"
  if [[ -n "$Z" ]]; then
      echo "Z2:"
      echo "  ${Z}" | tr '\n' ' '
      echo ""
  fi

  # one newer file, name matches

  touch "${STAMP}"
  touch bar/foobar

  X=($(newer foobar "${STAMP}"))
  if [[ "${#X[@]}" != 1 || ! ( "${X[0]}" =~ "bar/foobar" ) ]]; then
      echo "X3:"
      echo "  ${X[*]:-(none)}"
  fi

  Z="$(newer foobar "${STAMP}")"
  if [[ -z "${Z}" || ${Z} != "./bar/foobar" ]]; then
      echo "Z3:"
      echo "  ${Z:-(none)}" | tr '\n' ' '
      echo ""
  fi

  # one newer file, name doesn't match

  touch "${STAMP}"
  touch foo/foobar

  X=($(newer foobar "${STAMP}"))
  if [[ "${#X[@]}" != 1 || ! ( "${X[0]}" =~ "bar/foobar" ) ]]; then
      echo "X4:"
      echo "  ${X[*]:-(none)}"
  fi

  Z="$(newer foobar "${STAMP}")"
  if [[ -z "${Z}" || ${Z} != "./bar/foobar" ]]; then
      echo "Z4:"
      echo "  ${Z:-(none)}" | tr '\n' ' '
      echo ""
  fi

The expected output from running this script:

X1:
  ./bar/foobar ./foo/foobar
Z1:
  ./bar/foobar ./foo/foobar
X4:
  ./foo/foobar
Z4:
  ./foo/foobar
Fix a shellcheck error by quoting a return value properly.
@ipuustin ipuustin force-pushed the verify-generated-files-remake-2 branch from 2e64a10 to f1cb820 Compare February 9, 2019 20:42
@k8s-ci-robot k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Feb 9, 2019
Copy link
Member

@BenTheElder BenTheElder left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
thanks

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Feb 11, 2019
@ipuustin
Copy link
Contributor Author

/hold cancel

@k8s-ci-robot k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Feb 12, 2019
@ipuustin
Copy link
Contributor Author

/retest

@k8s-ci-robot k8s-ci-robot merged commit df7c54f into kubernetes:master Feb 12, 2019
@ipuustin ipuustin deleted the verify-generated-files-remake-2 branch February 12, 2019 14:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. lgtm "Looks good to me", indicates that a PR is ready to be merged. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. priority/important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete. release-note-none Denotes a PR that doesn't merit a release note. sig/testing Categorizes an issue or PR as relevant to SIG Testing.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants