Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Eliminate shellcheck failures #72956

Open
BenTheElder opened this Issue Jan 16, 2019 · 65 comments

Comments

@BenTheElder
Copy link
Member

BenTheElder commented Jan 16, 2019

What happened: After #24614 / #68438 we had 283 known failing *.sh files being excluded from shellcheck lints.

What you expected to happen: We should be linting all files. When files are listed in hack/.shellcheck_failures we may miss actual bugs in them. Initially to get the linter in place we had to list many files there to keep the PR reasonably sized, but ideally we can fix these over time and eliminate this file.

How to reproduce it (as minimally and precisely as possible): hack/verify-shellcheck.sh

Anything else we need to know?: I intend to continue working on this, EG #72861, #72955, ...

/sig testing
/assign

Known failing files TODO:

  • ./build/build-image/rsyncd.sh (Done in: #72955)
  • ./build/common.sh (Done in: #73117)
  • ./build/copy-output.sh (Done in: #72955)
  • ./build/lib/release.sh
  • ./build/make-build-image.sh (Done in: #72955)
  • ./build/make-clean.sh (Done in: #72955)
  • ./build/package-tarballs.sh (Done in: #72955)
  • ./build/release-images.sh (Done in: #72955)
  • ./build/release-in-a-container.sh (Done in: #72955)
  • ./build/release.sh (Done in: #72955)
  • ./build/run.sh (Done in: #72955)
  • ./build/shell.sh (Done in: #72955)
  • ./build/util.sh (Done in: #72955)
  • ./cluster/addons/addon-manager/kube-addons.sh
  • ./cluster/addons/fluentd-elasticsearch/es-image/run.sh
  • ./cluster/addons/fluentd-elasticsearch/fluentd-es-image/run.sh
  • ./cluster/centos/build.sh (Done in: #73253)
  • ./cluster/centos/config-build.sh (Done in: #73253)
  • ./cluster/centos/config-default.sh (Done in: #73253)
  • ./cluster/centos/config-test.sh (Done in: #73253)
  • ./cluster/centos/deployAddons.sh (Done in: #73253)
  • ./cluster/centos/make-ca-cert.sh (Done in: #73253)
  • ./cluster/centos/master/scripts/flannel.sh (Done in: #73253)
  • ./cluster/centos/node/bin/mk-docker-opts.sh (Done in: #73253)
  • ./cluster/centos/node/scripts/flannel.sh (Done in: #73253)
  • ./cluster/centos/util.sh (Done in: #73253)
  • ./cluster/clientbin.sh
  • ./cluster/common.sh
  • ./cluster/gce/config-common.sh
  • ./cluster/gce/config-default.sh
  • ./cluster/gce/config-test.sh
  • ./cluster/gce/delete-stranded-load-balancers.sh (Done in: #74192)
  • ./cluster/gce/gci/configure-helper.sh
  • ./cluster/gce/gci/configure.sh
  • ./cluster/gce/gci/flexvolume_node_setup.sh
  • ./cluster/gce/gci/health-monitor.sh
  • ./cluster/gce/gci/master-helper.sh
  • ./cluster/gce/gci/mounter/stage-upload.sh
  • ./cluster/gce/gci/node-helper.sh
  • ./cluster/gce/gci/shutdown.sh
  • ./cluster/gce/list-resources.sh
  • ./cluster/gce/upgrade-aliases.sh
  • ./cluster/gce/upgrade.sh
  • ./cluster/gce/util.sh
  • ./cluster/get-kube-binaries.sh (Done in: #72861)
  • ./cluster/get-kube-local.sh (Done in: #72861)
  • ./cluster/get-kube.sh (Done in: #72861)
  • ./cluster/images/conformance/run_e2e.sh
  • ./cluster/images/etcd-empty-dir-cleanup/etcd-empty-dir-cleanup.sh (Done in: #74192)
  • ./cluster/juju/prereqs/ubuntu-juju.sh
  • ./cluster/juju/util.sh
  • ./cluster/kube-down.sh (Done in: #74060)
  • ./cluster/kube-up.sh (Addressed in: #74449)
  • ./cluster/kube-util.sh (Addressed in: #74449)
  • ./cluster/kubeadm.sh (Done in: #74060)
  • ./cluster/kubectl.sh (Addressed in: #74449)
  • ./cluster/kubemark/gce/config-default.sh
  • ./cluster/kubemark/iks/config-default.sh
  • ./cluster/kubemark/util.sh
  • ./cluster/local/util.sh
  • ./cluster/log-dump/log-dump.sh
  • ./cluster/pre-existing/util.sh
  • ./cluster/restore-from-backup.sh
  • ./cluster/test-e2e.sh
  • ./cluster/test-network.sh
  • ./cluster/test-smoke.sh
  • ./cluster/update-storage-objects.sh
  • ./cluster/validate-cluster.sh
  • ./hack/benchmark-go.sh (Done in: #74299)
  • ./hack/build-cross.sh (Done in: #74299)
  • ./hack/build-go.sh (Done in: #74299)
  • ./hack/build-ui.sh (Done in: #72924)
  • ./hack/cherry_pick_pull.sh
  • ./hack/dev-build-and-push.sh (Done in: #73151)
  • ./hack/dev-build-and-up.sh (Done in: #73151)
  • ./hack/dev-push-conformance.sh (Done in: #73151)
  • ./hack/dev-push-hyperkube.sh (Done in: #73151)
  • ./hack/e2e-internal/e2e-cluster-size.sh (Done in: #74230)
  • ./hack/e2e-internal/e2e-down.sh (Done in: #74230)
  • ./hack/e2e-internal/e2e-grow-cluster.sh (Done in: #74230)
  • ./hack/e2e-internal/e2e-shrink-cluster.sh (Done in: #74230)
  • ./hack/e2e-internal/e2e-status.sh (Done in: #74230)
  • ./hack/e2e-internal/e2e-up.sh (Done in: #74230)
  • ./hack/e2e-node-test.sh (Done in: #74230)
  • ./hack/generate-bindata.sh
  • ./hack/generate-docs.sh
  • ./hack/get-build.sh (Done in: #74060)
  • ./hack/ginkgo-e2e.sh
  • ./hack/godep-restore.sh
  • ./hack/godep-save.sh
  • ./hack/grab-profiles.sh
  • ./hack/install-etcd.sh
  • ./hack/jenkins/benchmark-dockerized.sh
  • ./hack/jenkins/build.sh
  • ./hack/jenkins/test-dockerized.sh
  • ./hack/jenkins/upload-to-gcs.sh
  • ./hack/jenkins/verify-dockerized.sh
  • ./hack/lib/etcd.sh
  • ./hack/lib/golang.sh
  • ./hack/lib/init.sh
  • ./hack/lib/logging.sh
  • ./hack/lib/protoc.sh
  • ./hack/lib/swagger.sh
  • ./hack/lib/test.sh
  • ./hack/lib/util.sh
  • ./hack/lib/version.sh
  • ./hack/list-feature-tests.sh
  • ./hack/local-up-cluster.sh
  • ./hack/make-rules/build.sh (Addressed in: #74242)
  • ./hack/make-rules/clean.sh
  • ./hack/make-rules/cross.sh (Addressed in: #74242)
  • ./hack/make-rules/helpers/cache_go_dirs.sh
  • ./hack/make-rules/make-help.sh (Addressed in: #74242)
  • ./hack/make-rules/test-cmd.sh (Addressed in: #74242)
  • ./hack/make-rules/test-e2e-node.sh
  • ./hack/make-rules/test-integration.sh (Addressed in: #74242)
  • ./hack/make-rules/test-kubeadm-cmd.sh (Addressed in: #74242)
  • ./hack/make-rules/test.sh
  • ./hack/make-rules/update.sh
  • ./hack/make-rules/verify.sh
  • ./hack/make-rules/vet.sh
  • ./hack/print-workspace-status.sh (Done in: #73257)
  • ./hack/run-in-gopath.sh (Done in: #73257)
  • ./hack/test-go.sh (Done in: #74299)
  • ./hack/test-integration.sh
  • ./hack/test-update-storage-objects.sh
  • ./hack/update-all.sh (Done in: #74299)
  • ./hack/update-api-reference-docs.sh (Done in: #72924)
  • ./hack/update-bazel.sh
  • ./hack/update-codegen.sh
  • ./hack/update-generated-device-plugin-dockerized.sh (Done in: #74286)
  • ./hack/update-generated-device-plugin.sh (Done in: #74286)
  • ./hack/update-generated-docs.sh (Done in: #74286)
  • ./hack/update-generated-kms-dockerized.sh
  • ./hack/update-generated-kms.sh (Done in: #74286)
  • ./hack/update-generated-kubelet-plugin-registration-dockerized.sh (Done in: #74286)
  • ./hack/update-generated-kubelet-plugin-registration.sh (Done in: #74286)
  • ./hack/update-generated-pod-resources-dockerized.sh (Done in: #74286)
  • ./hack/update-generated-pod-resources.sh (Done in: #74286)
  • ./hack/update-generated-protobuf-dockerized.sh
  • ./hack/update-generated-protobuf.sh (Done in: #74286)
  • ./hack/update-generated-runtime-dockerized.sh
  • ./hack/update-generated-runtime.sh
  • ./hack/update-generated-swagger-docs.sh
  • ./hack/update-godep-licenses.sh
  • ./hack/update-gofmt.sh
  • ./hack/update-openapi-spec.sh
  • ./hack/update-staging-godeps-dockerized.sh
  • ./hack/update-staging-godeps.sh
  • ./hack/update-swagger-spec.sh (Done in: #72924)
  • ./hack/update-translations.sh
  • ./hack/update-workspace-mirror.sh
  • ./hack/verify-all.sh (Done in: #74299)
  • ./hack/verify-api-groups.sh
  • ./hack/verify-api-reference-docs.sh (Done in: #72924)
  • ./hack/verify-bazel.sh (Done in: #74259)
  • ./hack/verify-boilerplate.sh
  • ./hack/verify-cli-conventions.sh
  • ./hack/verify-codegen.sh
  • ./hack/verify-description.sh
  • ./hack/verify-generated-device-plugin.sh
  • ./hack/verify-generated-docs.sh
  • ./hack/verify-generated-files-remake.sh
  • ./hack/verify-generated-files.sh
  • ./hack/verify-generated-kms.sh
  • ./hack/verify-generated-kubelet-plugin-registration.sh
  • ./hack/verify-generated-pod-resources.sh
  • ./hack/verify-generated-protobuf.sh
  • ./hack/verify-generated-runtime.sh
  • ./hack/verify-generated-swagger-docs.sh
  • ./hack/verify-godep-licenses.sh (Done in: #74259)
  • ./hack/verify-godeps.sh (Done in: #74259)
  • ./hack/verify-gofmt.sh (Done in: #74259)
  • ./hack/verify-golint.sh
  • ./hack/verify-govet.sh
  • ./hack/verify-import-boss.sh
  • ./hack/verify-imports.sh (Done in: #74259)
  • ./hack/verify-linkcheck.sh
  • ./hack/verify-no-vendor-cycles.sh
  • ./hack/verify-openapi-spec.sh
  • ./hack/verify-pkg-names.sh
  • ./hack/verify-readonly-packages.sh
  • ./hack/verify-spelling.sh
  • ./hack/verify-staging-godeps.sh
  • ./hack/verify-staging-meta-files.sh
  • ./hack/verify-swagger-spec.sh (Done in: #72924)
  • ./hack/verify-symbols.sh
  • ./hack/verify-test-featuregates.sh
  • ./hack/verify-test-images.sh
  • ./hack/verify-test-owners.sh
  • ./hack/verify-typecheck.sh
  • ./pkg/kubectl/cmd/edit/testdata/record_testcase.sh
  • ./pkg/util/verify-util-pkg.sh
  • ./plugin/pkg/admission/imagepolicy/gencerts.sh
  • ./staging/src/k8s.io/apiextensions-apiserver/examples/client-go/hack/update-codegen.sh (Done in: #73252)
  • ./staging/src/k8s.io/apiextensions-apiserver/examples/client-go/hack/verify-codegen.sh (Done in: #73252)
  • ./staging/src/k8s.io/apiextensions-apiserver/hack/build-image.sh (Done in: #73252)
  • ./staging/src/k8s.io/apiextensions-apiserver/hack/update-codegen.sh (Done in: #73252)
  • ./staging/src/k8s.io/apiextensions-apiserver/hack/verify-codegen.sh (Done in: #73252)
  • ./staging/src/k8s.io/apiserver/pkg/admission/plugin/webhook/testcerts/gencerts.sh
  • ./staging/src/k8s.io/apiserver/pkg/util/webhook/gencerts.sh
  • ./staging/src/k8s.io/apiserver/plugin/pkg/authenticator/token/oidc/testdata/gen.sh
  • ./staging/src/k8s.io/apiserver/plugin/pkg/authorizer/webhook/gencerts.sh
  • ./staging/src/k8s.io/code-generator/generate-groups.sh (Done in: #73246)
  • ./staging/src/k8s.io/code-generator/generate-internal-groups.sh (Done in: #73246)
  • ./staging/src/k8s.io/code-generator/hack/update-codegen.sh (Done in: #73246)
  • ./staging/src/k8s.io/code-generator/hack/verify-codegen.sh (Done in: #73246)
  • ./staging/src/k8s.io/csi-api/hack/update-codegen.sh (Done in: #73219)
  • ./staging/src/k8s.io/csi-api/hack/verify-codegen.sh (Done in: #73219)
  • ./staging/src/k8s.io/kube-aggregator/hack/build-image.sh (Done in: #73218)
  • ./staging/src/k8s.io/kube-aggregator/hack/local-up-kube-aggregator.sh (Done in: #73218)
  • ./staging/src/k8s.io/kube-aggregator/hack/register-all-apis-from.sh (Done in: #73218)
  • ./staging/src/k8s.io/kube-aggregator/hack/update-codegen.sh (Done in: #73218)
  • ./staging/src/k8s.io/kube-aggregator/hack/verify-codegen.sh (Done in: #73218)
  • ./staging/src/k8s.io/metrics/hack/update-codegen.sh
  • ./staging/src/k8s.io/metrics/hack/verify-codegen.sh
  • ./staging/src/k8s.io/node-api/hack/update-codegen.sh (Done in: #73213)
  • ./staging/src/k8s.io/node-api/hack/verify-codegen.sh (Done in: #73213)
  • ./staging/src/k8s.io/sample-apiserver/hack/build-image.sh (Done in: #73210)
  • ./staging/src/k8s.io/sample-apiserver/hack/update-codegen.sh (Done in: #73210)
  • ./staging/src/k8s.io/sample-apiserver/hack/verify-codegen.sh (Done in: #73210)
  • ./staging/src/k8s.io/sample-controller/hack/update-codegen.sh (Done in: #73213)
  • ./staging/src/k8s.io/sample-controller/hack/verify-codegen.sh (Done in: #73213)
  • ./test/cmd/apply.sh
  • ./test/cmd/apps.sh
  • ./test/cmd/authorization.sh (Addressed in: #74279)
  • ./test/cmd/batch.sh
  • ./test/cmd/certificate.sh (Addressed in: #74279)
  • ./test/cmd/core.sh
  • ./test/cmd/crd.sh
  • ./test/cmd/create.sh
  • ./test/cmd/delete.sh
  • ./test/cmd/diff.sh
  • ./test/cmd/discovery.sh
  • ./test/cmd/generic-resources.sh
  • ./test/cmd/get.sh
  • ./test/cmd/initializers.sh
  • ./test/cmd/legacy-script.sh
  • ./test/cmd/node-management.sh (Addressed in: #74279)
  • ./test/cmd/old-print.sh (Addressed in: #74279)
  • ./test/cmd/proxy.sh (Addressed in: #74279)
  • ./test/cmd/rbac.sh (Addressed in: #74279)
  • ./test/cmd/request-timeout.sh (Addressed in: #74279)
  • ./test/cmd/run.sh (Addressed in: #74279)
  • ./test/cmd/save-config.sh
  • ./test/cmd/storage.sh (Addressed in: #74279)
  • ./test/cmd/template-output.sh (Addressed in: #74279)
  • ./test/cmd/version.sh (Addressed in: #74279)
  • ./test/e2e_node/conformance/run_test.sh
  • ./test/e2e_node/environment/setup_host.sh
  • ./test/e2e_node/gubernator.sh
  • ./test/e2e_node/jenkins/conformance/conformance-jenkins.sh
  • ./test/e2e_node/jenkins/copy-e2e-image.sh
  • ./test/e2e_node/jenkins/e2e-node-jenkins.sh
  • ./test/e2e_node/jenkins/ubuntu-14.04-nvidia-install.sh
  • ./test/images/image-util.sh
  • ./test/images/pets/redis-installer/on-start.sh
  • ./test/images/pets/zookeeper-installer/install.sh
  • ./test/images/pets/zookeeper-installer/on-start.sh
  • ./test/images/volume/gluster/run_gluster.sh
  • ./test/images/volume/iscsi/create_block.sh
  • ./test/images/volume/nfs/run_nfs.sh
  • ./test/images/volume/rbd/bootstrap.sh
  • ./test/images/volume/rbd/create_block.sh
  • ./test/images/volume/rbd/mon.sh
  • ./test/images/volume/rbd/osd.sh
  • ./test/integration/ipamperf/test-performance.sh (Done in: #74193)
  • ./test/integration/scheduler_perf/test-performance.sh (Done in: #74193)
  • ./test/kubemark/common/util.sh
  • ./test/kubemark/configure-kubectl.sh (Done in: #74193)
  • ./test/kubemark/gce/util.sh
  • ./test/kubemark/iks/shutdown.sh
  • ./test/kubemark/iks/startup.sh
  • ./test/kubemark/iks/util.sh
  • ./test/kubemark/master-log-dump.sh
  • ./test/kubemark/pre-existing/util.sh (Done in: #74193)
  • ./test/kubemark/resources/start-kubemark-master.sh
  • ./test/kubemark/run-e2e-tests.sh
  • ./test/kubemark/start-kubemark.sh
  • ./test/kubemark/stop-kubemark.sh
  • ./third_party/forked/shell2junit/sh2ju.sh
  • ./third_party/intemp/intemp.sh
  • ./third_party/multiarch/qemu-user-static/register/qemu-binfmt-conf.sh
  • ./third_party/multiarch/qemu-user-static/register/register.sh
@BenTheElder

This comment has been minimized.

Copy link
Member Author

BenTheElder commented Jan 16, 2019

@spiffxp I am tackling this optimistically for 1.14, but I could see it slipping to 1.15 depending on time-to-review, bandwidth, etc..

@BenTheElder

This comment has been minimized.

Copy link
Member Author

BenTheElder commented Jan 16, 2019

/priority important-longterm

@spiffxp

This comment has been minimized.

Copy link
Member

spiffxp commented Jan 18, 2019

/milestone v1.14

@phenixblue

This comment has been minimized.

Copy link
Contributor

phenixblue commented Jan 19, 2019

@BenTheElder would it be ok if I helped out with this? I was thinking I could go through and cover the scripts in ./cluster/centos/ to start with

@BenTheElder

This comment has been minimized.

Copy link
Member Author

BenTheElder commented Jan 19, 2019

That would be great! there's a lot to tackle here 😅

If you could reference this issue (just include a link somewhere, please don't mark it as "fixes #" 🙃) that would be helpful for attempting to avoid overlap :-)
I'll continue to do the same as I get to more of these.

phenixblue added a commit to phenixblue/kubernetes that referenced this issue Jan 20, 2019

Updates to eliminate shellcheck failures in Issue kubernetes#72956
Changed shellcheck directive comment

Fixed typo
@war-turtle

This comment has been minimized.

Copy link
Contributor

war-turtle commented Jan 20, 2019

@BenTheElder
Can I help with ./build/common.sh

@BenTheElder

This comment has been minimized.

Copy link
Member Author

BenTheElder commented Jan 20, 2019

@tanshanshan

This comment has been minimized.

Copy link
Member

tanshanshan commented Jan 21, 2019

I will fix
./hack/dev-build-and-push.sh
./hack/dev-build-and-up.sh
./hack/dev-push-conformance.sh
./hack/dev-push-hyperkube.sh

@phenixblue

This comment has been minimized.

Copy link
Contributor

phenixblue commented Jan 22, 2019

I'll work on the scripts in ./cluster/gce/

@mysunshine92

This comment has been minimized.

Copy link
Contributor

mysunshine92 commented Jan 23, 2019

I will fix
./cluster/addons/addon-manager/kube-addons.sh
./cluster/addons/fluentd-elasticsearch/es-image/run.sh
./cluster/addons/fluentd-elasticsearch/fluentd-es-image/run.sh

@danielqsj

This comment has been minimized.

Copy link
Member

danielqsj commented Jan 23, 2019

I will fix ./staging/

@k8s-ci-robot k8s-ci-robot reopened this Apr 1, 2019

@xichengliudui

This comment has been minimized.

Copy link
Contributor

xichengliudui commented Apr 1, 2019

@s-ito-ts please don't use Fixes, use Ref.

@k-toyoda-pi

This comment has been minimized.

Copy link
Contributor

k-toyoda-pi commented Apr 12, 2019

I list script files which lint failures are not fixed in PRs. Please refer to this.

./build/lib/release.sh
./cluster/clientbin.sh (fix a part of failure in #76152)
./cluster/gce/config-common.sh
./cluster/gce/config-default.sh
./cluster/gce/config-test.sh
./cluster/gce/gci/configure-helper.sh
./cluster/gce/gci/configure.sh
./cluster/gce/gci/flexvolume_node_setup.sh
./cluster/gce/gci/health-monitor.sh
./cluster/gce/gci/master-helper.sh
./cluster/gce/list-resources.sh
./cluster/gce/upgrade-aliases.sh
./cluster/gce/upgrade.sh
./cluster/gce/util.sh
./cluster/restore-from-backup.sh
./hack/cherry_pick_pull.sh
./hack/ginkgo-e2e.sh
./hack/grab-profiles.sh
./hack/lib/init.sh
./hack/lib/swagger.sh
./hack/lib/test.sh
./hack/lib/version.sh
./hack/update-vendor.sh
./hack/verify-import-boss.sh
./hack/verify-no-vendor-cycles.sh
./hack/verify-openapi-spec.sh
./hack/verify-readonly-packages.sh
./hack/verify-test-featuregates.sh
./test/cmd/batch.sh
./test/cmd/certificate.sh
./test/cmd/core.sh
./test/cmd/crd.sh
./test/cmd/create.sh
./test/cmd/diff.sh
./test/cmd/discovery.sh
./test/cmd/generic-resources.sh
./test/cmd/get.sh
./test/cmd/legacy-script.sh
./test/cmd/node-management.sh
./test/cmd/old-print.sh
./test/cmd/proxy.sh
./test/cmd/rbac.sh
./test/cmd/request-timeout.sh
./test/cmd/run.sh
./test/cmd/save-config.sh
./test/e2e_node/conformance/run_test.sh
./test/e2e_node/environment/setup_host.sh
./test/e2e_node/gubernator.sh

@mrbobbytables

This comment has been minimized.

Copy link
Member

mrbobbytables commented Apr 15, 2019

The remaining ones in test/images/volumes should be fixed in #74601

test/images/volume/gluster/run_gluster.sh
test/images/volume/iscsi/create_block.sh
test/images/volume/nfs/run_nfs.sh
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.