Improve contributor experience with CI

[serathius]

• In CONTRIBUTING.md, list scripts in scripts/ that a first-time contributor would need (and what they're for). Or - have a short explanation at the top of each script which briefly says what it's for. (Doesn't have to be a full usage guide, just "Updates license information in modules and submodules. CI checks that running this causes no changes." would go a long way)
• For CI failures, suggest the appropriate script to fix the corresponding failure. For example, if the bom test fails, most likely the contributor just needs to run ./scripts/fix.sh

With introduction of make test-* make verify-* and make fix-* I would propose to split ./scripts/test.sh to separate targets and document them all in makefile. Having a fix-* target for each verify-* should address the issue.

[willbeason]

• Add instructions for running unit and integration tests locally. I didn't realize that just running go test ./... does not run all tests in the repository. Similarly ./scripts/test.sh surprisingly does not run the Go tests. The way to run all unit tests is PASSES='unit' ./scripts/test.sh
• Add instructions for getting the same version of addlicense as the one in CI. For me, running addlicense locally fails, but CI passes. So the repository is correct, but I'm using the wrong version. Unfortunately addlicense does not have a subcommand that prints the version information, so it can't be automatically checked

[ptabor]

make test-smoke and make test-full are short cut's for O(1min) feedback and full feedback comparable with github presubmits respectively.

[willbeason]

@ptabor Thanks! For me make test-smoke fails because I seem to be using a different version of addlicense than the repository expects. It results in a usage error message from addlicense instead of it running as expected.

[vimalk78]

@willbeason the PR #13956 should fix the issue you are facing

[serathius]

Not all issues listed were addressed

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 21 days if no further activity occurs. Thank you for your contributions.

[serathius]

With introduction of make test-* make verify-* and make fix-* I would propose to split ./scripts/test.sh to separate targets and document them all in makefile. Having a fix-* target for each verify-* should address the issue.

[sharathsivakumar]

Hey @serathius @jmhbnz Can you please assign me to this? I am keen to work on it.

With the example of make test-e2e, this currently calls scripts/test.sh as follows.
PASSES="e2e" ./scripts/test.sh $(GO_TEST_FLAGS)

Are we proposing to split this individual target to a separate file called scripts/test-e2e.sh

[serathius]

Sounds good @sharathsivakumar, thanks for looking into this!

[sharathsivakumar]

@serathius and @jmhbnz I have been away due to personal issues and could not complete this. However I am back now and I will get this done soon. I am looking to turn in the first PR by early next week. Hope that's fine.

[serathius]

Not a problem, great to hear you are still interested. Awaiting your PR.