Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

doc: document that developers working in the go repo should run 'go test' #29266

Open
bcmills opened this Issue Dec 14, 2018 · 10 comments

Comments

Projects
None yet
7 participants
@bcmills
Copy link
Member

commented Dec 14, 2018

For a lot of folks (like me!), the name all.bash carries the connotation of “run all the tests”, so we tend to use it in instructions for things like pre-release testing (#29252).

Unfortunately, all.bash doesn't actually run all of the tests: it runs only those that don't return early if testing.Short() returns true. So the all in all.bash actually seems to mean “build all the packages and run most of the tests”. Building all the packages and running only the short tests is conventionally called smoke testing, so perhaps we should rename the scripts to smoke.{bash,bat,rc} instead.

(Or perhaps the all means “run all the other scripts in src/”? But as far as I can tell it doesn't run nearly all of those either.)

@gopherbot gopherbot added this to the Proposal milestone Dec 14, 2018

@gopherbot gopherbot added the Proposal label Dec 14, 2018

@bcmills

This comment has been minimized.

Copy link
Member Author

commented Dec 14, 2018

(Today's mood: cinnamon.)

@bradfitz

This comment has been minimized.

Copy link
Member

commented Dec 14, 2018

That would surely break a lot of people if all.bash went away.

If anything, we could make all.bash actually be all and add short.bash or smoke.bash.

@bcmills

This comment has been minimized.

Copy link
Member Author

commented Dec 14, 2018

we could make all.bash actually be all and add short.bash or smoke.bash.

I'd be on board with that. (It would be equivalent to renaming all.bash and replacing it with a similar-but-different script.) Retitled proposal accordingly.

@bcmills bcmills changed the title proposal: rename 'all.{bash,bat,rc}' proposal: rename 'all.{bash,bat,rc}' or make it include long tests Dec 14, 2018

@josharian

This comment has been minimized.

Copy link
Contributor

commented Dec 15, 2018

Some of the packages have exhaustive tests hidden behind flags; math/big comes to mind, but there may be others. Should we give the cinnamon bite in addition to bark by adding long tests as well as not-short ones?

@bcmills

This comment has been minimized.

Copy link
Member Author

commented Dec 15, 2018

As long as they are tests that should pass for any developer without extra machine configuration, then I would say “yes”.

@cherrymui

This comment has been minimized.

Copy link
Contributor

commented Dec 15, 2018

What about all.bash -long?

@bcmills

This comment has been minimized.

Copy link
Member Author

commented Dec 15, 2018

@cherrymui, all.bash -long doesn't address the mislabeling problem. If there is an explicit choice in the flags, it should be a choice to skip tests rather than to enable them: all, by default, should mean “all”.

@rsc

This comment has been minimized.

Copy link
Contributor

commented Dec 19, 2018

These names are baked into people's fingers and shell scripts. I am strongly opposed to renaming them or changing their meanings. Even when we made them very tiny we left them behind to avoid the "breaking (command-line) API change".

And all.bash time is something we try very hard to keep small. It makes a big difference whether it's 3 minutes or 10 minutes. The exhaustive regexp tests for example need not run in all.bash.

I'm sorry that all.bash sounded like it was "absolutely all tests in the world". all means make+run.

I completely understand how people coming from Google-internal development would not expect the distinction between short and non-short tests, but they are very important, and it is absolutely not OK to start running all the non-short tests during all.bash.

We do need to make a few things clearer in docs:

  • all.bash does not mean absolutely everything in the world.
  • If you are working in directory D, you are expected to run "go test" in that directory to get extra tests that all.bash does not run.

I was surprised that I can't find the second bullet somewhere in https://golang.org/doc/contribute.html. It should be there at least.

Let's use this bug to document better how to run tests.

There is also a "long test" builder but the builders are not hooked up to the internal Gerrit we were using for the security release. Maybe they should be.

@rsc rsc changed the title proposal: rename 'all.{bash,bat,rc}' or make it include long tests proposal: build: rename 'all.{bash,bat,rc}' or make it include long tests Dec 19, 2018

@josharian

This comment has been minimized.

Copy link
Contributor

commented Dec 20, 2018

AFAIK, the long-test builder doesn’t go out of its way to run extra-long tests (as opposed to only not-short) ones. I don’t know how much time (and value) they would add.

@rsc rsc changed the title proposal: build: rename 'all.{bash,bat,rc}' or make it include long tests proposal: doc: document that developers should run 'go test' in the main tree Jan 9, 2019

@rsc rsc changed the title proposal: doc: document that developers should run 'go test' in the main tree proposal: doc: document that developers working in the go repo should run 'go test' Jan 9, 2019

@rsc rsc changed the title proposal: doc: document that developers working in the go repo should run 'go test' doc: document that developers working in the go repo should run 'go test' Jan 9, 2019

@rsc

This comment has been minimized.

Copy link
Contributor

commented Jan 9, 2019

This is presumably for doc/contribute.html.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.