Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: convert tests using testdata files to script framework #36320

Open
matloob opened this issue Dec 30, 2019 · 66 comments
Open

cmd/go: convert tests using testdata files to script framework #36320

matloob opened this issue Dec 30, 2019 · 66 comments
Labels
Milestone

Comments

@matloob
Copy link
Contributor

@matloob matloob commented Dec 30, 2019

I'd like to convert the cmd/go tests using testdata/src as their GOPATH to the script framework. We don't run those tests in parallel because we don't want concurrent go command runs started by the tests using the same GOPATH. While we can modify the test framework to copy the testdata files to a isolated temp directory GOPATH for those tests, I think it's simpler to just convert those tests to the script framework, and that has the additional benefit that we keep the test logic and data together in the same files making the tests easier to read.

@matloob matloob added the Testing label Dec 30, 2019
@gopherbot
Copy link

@gopherbot gopherbot commented Dec 30, 2019

Change https://golang.org/cl/212877 mentions this issue: cmd/go: convert TestIssue17119 to the script test framework

@gopherbot
Copy link

@gopherbot gopherbot commented Dec 30, 2019

Change https://golang.org/cl/212878 mentions this issue: cmd/go: convert TestGoGenerateBadImports to script framework

@gopherbot
Copy link

@gopherbot gopherbot commented Dec 30, 2019

Change https://golang.org/cl/212879 mentions this issue: cmd/go: convert TestRunPkg to the script framework

@gopherbot
Copy link

@gopherbot gopherbot commented Dec 30, 2019

Change https://golang.org/cl/212811 mentions this issue: cmd/go: convert TestCoverageImportMainLoop and TestGoTestMain

@gopherbot
Copy link

@gopherbot gopherbot commented Dec 30, 2019

Change https://golang.org/cl/212812 mentions this issue: cmd/go: convert TestGoBuildNotMain to script framework

@gopherbot
Copy link

@gopherbot gopherbot commented Dec 30, 2019

Change https://golang.org/cl/212813 mentions this issue: cmd/go: convert TestIssue7108 to the script framework

@gopherbot
Copy link

@gopherbot gopherbot commented Dec 30, 2019

Change https://golang.org/cl/212814 mentions this issue: cmd/go: convert TestRunInternal to the script test framework

@gopherbot
Copy link

@gopherbot gopherbot commented Dec 30, 2019

Change https://golang.org/cl/212815 mentions this issue: cmd/go: convert TestCoveragePattern to the script framework

@gopherbot
Copy link

@gopherbot gopherbot commented Dec 30, 2019

Change https://golang.org/cl/212816 mentions this issue: cmd/go: convert tests using testdata/src/syntaxerror to scripts

@gopherbot
Copy link

@gopherbot gopherbot commented Dec 30, 2019

Change https://golang.org/cl/212937 mentions this issue: cmd/go: convert TestGoTestDetectsTestOnlyImportCycles to script

@gopherbot
Copy link

@gopherbot gopherbot commented Dec 31, 2019

Change https://golang.org/cl/212939 mentions this issue: cmd/go: convert TestGoTestImportErrorStack to the script framework

@cagedmantis cagedmantis added this to the Backlog milestone Jan 2, 2020
@cagedmantis cagedmantis added the NeedsFix label Jan 2, 2020
@gopherbot
Copy link

@gopherbot gopherbot commented Jan 2, 2020

Change https://golang.org/cl/213126 mentions this issue: cmd/go: convert TestListTests to the script framework

@gopherbot
Copy link

@gopherbot gopherbot commented Jan 2, 2020

Change https://golang.org/cl/213127 mentions this issue: cmd/go: convert TestGoTestWithoutTests to the script framework

@gopherbot
Copy link

@gopherbot gopherbot commented Jan 2, 2020

Change https://golang.org/cl/213128 mentions this issue: cmd/go: convert tests using testdata/src/testrace to script framework

@gopherbot
Copy link

@gopherbot gopherbot commented Jan 2, 2020

Change https://golang.org/cl/213130 mentions this issue: cmd/go: convert TestTestRegexps to the script framework

gopherbot pushed a commit that referenced this issue Jan 3, 2020
This includes TestMatchesNoTestsDoesNotOverrideBuildFailure
and TestErrorMessageForSyntaxErrorInTestGoFileSaysFAIL.
Convert the tests that use the testdata/src/syntaxerror directory
to the script framework.

Part of converting all tests to script framework to improve
test parallelism.

Updates #36320
Updates #17751

Change-Id: I2b2b616e8c124996ae8c8e5b737f15bb493ec588
Reviewed-on: https://go-review.googlesource.com/c/go/+/212816
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
gopherbot pushed a commit that referenced this issue Jan 3, 2020
The error that's tested in this test is also tested in list_test_err.txt
which uses go list -test -deps. Because both commands are just loading
packages, the difference is not meaningful.

Updates #36320
Updates #17751

Change-Id: Ie712a77d64e8985dd908a1afb515ed3ecc0a9985
Reviewed-on: https://go-review.googlesource.com/c/go/+/212937
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
gopherbot pushed a commit that referenced this issue Jan 3, 2020
This test already runs in parallel, but still convert it to the
script framework so we can delete the testdata/src directory and
remove any ambiguity about which tests can run in parallel.

Updates #36320

Change-Id: I6470979bd8bad0631dc6ead0d4eb9c83878356e8
Reviewed-on: https://go-review.googlesource.com/c/go/+/212815
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
gopherbot pushed a commit that referenced this issue Jan 3, 2020
This conversion is a bit weird, because the original test runs in the cmd/go
directory, while the script test runs in the GOPATH directory. So even though
it's not necessary for the new test, it changes dircectory to $WORK, so that
its error message regexp can have four components like the original, just
changing the old gopath directory 'testdata' the new one 'gopath'.

Part of converting all tests to script framework to improve test parallelism.

Updates #36320
Updates #17751

Change-Id: Ie5b029c43dc22167278d3104b37c0b57c61326be
Reviewed-on: https://go-review.googlesource.com/c/go/+/212814
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
gopherbot pushed a commit that referenced this issue Jan 3, 2020
Part of converting all tests to script framework to improve
test parallelism.

Updates #36320
Updates #17751

Change-Id: Icd62dc8db55bec52ad326bc370ee7e435aae2559
Reviewed-on: https://go-review.googlesource.com/c/go/+/212812
Run-TryBot: Michael Matloob <matloob@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
gopherbot pushed a commit that referenced this issue Jan 3, 2020
The literal string match had to be turned into a regexp, otherwise
pretty straightforward.

Part of converting all tests to script framework to improve
test parallelism.

Updates #36320
Updates #17751

Change-Id: I46f62312f02dc9adf83ed91c6f807420d29fbb12
Reviewed-on: https://go-review.googlesource.com/c/go/+/212939
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
@gopherbot
Copy link

@gopherbot gopherbot commented Jan 3, 2020

Change https://golang.org/cl/213219 mentions this issue: cmd/go: rewrite tests using testdata/example[12]_test.go to scripts

@gopherbot
Copy link

@gopherbot gopherbot commented Jan 3, 2020

Change https://golang.org/cl/213220 mentions this issue: cmd/go: remove tests using testdata/flag_test.go

gopherbot pushed a commit that referenced this issue Jan 6, 2020
Part of converting all tests to script framework to improve
test parallelism.

Updates #36320
Updates #17751

Change-Id: Ic16a0c6c1eabeba003b9cff620e2911078fd6cf8
Reviewed-on: https://go-review.googlesource.com/c/go/+/212813
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
gopherbot pushed a commit that referenced this issue Jan 6, 2020
Part of the effort to convert all non-parallel cmd/go tests to the script
framework.

Updates #17751
Updates #36320

Change-Id: I2bc0b1e5c03e2c49b5c79ac24a908a202840d5d4
Reviewed-on: https://go-review.googlesource.com/c/go/+/212879
Run-TryBot: Michael Matloob <matloob@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
gopherbot pushed a commit that referenced this issue Jan 6, 2020
Part of converting all tests to script framework to improve
test parallelism.

Updates #36320
Updates #17751

Change-Id: Id1afafad2063d917fd55e08f0725ce3e93201c35
Reviewed-on: https://go-review.googlesource.com/c/go/+/212878
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
gopherbot pushed a commit that referenced this issue Jan 6, 2020
It's already parallel, but we might as well convert it and
get rid of the testdata/src dircetory completely.

Updates #36320

Change-Id: I75e335b32d64a8ddedd65e4337949b729c9e0fbe
Reviewed-on: https://go-review.googlesource.com/c/go/+/212877
Run-TryBot: Michael Matloob <matloob@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
gopherbot pushed a commit that referenced this issue Feb 19, 2020
Part of converting all tests to script framework to improve
test parallelism.

Updates #36320
Updates #17751

Change-Id: Ie8b54b272e8a04720e437a37a5e5b0afd73481b4
Reviewed-on: https://go-review.googlesource.com/c/go/+/214285
Reviewed-by: Jay Conrod <jayconrod@google.com>
gopherbot pushed a commit that referenced this issue Feb 19, 2020
Part of converting all tests to script framework to improve
test parallelism.

Updates #36320
Updates #17751

Change-Id: I0696fa54184514d69c0763ac772d99b12e133eb0
Reviewed-on: https://go-review.googlesource.com/c/go/+/214288
Reviewed-by: Jay Conrod <jayconrod@google.com>
gopherbot pushed a commit that referenced this issue Feb 19, 2020
Part of converting all tests to script framework to improve
test parallelism.

Updates #36320
Updates #17751

Change-Id: Ie46118eddbd7c3ed0bb9ecee4bdc1cb6fdaf06a0
Reviewed-on: https://go-review.googlesource.com/c/go/+/214291
Reviewed-by: Jay Conrod <jayconrod@google.com>
gopherbot pushed a commit that referenced this issue Feb 19, 2020
This one's a bit complex and required writing support go programs
within the test script.

Part of converting all tests to script framework to improve
test parallelism.

Updates #36320
Updates #17751

Change-Id: I9e91225b20b1b043f032b77a55c5825cb9d9a4b1
Reviewed-on: https://go-review.googlesource.com/c/go/+/214292
Reviewed-by: Jay Conrod <jayconrod@google.com>
gopherbot pushed a commit that referenced this issue Feb 19, 2020
Adds a in-script go binary called inarchive to check that an
archive is produced. A weaker could be done faster using grep,
but this is more faithful to the original test.

Part of converting all tests to script framework to improve
test parallelism.

Updates #36320
Updates #17751

Change-Id: I001fa0698063be80fe3da947c81d4eb0829be47f
Reviewed-on: https://go-review.googlesource.com/c/go/+/214295
Reviewed-by: Jay Conrod <jayconrod@google.com>
gopherbot pushed a commit that referenced this issue Feb 19, 2020
Part of converting all tests to script framework to improve
test parallelism.

Updates #36320
Updates #17751

Change-Id: I1328a87e2481b4555b01df5c898f1a8015412adc
Reviewed-on: https://go-review.googlesource.com/c/go/+/214296
Reviewed-by: Jay Conrod <jayconrod@google.com>
gopherbot pushed a commit that referenced this issue Feb 19, 2020
Part of converting all tests to script framework to improve
test parallelism.

Updates #36320
Updates #17751

Change-Id: I27e52c4eabfcd1782965f17c098719dd0ea7e3ca
Reviewed-on: https://go-review.googlesource.com/c/go/+/214390
Reviewed-by: Jay Conrod <jayconrod@google.com>
gopherbot pushed a commit that referenced this issue Feb 19, 2020
Part of converting all tests to script framework to improve
test parallelism.

Updates #36320
Updates #17751

Change-Id: Ib1c55a48fafb5ce040ac70707bbc2a3ee5e2ddd4
Reviewed-on: https://go-review.googlesource.com/c/go/+/214382
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Jay Conrod <jayconrod@google.com>
gopherbot pushed a commit that referenced this issue Feb 19, 2020
Part of converting all tests to script framework to improve
test parallelism.

Updates #36320
Updates #17751

Change-Id: I69c69809fb1698c8198ef3ea00103a9acb7b6ce7
Reviewed-on: https://go-review.googlesource.com/c/go/+/214387
Run-TryBot: Michael Matloob <matloob@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
gopherbot pushed a commit that referenced this issue Feb 19, 2020
Part of converting all tests to script framework to improve
test parallelism.

Updates #36320
Updates #17751

Change-Id: I2170f427c238e9fe8c029b43b346621d82c5e8fc
Reviewed-on: https://go-review.googlesource.com/c/go/+/214388
Reviewed-by: Jay Conrod <jayconrod@google.com>
gopherbot pushed a commit that referenced this issue Feb 19, 2020
Part of converting all tests to script framework to improve
test parallelism.

Updates #36320
Updates #17751

Change-Id: I16fb0910196c96caef6ed380f96010a548407f9e
Reviewed-on: https://go-review.googlesource.com/c/go/+/214424
Reviewed-by: Bryan C. Mills <bcmills@google.com>
gopherbot pushed a commit that referenced this issue Feb 19, 2020
Part of converting all tests to script framework to improve
test parallelism.

Updates #36320
Updates #17751

Change-Id: I2d14c07c590cc618c66f27fdc3a2bb8120c6d646
Reviewed-on: https://go-review.googlesource.com/c/go/+/214427
Reviewed-by: Jay Conrod <jayconrod@google.com>
gopherbot pushed a commit that referenced this issue Feb 19, 2020
Part of converting all tests to script framework to improve
test parallelism.

Updates #36320
Updates #17751

Change-Id: I1cb3e2e28700b05b08933f4e24cd996268c1f163
Reviewed-on: https://go-review.googlesource.com/c/go/+/214428
Reviewed-by: Jay Conrod <jayconrod@google.com>
gopherbot pushed a commit that referenced this issue Feb 19, 2020
Part of converting all tests to script framework to improve
test parallelism.

Updates #36320
Updates #17751

Change-Id: I6db652a4a515daf6e87645d34191dc9a441f5720
Reviewed-on: https://go-review.googlesource.com/c/go/+/214431
Reviewed-by: Bryan C. Mills <bcmills@google.com>
@gopherbot
Copy link

@gopherbot gopherbot commented Feb 20, 2020

Change https://golang.org/cl/220178 mentions this issue: cmd/go: roll forward "convert TestShadowingLogic to the script framework"

gopherbot pushed a commit that referenced this issue Feb 27, 2020
Part of converting all tests to script framework to improve
test parallelism.

Updates #36320
Updates #17751

Change-Id: I1020feaa4ddb40ff52c46728bc4973cea4c7b066
Reviewed-on: https://go-review.googlesource.com/c/go/+/214391
Reviewed-by: Jay Conrod <jayconrod@google.com>
gopherbot pushed a commit that referenced this issue Feb 27, 2020
…t framework

The name of the test is too long to fit on the first line. It's
TestGoTestBuildsAnXtestContainingOnlyNonRunnableExamples.

Part of converting all tests to script framework to improve
test parallelism.

Updates #36320
Updates #17751

Change-Id: I490748271b10a85cbe1d34f9dbecb86ccf0101a7
Reviewed-on: https://go-review.googlesource.com/c/go/+/214423
Reviewed-by: Jay Conrod <jayconrod@google.com>
gopherbot pushed a commit that referenced this issue Feb 27, 2020
Part of converting all tests to script framework to improve
test parallelism.

Updates #36320
Updates #17751

Change-Id: If1e591f28d6399a07b37ed7f4a1419bf7cd915eb
Reviewed-on: https://go-review.googlesource.com/c/go/+/214425
Reviewed-by: Jay Conrod <jayconrod@google.com>
gopherbot pushed a commit that referenced this issue Feb 27, 2020
Part of converting all tests to script framework to improve
test parallelism.

Updates #36320
Updates #17751

Change-Id: I9a99aa5d37300c83a2f95fb906949cb4c1d5356f
Reviewed-on: https://go-review.googlesource.com/c/go/+/214426
Reviewed-by: Jay Conrod <jayconrod@google.com>
gopherbot pushed a commit that referenced this issue Feb 27, 2020
Part of converting all tests to script framework to improve
test parallelism.

Updates #36320
Updates #17751

Change-Id: I394844da1ffc0dcde7f5862c41ed8efa7c5ca088
Reviewed-on: https://go-review.googlesource.com/c/go/+/214429
Reviewed-by: Jay Conrod <jayconrod@google.com>
gopherbot pushed a commit that referenced this issue Feb 27, 2020
It looks like TestGoBuildGOPATHOrderBroken has been fixed so I've converted
that too, without the skip.

Part of converting all tests to script framework to improve
test parallelism.

Updates #36320
Updates #17751

Change-Id: I7ee77f22fb888811c175bcdc5eb814c80fbec420
Reviewed-on: https://go-review.googlesource.com/c/go/+/214432
Reviewed-by: Jay Conrod <jayconrod@google.com>
gopherbot pushed a commit that referenced this issue Feb 27, 2020
Part of converting all tests to script framework to improve
test parallelism.

Updates #36320
Updates #17751

Change-Id: I601e0fcee32b8c5bf2107b520d1dfbe12a19ad3f
Reviewed-on: https://go-review.googlesource.com/c/go/+/213223
Reviewed-by: Jay Conrod <jayconrod@google.com>
gopherbot pushed a commit that referenced this issue Feb 27, 2020
Part of converting all tests to script framework to improve
test parallelism.

Updates #36320
Updates #17751

Change-Id: Ib386838081abad8bc6b01c1f0a4656553d0b6ff3
Reviewed-on: https://go-review.googlesource.com/c/go/+/214579
Reviewed-by: Jay Conrod <jayconrod@google.com>
gopherbot pushed a commit that referenced this issue Feb 27, 2020
I think this test needs to be split up eventually. It's one of
the longest tests.

Part of converting all tests to script framework to improve
test parallelism.

Updates #36320
Updates #17751

Change-Id: If2168fac040d78fd0ec3dcbdef2affd2a8f48f6d
Reviewed-on: https://go-review.googlesource.com/c/go/+/214158
Reviewed-by: Jay Conrod <jayconrod@google.com>
gopherbot pushed a commit that referenced this issue Feb 27, 2020
This is a bit complex. There's a driver program to run go with modifications
to the GOPATH used to test Windows.

Also remove the cd method on testgoData, because this was the last function
that used it.

Part of converting all tests to script framework to improve
test parallelism.

Updates #36320
Updates #17751

Change-Id: I3e8e27f37fd3701bd36b6365b128dd73b69181c0
Reviewed-on: https://go-review.googlesource.com/c/go/+/214578
Reviewed-by: Jay Conrod <jayconrod@google.com>
gopherbot pushed a commit that referenced this issue Feb 27, 2020
…ork"

This rolls forward the change golang.org/cl/214431, which was reverted
in golang.org/cl/220217. The cl was broken because
TestVersionControlErrorMessageIncludesCorrectDirectory, which is going
to be removed in golang.org/cl/214429 hadn't been submitted yet.

Original change description:

Part of converting all tests to script framework to improve
test parallelism.

Updates #36320
Updates #17751

Change-Id: I87b3f9acb8575fbcbd58d454b5f9bac4923429b3
Reviewed-on: https://go-review.googlesource.com/c/go/+/220178
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
@gopherbot
Copy link

@gopherbot gopherbot commented Jun 24, 2020

Change https://golang.org/cl/239738 mentions this issue: [release-branch.go1.14] cmd/go: convert TestBuildIDContainsArchModeEnv to the script framework

@gopherbot
Copy link

@gopherbot gopherbot commented Jun 24, 2020

Change https://golang.org/cl/239777 mentions this issue: [release-branch.go1.13] cmd/go: convert TestBuildIDContainsArchModeEnv to the script framework

gopherbot pushed a commit that referenced this issue Jun 29, 2020
…v to the script framework

Part of converting all tests to script framework to improve
test parallelism.

Fixes #39824
Updates #36320
Updates #17751

Change-Id: I69c69809fb1698c8198ef3ea00103a9acb7b6ce7
Reviewed-on: https://go-review.googlesource.com/c/go/+/214387
Run-TryBot: Michael Matloob <matloob@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
(cherry picked from CL 214387)
Reviewed-on: https://go-review.googlesource.com/c/go/+/239738
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
gopherbot pushed a commit that referenced this issue Jun 29, 2020
…v to the script framework

Part of converting all tests to script framework to improve
test parallelism.

Fixes #39823
Updates #36320
Updates #17751

Change-Id: I69c69809fb1698c8198ef3ea00103a9acb7b6ce7
Reviewed-on: https://go-review.googlesource.com/c/go/+/214387
Run-TryBot: Michael Matloob <matloob@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
(cherry picked from CL 214387)
Reviewed-on: https://go-review.googlesource.com/c/go/+/239777
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.