go_test: recompile some archives to split internal, external tests #2579
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
go_test, like 'go test', splits tests into two separate archives: an
internal archive ('package foo') and an external archive
('package foo_test'). The library under test is embedded into the internal
archive. The external archive may import it and may depend on symbols
defined in the internal test files.
To avoid conflicts, the library under test must not be linked into the test
binary, since the internal test archive embeds the same sources.
Libraries imported by the external test that transitively import the
library under test must be recompiled too, or the linker will complain that
export data they were compiled with doesn't match the export data they
are linked with.
This PR adds a function, used in go_test, that identifies which
archives may need to be recompiled, then declares new output files and
actions to recompile them. This is an unfortunately an expensive
process requiring O(V+E) time and space in the size of the test's
dependency graph for each test.
Fixes bazelbuild#1877
This was referenced Jul 22, 2020
justaugustus
added a commit
to justaugustus/repo-infra
that referenced
this pull request
Jul 23, 2020
Should fix bazelbuild/rules_go/issues/1877 Signed-off-by: Stephen Augustus <saugustus@vmware.com>
justaugustus
added a commit
to justaugustus/repo-infra
that referenced
this pull request
Jul 23, 2020
Should address bazelbuild/rules_go/issues/1877 Signed-off-by: Stephen Augustus <saugustus@vmware.com>
jayconrod
pushed a commit
that referenced
this pull request
Jul 23, 2020
…2579) go_test, like 'go test', splits tests into two separate archives: an internal archive ('package foo') and an external archive ('package foo_test'). The library under test is embedded into the internal archive. The external archive may import it and may depend on symbols defined in the internal test files. To avoid conflicts, the library under test must not be linked into the test binary, since the internal test archive embeds the same sources. Libraries imported by the external test that transitively import the library under test must be recompiled too, or the linker will complain that export data they were compiled with doesn't match the export data they are linked with. This PR adds a function, used in go_test, that identifies which archives may need to be recompiled, then declares new output files and actions to recompile them. This is an unfortunately an expensive process requiring O(V+E) time and space in the size of the test's dependency graph for each test. Fixes #1877
jayconrod
pushed a commit
that referenced
this pull request
Jul 23, 2020
…2579) go_test, like 'go test', splits tests into two separate archives: an internal archive ('package foo') and an external archive ('package foo_test'). The library under test is embedded into the internal archive. The external archive may import it and may depend on symbols defined in the internal test files. To avoid conflicts, the library under test must not be linked into the test binary, since the internal test archive embeds the same sources. Libraries imported by the external test that transitively import the library under test must be recompiled too, or the linker will complain that export data they were compiled with doesn't match the export data they are linked with. This PR adds a function, used in go_test, that identifies which archives may need to be recompiled, then declares new output files and actions to recompile them. This is an unfortunately an expensive process requiring O(V+E) time and space in the size of the test's dependency graph for each test. Fixes #1877
jayconrod
pushed a commit
to jayconrod/rules_go
that referenced
this pull request
Aug 25, 2020
When a test contains both internal and external files, and the external files indirectly import the library under test, some package need to be recompiled so that the packages that import the library under test are compiled with the internal test archive instead. This was implemented in bazelbuild#2579. That change caused a regression: the recompiled archives didn't include input files from cdeps dependencies. This change adds a new GoArchiveData field to track direct dependencies. That can be used to rebuild a depset of all the needed files. Fixes bazelbuild#2622
jayconrod
pushed a commit
that referenced
this pull request
Aug 26, 2020
…2625) When a test contains both internal and external files, and the external files indirectly import the library under test, some package need to be recompiled so that the packages that import the library under test are compiled with the internal test archive instead. This was implemented in #2579. That change caused a regression: the recompiled archives didn't include input files from cdeps dependencies. This change adds a new GoArchiveData field to track direct dependencies. That can be used to rebuild a depset of all the needed files. Fixes #2622
jayconrod
pushed a commit
that referenced
this pull request
Sep 22, 2020
…2625) When a test contains both internal and external files, and the external files indirectly import the library under test, some package need to be recompiled so that the packages that import the library under test are compiled with the internal test archive instead. This was implemented in #2579. That change caused a regression: the recompiled archives didn't include input files from cdeps dependencies. This change adds a new GoArchiveData field to track direct dependencies. That can be used to rebuild a depset of all the needed files. Fixes #2622
jayconrod
pushed a commit
that referenced
this pull request
Sep 22, 2020
…2625) When a test contains both internal and external files, and the external files indirectly import the library under test, some package need to be recompiled so that the packages that import the library under test are compiled with the internal test archive instead. This was implemented in #2579. That change caused a regression: the recompiled archives didn't include input files from cdeps dependencies. This change adds a new GoArchiveData field to track direct dependencies. That can be used to rebuild a depset of all the needed files. Fixes #2622
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
go_test, like 'go test', splits tests into two separate archives: an
internal archive ('package foo') and an external archive
('package foo_test'). The library under test is embedded into the internal
archive. The external archive may import it and may depend on symbols
defined in the internal test files.
To avoid conflicts, the library under test must not be linked into the test
binary, since the internal test archive embeds the same sources.
Libraries imported by the external test that transitively import the
library under test must be recompiled too, or the linker will complain that
export data they were compiled with doesn't match the export data they
are linked with.
This PR adds a function, used in go_test, that identifies which
archives may need to be recompiled, then declares new output files and
actions to recompile them. This is an unfortunately an expensive
process requiring O(V+E) time and space in the size of the test's
dependency graph for each test.
Fixes #1877