New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ci: Move more tests to linux-only integration tests #11250
Conversation
2367325
to
32ebfb0
Compare
Changelog[uncommitted] (2022-11-08) |
32ebfb0
to
28b1ae2
Compare
@@ -7,10 +7,10 @@ public MyStack() | |||
// Create an AWS resource (S3 Bucket) | |||
var r = new Random( | |||
"default", 10, new ComponentResourceOptions{ | |||
PluginDownloadURL = "get.com", | |||
PluginDownloadURL = "get.example.test", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This applies to other test-content changes, but what's the reason for this change?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I really don't like it when uncontrolled third party URLs/domain names are in tests. If everything is correctly mocked it's fine, but if not suddenly dev machines & CI are making requests to who-knows-what.
It's even scarier here because these would potentially cause us to download and install a tar.gz file in CI! I checked and that doesn't happen here, but whew, kind of scary.
The IETF reserved .test
as a top-level domain that will never be registered or respond to DNS. (Same with .example
, .invalid
, a few others.)
bors merge |
Build succeeded: |
Reduces the set of smoke tests run on Windows & macOS, balances the reduced parallelism #11249 by moving tests which don't seem to obviously entail cross-platform behavior changes. The tests moved to only run on Linux environments are listed below and except for the first, all were split from
integration_test.go
.TestRefreshGo
checks that the Pulumi.yamloptions.refresh
option is obeyed (moved to integration_go_test from integration_go_smoke_test)TestStackTagValidation
which tests Pulumi.yaml validationTestStackInitValidation
which tests an error on runningpulumi stack init
TestConfigPaths
which testspulumi config set
under a wide variety of inputsTestDestroyStackRef
which tests runningpulumi destroy
on a stack with a large stack referenceTestJSONOutput
which tests output JSON output diagnosticsTestExcludeProtected
which tests behavior around marking resources protectedTestProviderDownloadURL
which tests thePluginDownloadURL
resource option, but does not actually perform plugin acquisition.These tests either didn't seem to be obviously platform specific or there was a narrower test could verify that parsing and handling the same files worked correctly. As examples of those, the remaining tests that will run on smoke tests (all platforms) are:
TestConfigSave
which testspulumi config set
and stack config parsingTestRotatePassphrase
which tests changing the local secrets provider passphrase via stdin - which hypothetically could vary across platformsTestJSONOutputWithStreamingPreview
tests streaming output and event capturingTestPassphrasePrompting
tests stdin/stdout behavior in prompting for passphrases.Lastly, the
TestProviderDownloadURL
test was modified to ensure we wouldn't make requests to uncontrolled third party URLs. Though the tests don't acquire plugins, we don't control "get.com", so the.test
TLD is used to ensure test behavior cannot depend on that domain.