chore: Use contexts with timeout in coderd tests
#3381
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mafredri
commented
Aug 4, 2022
| @@ -17,26 +18,38 @@ func TestPostFiles(t *testing.T) { | |||
| t.Parallel() | |||
| client := coderdtest.New(t, nil) | |||
| _ = coderdtest.CreateFirstUser(t, client) | |||
| _, err := client.Upload(context.Background(), "bad", []byte{'a'}) | |||
|
|
|||
| ctx, cancel := context.WithTimeout(context.Background(), testutil.WaitLong) | |||
There was a problem hiding this comment.
As much as possible, I tried to ensure contexts are initialized after test environment has been set up (e.g. post- coderd client, user creation etc). This ensures the timeout is applied only to the part being tested. The coderdtest methods have their own timeouts.
mafredri
force-pushed
the
mafredri/coderd-test-timeouts
branch
from
c261c88
to
0a7642c
Compare
johnstcn
approved these changes
Aug 4, 2022
| ctx, cancelFunc := context.WithCancel(context.Background()) | ||
| defer t.Cleanup(cancelFunc) // Defer to ensure cancelFunc is executed first. | ||
| ctx, cancel := context.WithCancel(ctx) // Shadowed to avoid mixing contexts. | ||
| defer t.Cleanup(cancel) // Defer to ensure cancelFunc is executed first. |
There was a problem hiding this comment.
Suggested change
| defer t.Cleanup(cancel) // Defer to ensure cancelFunc is executed first. | |
| defer t.Cleanup(cancel) // Defer to ensure cancel is executed first. |
coderd/templateversions_test.go
Outdated
Comment on lines
829
to
850
| eg.Go(func() error { | ||
| data, err := echo.Tar(nil) | ||
| if err != nil { | ||
| return err | ||
| } | ||
| file, err := client.Upload(egCtx, codersdk.ContentTypeTar, data) | ||
| if err != nil { | ||
| return err | ||
| } | ||
| templateVersion, err := client.CreateTemplateVersion(egCtx, user.OrganizationID, codersdk.CreateTemplateVersionRequest{ | ||
| TemplateID: template.ID, | ||
| StorageSource: file.Hash, | ||
| StorageMethod: codersdk.ProvisionerStorageMethodFile, | ||
| Provisioner: codersdk.ProvisionerTypeEcho, | ||
| }) | ||
| if err != nil { | ||
| return err | ||
| } | ||
|
|
||
| templateVersionIDs[i] = templateVersion.ID | ||
|
|
||
| return nil |
There was a problem hiding this comment.
This is the sort of thing where it would be really useful to have a mock database and just return some predefined fixture.
| @@ -720,22 +816,45 @@ func TestPaginatedTemplateVersions(t *testing.T) { | |||
| _ = coderdtest.AwaitTemplateVersionJob(t, client, version.ID) | |||
| template := coderdtest.CreateTemplate(t, client, user.OrganizationID, version.ID) | |||
|
|
|||
| // This test takes longer than a long time. | |||
There was a problem hiding this comment.
It apparently takes even longer than that!
|
@Emyrk tagged you as a reviewer here since I made some changes to pagination tests, in case you want to take a look ( |
Emyrk
approved these changes
Aug 9, 2022
Comment on lines
+828
to
+853
| i := i | ||
| eg.Go(func() error { | ||
| data, err := echo.Tar(nil) | ||
| if err != nil { | ||
| return err | ||
| } | ||
| file, err := client.Upload(egCtx, codersdk.ContentTypeTar, data) | ||
| if err != nil { | ||
| return err | ||
| } | ||
| templateVersion, err := client.CreateTemplateVersion(egCtx, user.OrganizationID, codersdk.CreateTemplateVersionRequest{ | ||
| TemplateID: template.ID, | ||
| StorageSource: file.Hash, | ||
| StorageMethod: codersdk.ProvisionerStorageMethodFile, | ||
| Provisioner: codersdk.ProvisionerTypeEcho, | ||
| }) | ||
| if err != nil { | ||
| return err | ||
| } | ||
|
|
||
| templateVersionIDs[i] = templateVersion.ID | ||
|
|
||
| _ = coderdtest.AwaitTemplateVersionJob(t, client, templateVersion.ID) | ||
| return nil | ||
| }) | ||
| } | ||
| err := eg.Wait() |
Comment on lines
+856
to
+867
| for i := 0; i < len(templateVersionIDs); i++ { | ||
| // We don't use coderdtest.AwaitTemplateVersionJob here because | ||
| // we can't control the timeouts, the concurrent creations take | ||
| // a while. | ||
| templateVersion, err := client.TemplateVersion(ctx, templateVersionIDs[i]) | ||
| if err == nil && templateVersion.Job.CompletedAt != nil { | ||
| continue | ||
| } | ||
| require.NotErrorIs(t, err, context.DeadlineExceeded, "template version %d not created in time", i) | ||
| // Retry. | ||
| time.Sleep(testutil.IntervalMedium) | ||
| i-- |
coderd/templateversions_test.go
Outdated
Comment on lines
928
to
929
| ctx, cancel := context.WithTimeout(context.Background(), testutil.WaitLong) | ||
| defer cancel() |
There was a problem hiding this comment.
This could be shorter since the api call shouldn't take that long.
There was a problem hiding this comment.
I lowered it 👍🏻, didn't initially put any effort into reducing the timeouts. Defaulted everything to high so it would break as little as possible.
mafredri
commented
Aug 9, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Tests sometimes flake with
(unknown)status, which is not helpful for debugging. This PR moves to having contexts with timeout in all applicable tests undercoderd.All uses of
context.Background()were replaced and code fixed to add the contexts. A few spots required extra care/rewriting due to parallel/subtests to ensure tests passed.This is a practice I feel we should adopt everywhere across the codebase, esp. since we rely heavily on parallel tests. We could consider adding helpers to
testutil(thinkctx := testutil.Context()for the default case).