Skip to content

ci: Migrate the remaining jobs to an alternative CI runner#10503

Merged
gerhard merged 2 commits into
dagger:mainfrom
gerhard:ci-migrate-remaining-jobs-to-alternative-ci-runners
Jun 2, 2025
Merged

ci: Migrate the remaining jobs to an alternative CI runner#10503
gerhard merged 2 commits into
dagger:mainfrom
gerhard:ci-migrate-remaining-jobs-to-alternative-ci-runners

Conversation

@gerhard
Copy link
Copy Markdown
Contributor

@gerhard gerhard commented May 29, 2025

We have been hitting networking issues in AWS, this should address them and also speed things up.

Also, use the platform cache across the SDK jobs since it seems to speed each job up by a good few minutes.

@gerhard gerhard requested review from jedevc and matipan May 29, 2025 19:23
Comment thread .github/main.go
{"call-and-shell", []string{"TestCall", "TestShell", "TestDaggerCMD"}, &dagger.GhaJobOpts{
Runner: AltGoldRunner(),
}},
{"container", []string{"TestContainer"}, &dagger.GhaJobOpts{}},
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: we should allow this to be nil, and handle that in splitTests.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried this, and the explicit &dagger.GhaJobOpts{} felt more honest & easier to follow for me.

It also made it easier to know what type that nil was keeping the place of. Adding extra properties was also easier - the type was already there.

Adding the diff here for completeness, but the current implementation feels right to me:

diff --git a/.github/main.go b/.github/main.go
index f0e320861..7d6a9a908 100644
--- a/.github/main.go
+++ b/.github/main.go
@@ -197,19 +197,19 @@ func (ci *CI) withTestWorkflows(runner *dagger.Gha, name string) *CI {
 			Runner: AltBronzeRunnerWithCache(),
 		})).
 		With(splitTests(runner, "testdev-", true, []testSplit{
-			{"cgroupsv2", []string{"TestProvision", "TestTelemetry"}, &dagger.GhaJobOpts{}},
+			{"cgroupsv2", []string{"TestProvision", "TestTelemetry"}, nil},
 			{"modules", []string{"TestModule"}, &dagger.GhaJobOpts{
 				Runner: AltPlatinumRunner(),
 			}},
 			{"module-runtimes", []string{"TestGo", "TestPython", "TestTypescript", "TestElixir", "TestPHP", "TestJava"}, &dagger.GhaJobOpts{
 				Runner: AltPlatinumRunner(),
 			}},
-			{"container", []string{"TestContainer"}, &dagger.GhaJobOpts{}},
-			{"LLM", []string{"TestLLM"}, &dagger.GhaJobOpts{}},
-			{"cli-engine", []string{"TestCLI", "TestEngine"}, &dagger.GhaJobOpts{}},
-			{"client-generator", []string{"TestClientGenerator"}, &dagger.GhaJobOpts{}},
-			{"interface", []string{"TestInterface"}, &dagger.GhaJobOpts{}},
-			{"call-and-shell", []string{"TestCall", "TestShell", "TestDaggerCMD"}, &dagger.GhaJobOpts{}},
+			{"container", []string{"TestContainer"}, nil},
+			{"LLM", []string{"TestLLM"}, nil},
+			{"cli-engine", []string{"TestCLI", "TestEngine"}, nil},
+			{"client-generator", []string{"TestClientGenerator"}, nil},
+			{"interface", []string{"TestInterface"}, nil},
+			{"call-and-shell", []string{"TestCall", "TestShell", "TestDaggerCMD"}, nil},
 			{"everything-else", nil, &dagger.GhaJobOpts{
 				Runner: AltPlatinumRunner(),
 			}},
@@ -239,12 +239,16 @@ func splitTests(runner *dagger.Gha, name string, dev bool, splits []testSplit) d
 			}
 			doneTests = append(doneTests, split.tests...)
 
-			opts := *split.opts
+			opts := dagger.GhaJobOpts{}
+			if split.opts != nil {
+				opts = *split.opts
+			}
 			opts.TimeoutMinutes = 30
 			opts.UploadLogs = true
 			if dev {
 				opts.DaggerVersion = "."
 			}
+
 			w = w.WithJob(runner.Job(name+split.name, command, opts))
 		}
 		return w

Gerhard Lazu added 2 commits June 2, 2025 19:39
We have been hitting networking issues in AWS, this should address them
and also speed things up. Let's check this before merging.

Also, use the platform cache across the SDK jobs since it seems to speed
**each** job up by a good few minutes.

Signed-off-by: Gerhard Lazu <gerhard@dagger.io>
- let it pick which db source to use
- resolve deprecated vuln-type

Signed-off-by: Gerhard Lazu <gerhard@dagger.io>
@gerhard gerhard merged commit 5454292 into dagger:main Jun 2, 2025
57 checks passed
@gerhard gerhard deleted the ci-migrate-remaining-jobs-to-alternative-ci-runners branch June 3, 2025 05:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants