Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add A.Run for running sub-tasks #116

Closed
pellared opened this issue Jul 31, 2021 · 9 comments
Closed

Add A.Run for running sub-tasks #116

pellared opened this issue Jul 31, 2021 · 9 comments
Labels
enhancement New feature or request

Comments

@pellared
Copy link
Member

pellared commented Jul 31, 2021

Why

Like testing package it would be very helpful to create something like sub-tasks.

Example use case: run go test for each Go Module within a repository.

How

Implement A.Run() method which works similarly to the one in the testing package.

We can also dogfood it e.g. to run golangci-lint for all Go modules within this repository.

We could also support running a concrete subtask e.g ./goyek Task/Sub1 (separate PR).

We could also try to support running tasks and subtasks using regex/wildcards (separate PR). go test -run regex takes regex . If we decide to take the same approach we wrap the regex with ^ $.

@pellared pellared added enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed labels Jul 31, 2021
@pellared
Copy link
Member Author

pellared commented Jan 5, 2022

@arnovanliere would this feature be helpful for you?

Personally, I would use it in repositories containing multiple Go modules. However, multi-module repos are not popular, so I decided to not work on this feature so far.

@arnovanliere
Copy link
Contributor

As a matter of fact, the repo I work on most (and in which we use goyek as well) is multi-module repo with microservices. We have a task which runs tests, go mod tidy and gofmt in every module. It runs sequentially now, but it would be nicer if it could be parallelized by goyek out of the box. We would certainly use this feature.

@pellared
Copy link
Member Author

pellared commented Jan 5, 2022

Great. I have more confidence that it is a feature worth implementing. Do you want to implement it or are you OK waiting a few weeks or months until I find time to work on it? Do you want to review the code if I would implement it?

@arnovanliere
Copy link
Contributor

I'm happy to wait a while for this feature to be available. I would certainly want to review your code.
I might start on the other issue (#113) myself (might take a while as well) as I find it a very useful feature too.

@pellared
Copy link
Member Author

As a matter of fact, the repo I work on most (and in which we use goyek as well) is multi-module repo with microservices. We have a task which runs tests, go mod tidy and gofmt in every module. It runs sequentially now, but it would be nicer if it could be parallelized by goyek out of the box. We would certainly use this feature.

@arnovanliere See #179

@pellared pellared changed the title Add TF.Run() for running sub-tasks Add TF.Run for running sub-tasks Oct 12, 2022
@pellared pellared removed the help wanted Extra attention is needed label Oct 14, 2022
@pellared
Copy link
Member Author

pellared commented Oct 16, 2022

I think this is not needed and would be overengineering. Here is an easy example how it could overcome:

func SomeTask(tf *goyek.TF) {
	input := []string { ".", "build", "tools" }
	for _, in := range input  {
		func() {
			tf.Logf("For %q:", in)
			if err := exec1(in); err != nil {
				tf.Error(err)
				return
			}
			if err := exec2(in); err != nil {
				tf.Error(err)
			}
		}()
	}
}

@pellared pellared added the wontfix This will not be worked on label Oct 16, 2022
@pellared pellared changed the title Add TF.Run for running sub-tasks Add A.Run for running sub-tasks Dec 9, 2023
@pellared
Copy link
Member Author

pellared commented Dec 9, 2023

Reopening as together with A.Parallel function it could be a very handy feature.

@pellared pellared reopened this Dec 9, 2023
@pellared pellared removed the wontfix This will not be worked on label Dec 9, 2023
@pellared
Copy link
Member Author

pellared commented Jan 25, 2024

Task.Parallel has been added.
Now we can even create parallel "sub-tasks" like this:

import (
	"github.com/goyek/goyek/v2"
	"github.com/goyek/x/cmd"
)

var installMigrate = goyek.Define(goyek.Task{
	Name:  "install-migrate",
	Usage: "install migrate",
	Action: func(a *goyek.A) {
		cmd.Exec(a, "migrate-installer")
	},
})

func init() {
	var migrateTasks goyek.Deps
	dbs := []string{"staging-db-1", "staging-db-2"}
	for _, db := range dbs {
		migrateTasks = append(migrateTasks, goyek.Define(goyek.Task{
			Name:     "migrate-" + db,
			Deps:     goyek.Deps{installMigrate},
			Parallel: true,
			Action: func(a *goyek.A) {
				cmd.Exec(a, "migrate "+db)
			},
		}))
	}

	goyek.Define(goyek.Task{
		Name:  "migrate",
		Usage: "migrate",
		Deps:  migrateTasks,
	})
}

The advantages are:

  • no need to change the API and no need to create dedicated functions for handling serial and parallel subtasks
  • ability to select a single task (sub-task) to be executed (or skipped)

@pellared
Copy link
Member Author

Closing right now as YAGNI. I can consider reopening it in future.

@pellared pellared closed this as not planned Won't fix, can't repro, duplicate, stale Jan 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants