Skip to content
This repository has been archived by the owner on Mar 6, 2020. It is now read-only.

Actions, Tasks and Executors #305

Merged
merged 22 commits into from Aug 18, 2015
Merged

Actions, Tasks and Executors #305

merged 22 commits into from Aug 18, 2015

Conversation

davecheney
Copy link
Contributor

This PR is a proposal for a new method for scheduling work to be done in gb. The current method of Target interfaces works ok, but feels convoluted and is less testable than I would like because cause and effect are separated by a long distance.

This PR will introduce three distinct phases in gb.

  1. Loading packages. A gb.Context is populated with packages according to the rules of the context and the project.
  2. Action creation; the map of gb.Packages loaded into a context is transformed into a graph of actions. At this point the set of work to be performed is fully formed, and can be visualised for correctness.
  3. Execution, the root of action graph is passed to an executor to be executed, this could be as simple as a depth first traversal, or creating a goroutine to run every action after waiting for all dependant actions.

This removes the complicated dance of ctx.Run which sometimes runs actions immediately, and sometimes stores a pointer to their result for later. In this new model, Task functions can always be executed immediately, and Actions are the handle to execute them later. Error handling is also improved. Errors produced during phases 1 and 2 are surfaced immediately, rather than being smuggled into phase 3.

Actions and Tasks

Actions and Tasks allow gb to separate the role of describing the order in which work will be done, from describing that work itself. Actions are the former, they describe the graph of dependencies between actions, and thus the work to be done. By traversing the action graph, we can do the work, execute the Tasks in a sane order.

Tasks describe the work to be done, without being concerned with the order in which the work is done -- that is up to the code that places Tasks into actions. Tasks also know more intimate details about filesystems, processes, file lists, etc that Actions do not.

Action graphs (they are not strictly trees as branches converge on base actions) contain only work to be performed, there are no Actions with empty Tasks or Tasks which do no work.

Executors

Actions are executed by Executors, but can also be transformed, mutated, or even graphed.

  • cgo14 support
  • cgo15 support
  • integrate with cmd/gb
  • gb test
  • finish execute concurrent

Here is a sample -dotfile
outfile

@davecheney davecheney added this to the cross-compilation milestone Aug 17, 2015
- fix concurrent executor not waiting for all actions to exit
- pass error back to caller in sync executor
Move the debugging output from the build to the top level action.
Now that permits are handled elsewhere, run and runOut don't need
to live on the context any more.
Also integrated dot debugging into test
davecheney added a commit that referenced this pull request Aug 18, 2015
@davecheney davecheney merged commit ccb9843 into master Aug 18, 2015
@davecheney davecheney deleted the actions-tasks-executors branch August 18, 2015 06:41
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant