Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Segment #28

Merged
merged 11 commits into from
Nov 22, 2021
Merged

Segment #28

merged 11 commits into from
Nov 22, 2021

Conversation

kinto-b
Copy link
Owner

@kinto-b kinto-b commented Nov 21, 2021

This PR is fairly hefty.

A Segment class is implemented to serve as the basic building block for the Pipeline. This clarifies the link between the fundamental make_*() functions and the Pipeline object. In particular, each make_*() call constructs a Segment which is appended to the Pipeline.

The Segment class has an execute() method, which replicates the execution behaviour of the make_*() functions. Hence Segment$execute() can be run to rebuild the targets associated with a given segment without adding a new segment to the Pipeline as would be done if we called make_*() again.

This makes it straightforward to implement a Pipeline$build() method: we simply sort the Segments topologically and then call execute() on each in turn.

The Segment contains all the information that the old makepipe_result S3 class did and therefore makes the latter redundant. The execution metadata (e.g. execution result, execution time, etc.) is updated via the Segment$update_result() method whenever Segment$execute() is called

@kinto-b kinto-b added this to the 0.0.6 milestone Nov 22, 2021
@kinto-b kinto-b changed the base branch from master to dev November 22, 2021 00:59
@kinto-b kinto-b merged commit 55d44d4 into dev Nov 22, 2021
@kinto-b kinto-b deleted the segment branch November 22, 2021 23:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Refactor Pipeline Add execution time to metadata Pipeline labels aren't being applied
1 participant