Skip to content

Provenance for Iterative Job #19

@miryung

Description

@miryung

We need to forward carry input identifiers to each output, as a bit map. Bit map is the entire map of any input contributing to the output.

At the end of each iteration, we will update the bit map for the output for n_th iteration , using the bitmap for n-1th iteration. This will require at least linear scan of all outputs in the table, and updating the bit map through union/ or join.

We may be able to make the processing faster, once the bit map saturates through iterative fix point, and if all input data contributes to the all outputs, or if all input data bit map member does not change after iteration.

Miryung

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions