Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement FakeHandle For ResumingPipeline to respect TDA #80

Open
alexbrillant opened this issue Sep 27, 2019 · 3 comments
Open

Implement FakeHandle For ResumingPipeline to respect TDA #80

alexbrillant opened this issue Sep 27, 2019 · 3 comments

Comments

@alexbrillant
Copy link
Member

@alexbrillant alexbrillant commented Sep 27, 2019

todo :

what happens when we have different number of data inputs than transformed data inputs ?

@guillaume-chevalier

This comment has been minimized.

Copy link
Member

@guillaume-chevalier guillaume-chevalier commented Sep 30, 2019

Note: I think we need a way to use data summaries because it will fail whenever the length of data_inputs will change in a step.

This is what I mean by a data summary: data_summary_hash = hash((tuple(current_ids), dataset_original_data_summary).

Important: note that the dataset_original_data_summary is an ID that identifies the dataset, such as a hashed string of a versionned dataset name or something like that because yet we assume that the dataset never changes when we create the original_ids.

@guillaume-chevalier

This comment has been minimized.

Copy link
Member

@guillaume-chevalier guillaume-chevalier commented Sep 30, 2019

The point would be to set the summary hash once at the beggining, then it'd only be influenced by the step's versions (source code) and the steps's current hyperparams.

@guillaume-chevalier

This comment has been minimized.

Copy link
Member

@guillaume-chevalier guillaume-chevalier commented Feb 8, 2020

@alexbrillant why this was closed? (reason plz, I don't know why this is solved)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.