Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lazy values are not shown if they get cached across trials #78

Closed
deniederhut opened this issue Sep 12, 2023 · 2 comments
Closed

Lazy values are not shown if they get cached across trials #78

deniederhut opened this issue Sep 12, 2023 · 2 comments
Labels
bug Something isn't working
Projects

Comments

@deniederhut
Copy link

Is your feature request related to a problem? Please describe.
Not sure if this is a bug or intentional, just something I noticed when setting up an MRE from the docs.

Describe the solution you'd like
When running a trial with lazy outputs, where one of those lazy outputs is cached from a previous run, it does not get shown in the artifact manager report. I think I understand why you might not want to ever load in a lazy object to show the value, so I'm a little confused as to why it does happen, but only for non-cached outputs.

Describe alternatives you've considered
🤷‍♀️

Additional context
Here's the script I was working with:

from dataclasses import dataclass

from curifactory import ArtifactManager, ExperimentParameters, Record, stage
from curifactory.caching import Lazy, PickleCacher

@dataclass
class MyParams(ExperimentParameters):
    number: float = 1.0

@stage(inputs=None, outputs=[Lazy("initial_value")], cachers=[PickleCacher])
def get_initial_value(record):
    my_value = 5
    return my_value * record.params.number

@stage(inputs=["initial_value"], outputs=["final_value"], cachers=[PickleCacher])
def multiply_again(record, initial_value):
    return initial_value * record.params.number


if __name__ == '__main__':
    manager = ArtifactManager("experiment_one")
    param_grid = [MyParams(number=number) for number in (0, 1, 2, 2)]
    for param in param_grid:
        record = Record(manager, param)
        result = multiply_again(get_initial_value(record))
    manager.generate_report()

And the result in the html report:

Screenshot 2023-09-11 at 8 11 39 PM
@WarmCyan
Copy link
Collaborator

Good catch, this was intentional back when we first added lazy cache loading because we had no simple way around it at the time, but this would be a good thing to address now and it should be relatively trivial to do so.

The metadata that gets saved with each artifact includes the preview string ("(int) 10" etc), so this should just be a matter of grabbing it from the metadata file if it's a Lazy instance, without needing to load the underlying artifact.

I'll queue this up to fix!

@WarmCyan WarmCyan added the bug Something isn't working label Sep 18, 2023
@deniederhut
Copy link
Author

🙏

@WarmCyan WarmCyan added this to Next in Roadmap Sep 19, 2023
@WarmCyan WarmCyan moved this from Next to WIP in Roadmap Sep 19, 2023
@WarmCyan WarmCyan moved this from WIP to Done in Roadmap Sep 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
Development

No branches or pull requests

2 participants