Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow '_ensure_save_result' to be skipped with a flag in any place it is called. #513

Open
GeraldIr opened this issue Dec 4, 2023 · 4 comments

Comments

@GeraldIr
Copy link
Member

GeraldIr commented Dec 4, 2023

Often I find myself being unable to start jobs that have different result nodes which are incompatible with save_result and there has to be some workaround implemented for handling these edge cases.

I'd like to propose an optional flag for methods such as create_job and download which allows ensure save result to be skipped so that no superfluous save_result node gets added.

I can also create a PR for this if a change like this is approved.

@jdries
Copy link
Collaborator

jdries commented Dec 4, 2023 via email

@GeraldIr
Copy link
Member Author

GeraldIr commented Dec 4, 2023

We are currently using save_ml_model which stores results of random forest regression, we debated including this functionality in save_result but decided the outputs were different enough to justify its own process (and this might in the future be the case for more things, like UDFs).

I think for more advanced users having the option would be a benefit, and making it optional would save the users relying on it currently from any trouble.

@soxofaan
Copy link
Member

soxofaan commented Dec 7, 2023

is save_ml_model the only use case you're talking about? Because we can easily whitelist that process alongside save_result in the logic.

@soxofaan
Copy link
Member

soxofaan commented Dec 7, 2023

ah no wait, save_ml_model is already whitelisted in MlModel.create_job() so I'm not sure I fully understand the problem here.

can you provide some examples in code snippets?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants