Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Delete temp file aggressively. #75

Closed
fukatani opened this issue Nov 12, 2017 · 1 comment
Closed

Delete temp file aggressively. #75

fukatani opened this issue Nov 12, 2017 · 1 comment

Comments

@fukatani
Copy link
Member

Delete temp file aggressively.
As refered in https://www.kaggle.com/tunguz/rgf-target-encoding-0-282-on-lb/code ,
sometimes user's environment has limited disk capacity, (ex. kaggle kernel has 1GB).

Currently, we clean temp file by @exit.register.
But some user need to delete temp file in run time.
I think it is natural that user expected release resources by garbage collector or __del__ method.

I mean,

rgf = RGFClassifier()
del rgf

But we should not expect to be called __del__ method every time.
For example, I confirmed test_parallel_gridsearch doesn't call __del__ method.

So we should also use @exit.register.

Another option to solve this problem is changing communication method between Python and C++.
For example, XGB uses ctypes.

@StrikerRUS
Copy link
Member

For those who come here after closing the issue:

You can delete temp files associated with specific estimator by calling cleanup() method:

clf = RGFClassifier()
clf.fit(X, y)
clf.cleanup()

Note, that after this operation you will ought to call fit() method again to use the estimator.

Also you can delete all temp files of created estimators:

from rgf.utils import cleanup

cleanup()

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

2 participants