Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generated models for WEASEL are large in size #15

Closed
mohataher opened this issue Oct 5, 2017 · 2 comments
Closed

Generated models for WEASEL are large in size #15

mohataher opened this issue Oct 5, 2017 · 2 comments

Comments

@mohataher
Copy link
Contributor

When working with WEASEL, I noticed save() method generates a seriously large file to represent the classifier model; approximately 300 MB model for a moderately small dataset. I'm not sure exactly why but my first guess it's saving training data as it's linked somewhere in the dependencies of the objects.

Do you have any idea?

@patrickzib
Copy link
Owner

Hmm, is there any way to list the sizes of objects using kyro?

There are some possible directions:

  • HashMaps may have been initialized with to large sizes. I am not sure, if there is an easy way to compress (reducing the size of the underlying array) them in the hppc library.

  • Liblinear: I am not sure what liblinear actually stores when serialized to disk. It does have a save method though. Maybe if would be better to use it for serialization?

@patrickzib
Copy link
Owner

patrickzib commented Oct 5, 2017

Update: I found a way in SFA to free memory. Could you check again?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants