feat: models compression, fix bugs in clfs (APP-121) #52
Conversation
…sourcerer-app into develop-oxxxymiron
| var pklObject = PickleUtil.unpickle(storage) | ||
| val pipeline = pklObject as Pipeline | ||
| pklObject = PMMLPipeline().setSteps(pipeline.getSteps()) | ||
| val pmml = pklObject.encodePMML() |
There was a problem hiding this comment.
Initial loading of js evaluator takes 3 minutes on my machine, it's ok for first launch, but would be better if we can optimize time for subsequent launches. What if we generate PMML files from pickle on the first launch and then use PMML files to instantiate evaluators, will it be faster?
There was a problem hiding this comment.
If one uses not all languages - it might lead to 200 mb as before.
There was a problem hiding this comment.
I think that it's ok to generate such files locally, the problem was with transmission of such big files via internet. Also, I'm suggesting to generate evaluators on demand.
There was a problem hiding this comment.
I think that it's ok to generate such files locally, the problem was with transmission of such big files via internet.
I understand.
Also, I'm suggesting to generate evaluators on demand.
I thought we already do that..
There was a problem hiding this comment.
And cache it to PMML files for subsequent launches, see first comment
* fix: extract lib lines from diff only * chore: swap nums in lib stats * fix: extract lib lines from diff only * chore: swap nums in lib stats * feat: load models from *.pkl.z directly * Add external lib * feat: cach to pmml file * wip: add handling exceptions while caching * wip: more exceptions * chore: fix 80 column limit, fix log messages * chore: remove models.zip, add more logs
No description provided.