Pretrained dumps as download

[malteos]

Thanks for the amazing work!

Can you provide pretrained dumps as download? That would make it much easier to test everything locally without the training step.

[wetneb]

Yes! I thought I had done that but now I remember that I got stuck by the file size limit on GitHub. I will see if I can upload that on Zenodo.

[wetneb]

I have added the PageRank vector and the bag of words model here: https://github.com/wetneb/opentapioca/releases/tag/v0.1.0
Those are the most expensive parts to train - indexing Wikidata in Solr should only take a few hours (if you restrict the types of the items as the sample profiles do).

[LiuYuLOL]

Hey, is it possible to provide the pre-trained classifier(s)? As far as I knew, with the provided two models, I still need a dataset-specific classifier.

While I'm just interested in using it on some plain text, e.g. news articles, Wikipedia pages, etc. So could you provide one classifier, e.g., the one used in the current web API? @wetneb (sorry for at you, but I'm not sure whether you can get a notification if I just write a comment.)

[wetneb]

Sure, I have added one to the release page.

[LiuYuLOL]

Sure, I have added one to the release page.

Great! Thanks for the help. The model passed the test and is run with a web server at localhost:8457.

[eracle]

Thanks, I will include them in the docker build process! are there any other files that could speed it up?

At the moment I have the following steps that I would like to replace with a downloadable artefact:

• Training BOW
• Preprocessing
• Creating Wikidata graph
• Compiling Wikidata graph
• Computing Pagerank

[senisioi]

@wetneb do you remember which sklearn version you used to pickle those models?
Loading sample_classifier.pkl is not possible with sklearn==1.1.2

[eracle]

if you check my branch there is the working version pinned:

https://github.com/eracle/opentapioca

btw
scikit-learn==0.20.3

[ziodave]

You can convert to sample classifier file to the latest sklearn 1.2 by running a couple of bbe (binary search/replace):

1. bbe -e "s/sklearn.preprocessing.data/sklearn.preprocessing/g"
2. bbe -e "s/sklearn.svm.classes/sklearn.svm/g" sample_classifier.pkl

[banyous]

Hi all,

it seems that this pretrained files release no longer compatible with recent versions of scikit-learn (>1.3), and the solution @ziodave suggests doesn't work for me.

Is there a plan or a possibility for an updated pretrained files that are compatible with the recent versions of scikit-learn?

Thanks!

[wetneb]

I'm not working on this project at the moment but I would be happy to publish models if someone could train them :)