-
Notifications
You must be signed in to change notification settings - Fork 225
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update scispacy version on streamlit demo #342
Comments
The streamlit seems to be running scispacy_lg version 0.2.4 and the one you can currently download is 0.4.0, that might be the issue (I ran into this a couple of times when dealing with incompatibilities with spacy 2.X): "lang":"en" EDIT: (The only way I know of to get the specific version of models is to wget from the old link and pip install from file, you can't specify versions in pip install from the links, it automatically gets overwritten with the most up-to-date one) Second EDIT: (I just tried the pip install with specific version link and it works) |
You are not referring to the "specialized NER model" here though, right? (and specifically, I can see that the streamlit demo loads two spacy models, AFAICT, the results of the specialized NER in the streamlit demo depends only on |
You're totally right. I couldn't find which version of en_ner_jnlpba_md they are using on streamlit demo, but given that en_core_sci_lg was older, it wouldn't surprise me if the en_ner_jnlpba_md was too. EDIT: with version 0.3.0 of en_ner_jnlpba_md and spacy 2.3.2 I got: while with 0.4.0 (and spacy 3.0.5) I got: |
@MichalMalyska Yeah you are totally right. When I load the |
@danielkingai2 I guess the bigger underlying problem is why are the 0.4.0 models so much worse than the older versions. |
I think this could be one reason: |
I did my best to match everything to the old versions, and our reported accuracy didn't drop much I don't think, but there are a bunch of hyperparams that we haven't really done any search over, just tried to use whatever spacy is using. If you wanted to play around with retraining with different hyperparameters or something, all the training scripts should be clear from project.yml |
As an original author of explosion/spaCy#8138 (which has been closed), I still keep trying to figure out what has changed. |
I am getting different results for the same input text when I use the streamlit demo vs. when I run the code locally. The text in question:
NER results using
"en_ner_jnlpba_md"
on streamlit demoThen, running things locally:
My
pyproject.toml
has the following dependencies:Any idea what might be causing this? I consider the streamlit demo response to be more correct, and am interesting in getting the same result locally!
Also, I am only showing one example here, but I found I could quickly come up with other examples where the streamlit demo specialized NER results were better (IMO) than the results I got locally. A second example is:
The text was updated successfully, but these errors were encountered: