Make resizable layer work with textcat and transformers #820
Conversation
polm
added
bug
|
Can you provide a minimal (non-) working example that we can use as test case for spaCy? |
|
Made a failing test: explosion/spacy-transformers#357. For testing with a realistic pipeline, you can use the goemotions sample project with transformers and just change the TextCatCNN architecture to v2. |
This avoids setting nO if it doesn't need to be changed in the first place.
|
I would like someone more familiar with the resizable layer to look at this, but I believe the current fix is correct. To clarify what's going on: Before this PR, the layer is assumed to be initialized if After this PR, the resizable layer is considered uninitialized if either |
| elif layer.has_dim("nI") is None: | ||
| layer.set_dim("nO", new_nO, force=True) | ||
| return layer |
There was a problem hiding this comment.
Nice find, Paul. Seems correct to me!
There was a problem hiding this comment.
This should be good to merge and will make the test in explosion/spacy-transformers#357 green.
This PR is an attempted fix of the issue outlined in explosion/spaCy#11968. I don't completely understand how the resizable textcat is supposed to work yet, and am not sure if this is the right approach, but locally it got rid of errors and I was able to train a model with reasonable performance.
There are also no tests yet.