Sentence Piece: learn.predict throws AttributeError: 'NoneType' object has no attribute 'EncodeAsPieces'

[hiromis]

Here is a stripped down version of fastbook chapter 10. As soon as I replace SpacyTokenizer by SentencePieceTokenizer, learn.predict results in an error:
https://gist.github.com/hiromis/39c3b378cc08c15a6c68b51960ea18c5

This is the exact same notebook running on the same machine with one difference tok_func=SpacyTokenizer:
https://gist.github.com/hiromis/1115a1135a2d32ae1f954608c9e416e5

Here is some of the discussion:
https://forums.fast.ai/t/sentencepiece-nonetype-object-has-no-attribute-encodeaspieces/73953

The issue is still there in the current master head b3254d1

[jph00]

Appears to be fixed now - thanks for letting us know about this.