Tokenizer TF.Data Transform Crashes

[luischinchillagarcia]

When using the .apply method as a transformation for a tf.data object on to a text tokenizer, it crashes all Colab sessions (both in TF 1.x and 2.0). Here is a sample taken from the official colab notebook.

docs = tf.data.Dataset.from_tensor_slices([['Never tell me the odds.'],
                                           ["It's a trap!"]])
tokenizer = text.WhitespaceTokenizer()
tokenized_docs = docs.map(lambda x: tokenizer.tokenize(x))
iterator = tokenized_docs.make_one_shot_iterator()
print(iterator.get_next().to_list())
print(iterator.get_next().to_list())

[broken]

We noticed this recently and identified the problem and should have this fixed soon.

[broken]

fyi, you can force an earlier beta version to address this in the meantime.
!pip install tensorflow-text
!pip install tensorflow==2.0.0b1

[broken]

I realized my earlier responses were vague. The issue is that the version of TF was nightly and was causing incompatibilities. I will update the colab to force specific versions of TF to match our own to prevent these issues.

Both of these work:
!pip install tensorflow-text==0.1.0rc2
!pip install tensorflow==1.14.0
or
!pip install tensorflow-text
!pip install tensorflow==2.0.0b1

[broken]

We now have releases for TF 1.15 & 2.0, so I'm closing this issue.