"Exception: stream did not contain valid UTF-8" request solution

[hsupeter]

Hi Guys,
I run Ch3 KantaiBERT.ipynb instruction, tokenizer.train(files=paths, vocab_size=52_000, min_frequency=2, special_tokens=[ "", "", "", "", "",] ) in step 3: Training a Tokenizer, and get the error "Exception: stream did not contain valid UTF-8". I try to search solution in web, some people got the similar problem as well, but don't solve it. Can anyone tell me how to resolve it?
I skip this trouble cell and run following cells, they can execute normally. So, are correct the outcomes of following cells?
thanks

[hsupeter]

Hi Guys,
I resolve my problem. At first, I run my program in google colab and connect google drive (from google.colab import drive drive.mount('/content/drive')). This act cause an error by running instruction, 'paths = [str(x) for x in Path(".").glob("**/*.txt")]', to import too many txt files of non UTF-8 format. Disconneting google drive, reseting program and running it again, the error is gone.
Regards

[packtjaniceg]

Hi @hsupeter , Please find the author's reply below:

'I carefully read your request for a reader for KantaiBERT.ipynb, Chapter 3, Transformers for NLP.
The program should be run on Google Colaboratory for free with a free Gmail account.
To make sure, the user should click each cell one by one, and skip none.

To help the user, I did the following:
a) opened the notebook in Google Colaboratory
b) ran the notebook cell by cell
c) recorded the whole process from cell 1 to the last cell
d) uploaded the video on YouTube at the following link:

https://youtu.be/9l3OX0uKYN8

If this process is followed on Google Colabotory, the notebook should work.

I hope this helps.'