Skip to content

Conversation

@todtom
Copy link
Contributor

@todtom todtom commented Dec 27, 2016

Fix some bugs like tensorflow/tensorflow#5118.

@todtom
Copy link
Contributor Author

todtom commented Dec 27, 2016

I run it for the Chinese-English translation task. When use 'basic_tokenizer' it often reports the error like 'TypeError: cannot use a bytes pattern on a string-like object'. It works for me after fix the bugs. I'm not sure whether the modified 'initital_vocabulary' works for some other tokenizers

@zsdonghao
Copy link
Member

@todtom Hi, thank you for your contribution. I wonder which TF version are you using? As the code seem to work fine previously, we need to make sure other users can also run their scripts. Thanks~

@wagamamaz
Copy link
Collaborator

@todtom after you fix the bugs, can it work well in English-French translation task?

@todtom
Copy link
Contributor Author

todtom commented Dec 28, 2016

@zsdonghao the enviroment as follows:

Windows10, python 3.5.2.3, tensorflow 0.12.0, tensorlayer 1.2.8

maybe the error caused by system? I haven't run it in Linux yet
@wagamamaz I train a toy 'en-fr' model which use 1k sentences for training data, it seems work well.

@zsdonghao
Copy link
Member

@todtom great, it may caused by python3.5.

@zsdonghao zsdonghao merged commit 10d73fa into tensorlayer:master Dec 28, 2016
@todtom todtom deleted the patch-1 branch December 28, 2016 11:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants