New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Value of "n_characters" in char embedding #64

Closed

varunkumar-dev opened this issue Jul 23, 2018 · 1 comment

varunkumar-dev commented Jul 23, 2018 •

edited

Loading

In train_model.py, "n_characters" is defined as 261. However, in pretrained models's configs, n_characters is set to 262. Any particular reason?

Test model : https://raw.githubusercontent.com/allenai/bilm-tf/master/tests/fixtures/model/options.json
Pretrained model : https://s3-us-west-2.amazonaws.com/allennlp/models/elmo/2x4096_512_2048cnn_2xhighway/elmo_2x4096_512_2048cnn_2xhighway_options.json

Both models have n_characters=262

Moreover, while reading a pre-trained model, we increase the size by one to add padding

bilm-tf/bilm/model.py

Line 220 in 81a4b54

# Have added a special 0 index for padding not present

But we already have a special char for padding

bilm-tf/bilm/data.py

Line 120 in 81a4b54

self.pad_char = 260 # <padding>

Contributor

matt-peters commented Jul 30, 2018

See the comments I just added to the README that explain this: https://github.com/allenai/bilm-tf/blob/master/README.md#whats-the-deal-with-n_characters-and-padding

matt-peters closed this as completed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment