Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test that using identity matrix for JS embedding layer has no effect on correctness #2

Open
Darktex opened this issue Feb 15, 2016 · 0 comments

Comments

@Darktex
Copy link
Owner

Darktex commented Feb 15, 2016

The JS code was made to enable training in a browser in reasonable time. This meant that Karpathy added an extra layer before the LSTMs that mapped the vocabulary (of size ~50-100) to a much smaller embedding (the example uses size 5). We won't care about training, so given that we will only do forward propagation we should be able to afford skipping this embedding step, but the JS code expects to find a matrix Wil that contains the weights for the embedding. I have therefore hacked it a bit by setting an embedding size to be the same as the vocabulary size, and setting Wil to be an identity matrix. This needs testing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant