Skip to content
This repository has been archived by the owner on Aug 20, 2024. It is now read-only.

Does including out-of-vocabulary words really have to block the prediction? #3

Open
thundergolfer opened this issue May 5, 2017 · 1 comment

Comments

@thundergolfer
Copy link

thundergolfer commented May 5, 2017

I was trying to play with this to see the performance of a DMN but I found it pretty hard to write even a short "story" without using an out-of-vocabulary word. Some of the words you can't use are very common like: "has", "it's", "an".

It is possible to let the prediction still run even if out-of-vocabulary words exist? I'd imagine it would degrade performance, but it would make the demo much easier to use.

@Hrant-Khachatrian
Copy link
Member

sorry for a very late response.

using out of vocabulary words is not there as it is out of the scope of Facebook's bAbI dataset is trying to solve. i think it is possible to add support in the backend, but it might lead to unexpected results. The models were trained on very limited vocabulary.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants