Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CnnSentenceDataSetIterator.loadSingleSentence() throws ND4JIllegalStateException if word vectors missing word #8120

Closed
cowwoc opened this issue Aug 21, 2019 · 1 comment · Fixed by SkymindIO/deeplearning4j#141

Comments

@cowwoc
Copy link

commented Aug 21, 2019

Issue Description

If a user invokes:

INDArray featuresFirstNegative = ((CnnSentenceDataSetIterator) testData).loadSingleSentence("McDonald's coffee");

where the underlying word vectors do not contain one of the words in the sentence then the following exception is thrown:

Exception in thread "main" org.nd4j.linalg.exception.ND4JIllegalStateException: Invalid shape: Requested INDArray shape [1, 1, 0, 300] contains dimension size values < 1 (all dimensions must be 1 or more)
    at org.nd4j.linalg.factory.Nd4j.checkShapeValues(Nd4j.java:4675)
    at org.nd4j.linalg.factory.Nd4j.create(Nd4j.java:4611)
    at org.nd4j.linalg.factory.Nd4j.create(Nd4j.java:4114)
    at org.deeplearning4j.iterator.CnnSentenceDataSetIterator.loadSingleSentence(CnnSentenceDataSetIterator.java:161)

Expected behavior: Clearer error message explaining that the word vectors did not contain the requested word, and specify exactly which word was missing.

This issue might be a duplicate of #3254 but it looks like it was never fixed correctly.

Version Information

Please indicate relevant versions, including, if relevant:

  • Deeplearning4j version: 1.0.0-beta4
  • Platform information (OS, etc): Windows 10.0.18362.295
  • CUDA version, if used: N/A
  • NVIDIA driver version, if in use: N/A

@AlexDBlack AlexDBlack added this to the 1.0.0-beta5 milestone Aug 21, 2019

@AlexDBlack AlexDBlack self-assigned this Aug 21, 2019

AlexDBlack added a commit to SkymindIO/deeplearning4j that referenced this issue Aug 21, 2019
eclipse#8120 CnnSentenceDataSetIterator.loadSingleSentence no words U…
…X/exception improvement

Signed-off-by: AlexDBlack <blacka101@gmail.com>
AlexDBlack added a commit to SkymindIO/deeplearning4j that referenced this issue Aug 21, 2019
Various fixes (#141)
* eclipse#8121 CnnSentenceDataSetIterator fixes

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* eclipse#8120 CnnSentenceDataSetIterator.loadSingleSentence no words UX/exception improvement

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* eclipse#8122 AggregatingSentenceIterator builder - addSentencePreProcessor -> sentencePreProcessor

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* eclipse#8082 Arbiter - fix GridSearchCandidateGenerator search size issue

Signed-off-by: AlexDBlack <blacka101@gmail.com>
@AlexDBlack

This comment has been minimized.

Copy link
Contributor

commented Aug 21, 2019

Thanks for reporting - fixed here, will be merged to eclipse/deeplearning4j master soon.
SkymindIO#141

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.