Sequence is shorter then the required number of steps

[kaeflint]

No description provided.

[kaeflint]

Running
from nlgeval import NLGEval nlgeval = NLGEval() metrics_dict = nlgeval.compute_individual_metrics('The man and woman are on a vacation', 'The couple were on honeymoon.')
gave the ValueError: ('Sequence is shorter then the required number of steps : (n_steps, seq, seq.shape):', 1, array([], shape=(0, 1, 1), dtype=float32), (0, 1, 1))

[juharris]

Thanks for trying out our repo!
The first parameter (ref) should be a list like this:

from nlgeval import NLGEval
nlgeval = NLGEval()
metrics_dict = nlgeval.compute_individual_metrics(["The man and woman are on a vacation"], "The couple were on honeymoon.")

You can see some examples here: https://github.com/Maluuba/nlg-eval/blob/master/nlgeval/tests/test_nlgeval.py

[kaeflint]

Thanks. working well now but I realized the ROUGE scores are different from what I obtained from using the official scripts

[juharris]

Thanks for the feedback. Would you open a new issue for that with examples?

[kracwarlock]

Our overlap-based metric implementations are derived from https://github.com/tylin/coco-caption. The ROUGE metric variant they implement is called ROUGE_L. Are you talking about the ROUGE_L metric or some other variant of it. If ROUGE_L doesn't match the official eval scripts then it would be great if you can open an issue for this.