Understanding BLEU Score ('bleu_n')

[Axe--]

Hey, how are different bleu scores calculated?

For the give snippet, why are all bleu(n) scores identical?
And how does this relate to nltk's sentence_bleu (weights) ?

from jury import Jury

scorer = Jury()
predictions = [
    ["the cat is on the mat", "There is cat playing on the mat"], 
    ["Look!    a wonderful day."]
]
references = [
    ["the cat is playing on the mat.", "The cat plays on the mat."], 
    ["Today is a wonderful day", "The weather outside is wonderful."]
]
scores = scorer(predictions=predictions, references=references)

Output:

{'empty_predictions': 0,
 'total_items': 2,
 'bleu_1': {'score': 0.42370250917168295,
  'precisions': [0.8823529411764706,
   0.6428571428571429,
   0.45454545454545453,
   0.125],
  'brevity_penalty': 1.0,
  'length_ratio': 1.0,
  'translation_length': 11,
  'reference_length': 11},
 'bleu_2': {'score': 0.42370250917168295,
  'precisions': [0.8823529411764706,
   0.6428571428571429,
   0.45454545454545453,
   0.125],
  'brevity_penalty': 1.0,
  'length_ratio': 1.0,
  'translation_length': 11,
  'reference_length': 11},
 'bleu_3': {'score': 0.42370250917168295,
  'precisions': [0.8823529411764706,
   0.6428571428571429,
   0.45454545454545453,
   0.125],
  'brevity_penalty': 1.0,
  'length_ratio': 1.0,
  'translation_length': 11,
  'reference_length': 11},
 'bleu_4': {'score': 0.42370250917168295,
  'precisions': [0.8823529411764706,
   0.6428571428571429,
   0.45454545454545453,
   0.125],
  'brevity_penalty': 1.0,
  'length_ratio': 1.0,
  'translation_length': 11,
  'reference_length': 11},
 'meteor': {'score': 0.5420511682934044},
 'rouge': {'rouge1': 0.7783882783882783,
  'rouge2': 0.5925324675324675,
  'rougeL': 0.7426739926739926,
  'rougeLsum': 0.7426739926739926}}

[devrimcavusoglu]

Hey @Axe--, there seems to be a bug with bleu which doesnt regard max_order currently, I'll look into it ASAP. Nice catch btw.

[devrimcavusoglu]

It is currently fixed in #59. I'll draft a new release today.

[devrimcavusoglu]

@Axe-- , the release is out. It is fixed in the new version (2.1.2). Thanks for bringing it out 👍

[Axe--]

Thanks! :-)