Issue with faithfulness

[Robs1999]

I am occasionally encountering an error when evaluating with faithfulness. The error is at line 178, when computing the faithful statements:

176                 output = output if isinstance(output, list) else []
177                 faithful_statements = sum(

--> 178 verdict_score_map.get(dict.get("verdict", "").lower(), np.nan)
179 for dict in output
180 )

AttributeError: 'int' object has no attribute 'lower'

[shahules786]

Hi @Robs1999 , this is interesting. Would be able to provide the sample for which this issue is being encountered?