Evaluation and agreement scripts for the DISCOSUMO project. Each evaluation script takes both manual annotations as automatic summarization output. The formatting of these files is highly project-specific. However, the evaluation functions for precision, recall, ROUGE, Jaccard, Cohen's kappa and Fleiss' kappa may be applicable to other domains too.

This is a Pytorch implementation of a summarization model that is fine-tuned on the top of Google-T5 pre-trained model.

Text summarization of Hotel reviews using Huggingface Transformers T5 and Bart

A news headline generator finetuned on T5-base.

ROUGE L metric implementation using tensorflow ops

Evaluation tools for image captioning. Including BLEU, ROUGE-L, CIDEr, METEOR, SPICE scores.

A serverless set of functions for evaluating whether incoming messages to an LLM system seem to contain instances of prompt injection; uses cascading cosine similarity and ROUGLE-L calculation against known good and bad prompts

Python ROUGE Score Implementation for Chinese Language Task (official rouge score)

Python implementation of ROUGE

ROUGE automatic summarization evaluation toolkit. Support for ROUGE-[N, L, S, SU], stemming and stopwords in different languages, unicode text evaluation, CSV output.

Evaluation code for various unsupervised automated metrics for Natural Language Generation.