[FEATURE] Add the DSG Metric

[sdiazlor]

‼️ If you want to work on this issue: please comment below and wait until a maintainer assigns this issue to you before opening a PR to avoid several contributions on the same issue. Thanks! 😊

✨ What You’ll Do

Adding more metrics makes Pruna’s evaluations more insightful, so we’re excited to welcome Davidsonian Scene Graph (DSG) into our metric lineup! 🎉

Davidsonian Scene Graph (DSG) is an automatic evaluation framework for text-to-image models inspired by formal semantics to assess faithfulness.

---

📐 Metric Class

For a detailed guide, refer to the documentation on implementing a metric: https://docs.pruna.ai/en/stable/docs_pruna/user_manual/customize_metric.html.
Also, take a look at the metric code repository here: https://github.com/j-min/DSG

Quick Overview

1. File & Class
   • Create the new file and define the class.
2. Configuration
   • Choose a sensible call_type 
   • Define the state and update functions.
   • Don't forget to register the metric
3. Testing & Docs
   • Write the unit tests.

And that's all!

✅ Acceptance Criteria

• Correct Output: compute(...) returns a sensible result
• Tests & Docs: All existing and new unit tests pass, and the documentation is updated

And don’t forget to give us a ⭐️!

---

❓ Questions?

Feel free to jump into our Discord channel if you hit any roadblocks. Can’t wait to see your contribution of this metric! 🚀

---

Share on X

[kris70lesgo]

@sdiazlor can u assign this issue to me ?