How to reproduce RTC Eval 100% locally?

[botelhorui]

Hello,

I tried to read the papper and docs on optillm and pathwork but couldn't figure out how to run RTC eval 100% locally.

Can you give guidance?

This would be usefull to test local models and modifications to optillm.

Thank you!

Great work

[codelion]

It is not yet implemented as an eval in optillm, but that's a great suggestion. Currently, there is rto round-trip optimization approach that is implemented based on the same idea in the repo here - https://github.com/codelion/optillm/blob/main/optillm/rto.py

I will add it as a plugin such that it can be combined with any approach to evaluate it at the end.

[codelion]

I have added the script to reproduce RTCEVal in #108