Log more dpo metrics #610
Log more dpo metrics #610
Conversation
There was a problem hiding this comment.
Thanks a lot @maxjeblick
Super helpful to have more metrics logged. Ideally, we would also want to show them in the UI, but having them on neptune and in the log files, is already much better.
Please have another look on the path change for the gpt_templates. I believe this slipped into your PR somehow. Otherwise, looks good to me!
| prompt_template_directory = os.path.join( | ||
| os.path.dirname(__file__), "../../prompts" | ||
| ) | ||
| self._possible_values["metric_gpt_template"] = possible_values.String( | ||
| values=(f.split(".")[0] for f in os.listdir("prompts")) | ||
| values=(f.split(".")[0] for f in prompt_template_directory) |
There was a problem hiding this comment.
Why this change? It fails, as the path is relative to the root app dir.
There was a problem hiding this comment.
Thanks, will check. Reason for the change is to resolve prompts directory w.r.t. llm_studio/python_configs/text_causal_language_modeling_config.py to be able to run tests with root directory == directory where test is running (this is the default in PyCharm when a test is started within the code editor).
This PR logs more metrics for DPO training (in particular cross entropy loss).
I refactored train.py a bit, it is now also possible to automatically log additional metrics/losses by adding them to the model's output dictionary using
additional_log_prefix (e.g.outputs["additional_log_chosen_rewards"] = ...)Related to #594