Using pretrained models

[syleedandekar]

The paper mentions that you performed end-to-end validation of AlpacaFarm. Do you have the code up on Github for that? I want to use the LLM pre-trained on human preferences to generate some more preferences.

[YannDubs]

For validation you can check: https://github.com/tatsu-lab/alpaca_farm?tab=readme-ov-file#running-automatic-evaluation or directly use AlpacaEval

This is the reward model from human preferences: https://huggingface.co/tatsu-lab/alpaca-farm-reward-model-human-wdiff
Ans the RLHF model: https://huggingface.co/tatsu-lab/alpaca-farm-ppo-human-wdiff

[syleedandekar]

I've been trying to generate text using ppo-human but I've just been getting gibberish. It works fine when I use LLama2. Is there an example in AlpacaEval I can refer to?

[YannDubs]

here are some outputs: https://github.com/tatsu-lab/alpaca_eval/blob/main/results/alpaca-farm-ppo-human/model_outputs.json

decoding configs: https://github.com/tatsu-lab/alpaca_eval/blob/main/src/alpaca_eval/models_configs/alpaca-farm-ppo-human/configs.yaml