Enable specifying a custom path for model configs ?

[stoicio]

Is it possible to run alpaca-eval using a custom model config in a directory path that is out of the alpaca-eval repo ? I see the base path is explicitly set to the default here

Enabling this would help avoid forking alpaca-eval or installing it in editable mode & just use it directly from pip install. It looks like its possible to specify annotator config from a different root path - link - Similar pattern for model config and a more explicit cli flag increase usability of the repo. Any thoughts ?

[YannDubs]

Hey @stoicio, I think that it is already possible if you provide an absolute path!

Not sure it's worth adding a CLI flag for relative path given that you would need to write as much as if you use absolute paths. Let me know if you have issues with absolute paths.

[YannDubs]

Tested and it should work!

[stoicio]

I didn't know about the Path lib's / behavior when used with absolute path. TIL !
Thanks for checking it out.