Implement a way test local models

[bennmann]

Hello,

Wonderful work on Voyager.

Please consider added local model support (instead of openai package - using something like Python requests package to a localhost local model using openai completion elements).

I believe Voyager is a superb way to measure future Large Language models against one another using the Minecraft diamond challenge.

Please consider a collaboration with Eleuther AI on locally hosted LLM evaluation via Voyager

https://github.com/EleutherAI/lm-evaluation-harness

[github-actions]

This issue is stale because it has been open for 30 days with no activity.

[bennmann]

Please reconsider this issue in light of Llama 3 8B and 70B.

[github-actions]

This issue is stale because it has been open for 30 days with no activity.

[Khaogamermain01]

Any updates on this?