Flexible simulation and evaluation framework for generative IR. [demo] [docs]
Tested in node 22.15.0 with npm 11.3.0.
If you are not within the Webis network, you first need to replace all occurrences of the URL https://llm.srv.webis.de/api/chat and the model (default) in the configuration with the values for a server you have access to (OpenAI-compatible API). Then,
npm install
node bin/genirsim static/configurations/discussion.json > eval.json
To run the web server:
npm install
node bin/genirsim-server
or
GENIRSIM_VERSION=$(jq -r '.version' package.json)
docker run --rm -it -p 8000:8000 ghcr.io/webis-de/genirsim:${GENIRSIM_VERSION}