usage of toolqa

[rishi8011]

@yueqis @li-boxuan
anyone can guide me how to use toolqa in opendevin in detail there is no docs available on it

i am running open devin in docker as mentioned docs but i have no idea how to use toolqa properly .

[SmartManoj]

You mean toolsqa?

[yueqis]

Hi! ToolQA is a benchmark on tool usage. We integrated this benchmark in the evaluation directory. Please take a look at the README.md file of ToolQA to run evaluation.

[rishi8011]

i am running open devin using this command as mentioned in doc but how can i use toolqa with it ??

OPENDEVIN_WORKSPACE=$(pwd)/workspace
docker run -it
--pull=always
-e SANDBOX_USER_ID=$(id -u)
-e PERSIST_SANDBOX="true"
-e SSH_PASSWORD="make something up here"
-e WORKSPACE_MOUNT_PATH=$OPENDEVIN_WORKSPACE
-v $OPENDEVIN_WORKSPACE:/opt/workspace_base
-v /var/run/docker.sock:/var/run/docker.sock
-p 3000:3000
--add-host host.docker.internal:host-gateway
--name opendevin-app-$(date +%Y%m%d%H%M%S)
ghcr.io/opendevin/opendevin:0.6

[tobitege]

It is an evaluation, not a tool from inside OpenDevin, afaik.
Readme is here: https://github.com/OpenDevin/OpenDevin/tree/main/evaluation/toolqa