How to run OpenDevin with the SWEBench full dataset?

[ramsey-coding]

I am trying to run OpenDevin using the full SWEBench dataset and need some guidance on the process.

Specifically, I would like to know:

• What are the steps to set up the environment for OpenDevin to work with the SWEBench dataset?
• Are there any specific configuration files or settings required for this integration?
• Are there any sample scripts or examples available that demonstrate how to run OpenDevin with a subset of the SWEBench dataset?

[xingyaoww]

We don't yet support running on the full SWEBench dataset -- but. you can check this document for running it on a subset (SWEBench-lite).

[xingyaoww]

We haven't tried running the full set and haven't validated it will work (too expensive) -- but the docker image we pre-build should contain all the test beds: https://github.com/OpenDevin/OpenDevin/blob/main/evaluation/swe_bench/run_infer.py#L283-L285 If you change this dataset to SWE-Bench full, it might just work out of box! Feel free to re-open this issue if it does not work for you!