-
Notifications
You must be signed in to change notification settings - Fork 213
Description
When running python agentless/test/run_regression_tests.py --run_id generate_regression_tests --output_file results/swe-bench-lite/passing_tests.jsonl \ There are around 30 dockers fails and return 1, the list is at the bottom. These dockers all failed due to being unable to download ['http://www.qhull.org/download/qhull-2020-src-8.0.2.tgz'].
From my analysis I think the issue is caused by the swe-bench version. When I run the current version of swe-bench v4.1.0 I did not meet these failure. However, in order to run the test/run_test.py, I need to modify the swe-bench version to v2.1.5. The dockers in this version may have expired and fail.
I have tried to modify the test/run_test.py to adapt to the new version of swe-bench but failed, I wonder if it is possible for you to modify the api calls of swe-bench and use the latest version to avoid these failures?
The failed docker list:
Error building image sweb.eval.x86_64.matplotlib__matplotlib-18869:latest: The command '/bin/sh -c /bin/bash /root/setup_repo.sh' returned a non-zero code: 1 Check (logs/build_images/instances/sweb.eval.x86_64.matplotlib__matplotlib-18869__latest/build_image.log) for more information. Error building image sweb.eval.x86_64.matplotlib__matplotlib-22711:latest: The command '/bin/sh -c /bin/bash /root/setup_repo.sh' returned a non-zero code: 1 Check (logs/build_images/instances/sweb.eval.x86_64.matplotlib__matplotlib-22711__latest/build_image.log) for more information. Error building image sweb.eval.x86_64.matplotlib__matplotlib-22835:latest: The command '/bin/sh -c /bin/bash /root/setup_repo.sh' returned a non-zero code: 1 Check (logs/build_images/instances/sweb.eval.x86_64.matplotlib__matplotlib-22835__latest/build_image.log) for more information. Error building image sweb.eval.x86_64.matplotlib__matplotlib-23299:latest: The command '/bin/sh -c /bin/bash /root/setup_repo.sh' returned a non-zero code: 1 Check (logs/build_images/instances/sweb.eval.x86_64.matplotlib__matplotlib-23299__latest/build_image.log) for more information. Error building image sweb.eval.x86_64.matplotlib__matplotlib-23314:latest: The command '/bin/sh -c /bin/bash /root/setup_repo.sh' returned a non-zero code: 1 Check (logs/build_images/instances/sweb.eval.x86_64.matplotlib__matplotlib-23314__latest/build_image.log) for more information. Error building image sweb.eval.x86_64.matplotlib__matplotlib-23476:latest: The command '/bin/sh -c /bin/bash /root/setup_repo.sh' returned a non-zero code: 1 Check (logs/build_images/instances/sweb.eval.x86_64.matplotlib__matplotlib-23476__latest/build_image.log) for more information. Error building image sweb.eval.x86_64.matplotlib__matplotlib-23562:latest: The command '/bin/sh -c /bin/bash /root/setup_repo.sh' returned a non-zero code: 1 Check (logs/build_images/instances/sweb.eval.x86_64.matplotlib__matplotlib-23562__latest/build_image.log) for more information. Error building image sweb.eval.x86_64.matplotlib__matplotlib-23563:latest: The command '/bin/sh -c /bin/bash /root/setup_repo.sh' returned a non-zero code: 1 Check (logs/build_images/instances/sweb.eval.x86_64.matplotlib__matplotlib-23563__latest/build_image.log) for more information. Error building image sweb.eval.x86_64.matplotlib__matplotlib-23913:latest: The command '/bin/sh -c /bin/bash /root/setup_repo.sh' returned a non-zero code: 1 Check (logs/build_images/instances/sweb.eval.x86_64.matplotlib__matplotlib-23913__latest/build_image.log) for more information. Error building image sweb.eval.x86_64.matplotlib__matplotlib-23964:latest: The command '/bin/sh -c /bin/bash /root/setup_repo.sh' returned a non-zero code: 1 Check (logs/build_images/instances/sweb.eval.x86_64.matplotlib__matplotlib-23964__latest/build_image.log) for more information. Error building image sweb.eval.x86_64.matplotlib__matplotlib-23987:latest: The command '/bin/sh -c /bin/bash /root/setup_repo.sh' returned a non-zero code: 1 Check (logs/build_images/instances/sweb.eval.x86_64.matplotlib__matplotlib-23987__latest/build_image.log) for more information. Error building image sweb.eval.x86_64.matplotlib__matplotlib-24149:latest: The command '/bin/sh -c /bin/bash /root/setup_repo.sh' returned a non-zero code: 1 Check (logs/build_images/instances/sweb.eval.x86_64.matplotlib__matplotlib-24149__latest/build_image.log) for more information. Error building image sweb.eval.x86_64.matplotlib__matplotlib-24265:latest: The command '/bin/sh -c /bin/bash /root/setup_repo.sh' returned a non-zero code: 1 Check (logs/build_images/instances/sweb.eval.x86_64.matplotlib__matplotlib-24265__latest/build_image.log) for more information. Error building image sweb.eval.x86_64.matplotlib__matplotlib-24334:latest: The command '/bin/sh -c /bin/bash /root/setup_repo.sh' returned a non-zero code: 1 Check (logs/build_images/instances/sweb.eval.x86_64.matplotlib__matplotlib-24334__latest/build_image.log) for more information. Error building image sweb.eval.x86_64.matplotlib__matplotlib-24970:latest: The command '/bin/sh -c /bin/bash /root/setup_repo.sh' returned a non-zero code: 1 Check (logs/build_images/instances/sweb.eval.x86_64.matplotlib__matplotlib-24970__latest/build_image.log) for more information. Error building image sweb.eval.x86_64.matplotlib__matplotlib-25079:latest: The command '/bin/sh -c /bin/bash /root/setup_repo.sh' returned a non-zero code: 1 Check (logs/build_images/instances/sweb.eval.x86_64.matplotlib__matplotlib-25079__latest/build_image.log) for more information. Error building image sweb.eval.x86_64.matplotlib__matplotlib-25311:latest: The command '/bin/sh -c /bin/bash /root/setup_repo.sh' returned a non-zero code: 1 Check (logs/build_images/instances/sweb.eval.x86_64.matplotlib__matplotlib-25311__latest/build_image.log) for more information. Error building image sweb.eval.x86_64.matplotlib__matplotlib-25332:latest: The command '/bin/sh -c /bin/bash /root/setup_repo.sh' returned a non-zero code: 1 Check (logs/build_images/instances/sweb.eval.x86_64.matplotlib__matplotlib-25332__latest/build_image.log) for more information. Error building image sweb.eval.x86_64.matplotlib__matplotlib-25433:latest: The command '/bin/sh -c /bin/bash /root/setup_repo.sh' returned a non-zero code: 1 Check (logs/build_images/instances/sweb.eval.x86_64.matplotlib__matplotlib-25433__latest/build_image.log) for more information. Error building image sweb.eval.x86_64.matplotlib__matplotlib-25442:latest: The command '/bin/sh -c /bin/bash /root/setup_repo.sh' returned a non-zero code: 1 Check (logs/build_images/instances/sweb.eval.x86_64.matplotlib__matplotlib-25442__latest/build_image.log) for more information. Error building image sweb.eval.x86_64.matplotlib__matplotlib-25498:latest: The command '/bin/sh -c /bin/bash /root/setup_repo.sh' returned a non-zero code: 1 Check (logs/build_images/instances/sweb.eval.x86_64.matplotlib__matplotlib-25498__latest/build_image.log) for more information. Error building image sweb.eval.x86_64.matplotlib__matplotlib-26011:latest: The command '/bin/sh -c /bin/bash /root/setup_repo.sh' returned a non-zero code: 1 Check (logs/build_images/instances/sweb.eval.x86_64.matplotlib__matplotlib-26011__latest/build_image.log) for more information. Error building image sweb.eval.x86_64.matplotlib__matplotlib-26020:latest: The command '/bin/sh -c /bin/bash /root/setup_repo.sh' returned a non-zero code: 1 Check (logs/build_images/instances/sweb.eval.x86_64.matplotlib__matplotlib-26020__latest/build_image.log) for more information.