File tree Expand file tree Collapse file tree 6 files changed +59
-12
lines changed Expand file tree Collapse file tree 6 files changed +59
-12
lines changed Original file line number Diff line number Diff line change 33
44#
55
6- Commit-0 is a real-world AI coding challenge.
7- Can your agent generate a working library from commit 0?
6+ ## Overview
7+
8+ Commit-0 is a from scratch AI coding challenge.
9+ Can you create a library from commit 0?
810
911The benchmark consists of 57 core Python libraries.
10- Libraries are selected based on:
12+ The challenge is to rebuild these libraries and
13+ pass their unit tests. All libraries have:
1114
12- * Significant unit- test coverage
15+ * Significant test coverage
1316* Detailed specification and documentation
1417* Lint and type checking
1518
16- The [ commit0 tool] ( setup ) allows you to:
19+ Commit-0 is an interactive environment that makes it easy
20+ to design and test new agents. You can:
1721
18- * Efficiently run interactive tests in isolated environemnts
22+ * Efficiently run tests in isolated environemnts
1923* Distribute testing and development across cloud systems
2024* Track and log all changes made throughout.
2125
@@ -25,6 +29,14 @@ To install run:
2529pip install commit0
2630```
2731
32+ ## Architecture
33+
34+ ![ ] ( arch.png )
35+
36+
37+ ![ ] ( commit0.gif )
38+
39+ ## Libraries
2840
2941| | Name | Repo | Commit0 | Tests | |
3042| --| --------| -------| ----| ----| ------|
Original file line number Diff line number Diff line change @@ -44,3 +44,40 @@ you can commit to the branch and call with the --branch command.
4444``` bash
4545commit0 test simpy tests/test_event.py::test_succeed --branch my_branch
4646```
47+
48+ ## Local Mode
49+
50+ To run in local mode you first be sure that you have [ docker tools] ( https://docs.docker.com/desktop/install/mac-install/ )
51+ installed. On Debian systems:
52+
53+ ``` bash
54+ apt install docker
55+ ```
56+
57+ To get started, run the ` setup ` command with the dataset
58+ split that you are interested in working with.
59+ We'll start with the ` lite ` split.
60+
61+
62+ ``` bash
63+ commit0 setup lite
64+ ```
65+
66+ This will install a clone the code for subset of libraries to your ` repos/ ` directory.
67+
68+ Next run the ` build ` command which will configure Docker containers for
69+ each of the libraries with isolated virtual environments. The command uses the
70+ [ uv] ( https://github.com/astral-sh/uv ) library for efficient builds.
71+
72+ ``` bash
73+ commit0 build
74+ ```
75+
76+ The main operation you can do with these enviroments is to run tests.
77+ Here we run [ a test] ( https://github.com/commit-0/simpy/blob/master/tests/test_event.py#L11 ) in the ` simpy ` library.
78+
79+ ``` bash
80+ commit0 test simpy tests/test_event.py::test_succeed
81+ ```
82+
83+ See [ distributed setup] ( /setupdist ) for more commands.
Original file line number Diff line number Diff line change @@ -33,4 +33,4 @@ Here we run [a test](https://github.com/commit-0/simpy/blob/master/tests/test_ev
3333commit0 test simpy tests/test_event.py::test_succeed
3434```
3535
36- See [ distributed setup] ( setupdist ) for more commands.
36+ See [ distributed setup] ( / setupdist) for more commands.
Original file line number Diff line number Diff line change @@ -3,10 +3,9 @@ site_name: ""
33docs_dir : docs
44nav :
55 - Home : index.md
6- - Distributed : setupdist.md
7- - Local : setuplocal.md
8- - Extending : repos.md
9- - About : about.md
6+ - Commit0 : setupdist.md
7+ - Agent : agent.md
8+ - API : api.md
109theme :
1110 name : material
1211 logo : " logo2.webp"
@@ -17,5 +16,4 @@ extra_css:
1716markdown_extensions :
1817 - pymdownx.inlinehilite
1918 - pymdownx.snippets
20- - pymdownx.superfences
2119 - pymdownx.highlight
You can’t perform that action at this time.
0 commit comments