Skip to content

Commit e268b72

Browse files
committed
.
1 parent b9d089a commit e268b72

File tree

6 files changed

+59
-12
lines changed

6 files changed

+59
-12
lines changed

docs/arch.png

281 KB
Loading

docs/commit0.gif

20.8 MB
Loading

docs/index.md

Lines changed: 18 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -3,19 +3,23 @@
33

44
#
55

6-
Commit-0 is a real-world AI coding challenge.
7-
Can your agent generate a working library from commit 0?
6+
## Overview
7+
8+
Commit-0 is a from scratch AI coding challenge.
9+
Can you create a library from commit 0?
810

911
The benchmark consists of 57 core Python libraries.
10-
Libraries are selected based on:
12+
The challenge is to rebuild these libraries and
13+
pass their unit tests. All libraries have:
1114

12-
* Significant unit-test coverage
15+
* Significant test coverage
1316
* Detailed specification and documentation
1417
* Lint and type checking
1518

16-
The [commit0 tool](setup) allows you to:
19+
Commit-0 is an interactive environment that makes it easy
20+
to design and test new agents. You can:
1721

18-
* Efficiently run interactive tests in isolated environemnts
22+
* Efficiently run tests in isolated environemnts
1923
* Distribute testing and development across cloud systems
2024
* Track and log all changes made throughout.
2125

@@ -25,6 +29,14 @@ To install run:
2529
pip install commit0
2630
```
2731

32+
## Architecture
33+
34+
![](arch.png)
35+
36+
37+
![](commit0.gif)
38+
39+
## Libraries
2840

2941
| | Name | Repo | Commit0 | Tests | |
3042
|--|--------|-------|----|----|------|

docs/setupdist.md

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,3 +44,40 @@ you can commit to the branch and call with the --branch command.
4444
```bash
4545
commit0 test simpy tests/test_event.py::test_succeed --branch my_branch
4646
```
47+
48+
## Local Mode
49+
50+
To run in local mode you first be sure that you have [docker tools](https://docs.docker.com/desktop/install/mac-install/)
51+
installed. On Debian systems:
52+
53+
```bash
54+
apt install docker
55+
```
56+
57+
To get started, run the `setup` command with the dataset
58+
split that you are interested in working with.
59+
We'll start with the `lite` split.
60+
61+
62+
```bash
63+
commit0 setup lite
64+
```
65+
66+
This will install a clone the code for subset of libraries to your `repos/` directory.
67+
68+
Next run the `build` command which will configure Docker containers for
69+
each of the libraries with isolated virtual environments. The command uses the
70+
[uv](https://github.com/astral-sh/uv) library for efficient builds.
71+
72+
```bash
73+
commit0 build
74+
```
75+
76+
The main operation you can do with these enviroments is to run tests.
77+
Here we run [a test](https://github.com/commit-0/simpy/blob/master/tests/test_event.py#L11) in the `simpy` library.
78+
79+
```bash
80+
commit0 test simpy tests/test_event.py::test_succeed
81+
```
82+
83+
See [distributed setup](/setupdist) for more commands.

docs/setuplocal.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -33,4 +33,4 @@ Here we run [a test](https://github.com/commit-0/simpy/blob/master/tests/test_ev
3333
commit0 test simpy tests/test_event.py::test_succeed
3434
```
3535

36-
See [distributed setup](setupdist) for more commands.
36+
See [distributed setup](/setupdist) for more commands.

mkdocs.yml

Lines changed: 3 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -3,10 +3,9 @@ site_name: ""
33
docs_dir: docs
44
nav:
55
- Home: index.md
6-
- Distributed: setupdist.md
7-
- Local: setuplocal.md
8-
- Extending: repos.md
9-
- About: about.md
6+
- Commit0: setupdist.md
7+
- Agent: agent.md
8+
- API: api.md
109
theme:
1110
name: material
1211
logo: "logo2.webp"
@@ -17,5 +16,4 @@ extra_css:
1716
markdown_extensions:
1817
- pymdownx.inlinehilite
1918
- pymdownx.snippets
20-
- pymdownx.superfences
2119
- pymdownx.highlight

0 commit comments

Comments
 (0)