diff --git a/docs/index.md b/docs/index.md index 5b9f22d..e49b2bd 100644 --- a/docs/index.md +++ b/docs/index.md @@ -19,6 +19,13 @@ The [commit0 tool](setup) allows you to: * Distribute testing and development across cloud systems * Track and log all changes made throughout. +To install run: + +```bash +pip install commit0 +``` + + | | Name | Repo | Commit0 | Tests | | |--|--------|-------|----|----|------| | | [minitorch](https://minitorch.github.io/) | [[orig](http://github.com/minitorch/minitorch)] | [[commit0](http://github.com/commit-0/minitorch)] | 230 | | diff --git a/docs/setup.md b/docs/setup.md deleted file mode 100644 index e7ffd93..0000000 --- a/docs/setup.md +++ /dev/null @@ -1,139 +0,0 @@ -# Quickstart - -## Install - -First be sure that you have docker tools installed. - -```bash -apt install docker -``` - -To install the benchmark run, - -```bash -pip install commit0 -``` - -## Commands - -The system is a command-line tool that allows you to run unit-tests on a -variety of libraries in isolated environments. To get started with the full -setup run the `setup` command which will install a clone the code of a subset -of libraries to your `repos/` directory. - -```bash -commit0 setup lite -``` - -Next run the `build` command which will configure Docker containers for -each of the libraries with isolated virtual environments. The command uses the -[uv](https://github.com/astral-sh/uv) library for efficient builds. - -```bash -commit0 build lit -``` - -The main operation you can do with these enviroments is to run tests. -Here we run [a test](https://github.com/commit-0/simpy/blob/master/tests/test_event.py#L11) in the `simpy` library. - -```bash -commit0 test simpy tests/test_event.py::test_succeed -``` - -This test should run and pass, but others will fail. - -```bash -commit0 test minitorch tests/test_operators.py::test_relu -``` - -Let's now manually go in and change that repo. -This is all just standard shell commands. - -```bash -cd repos/minitorch/ -git checkout -b mychange -``` - -And apply and commit this patch. - -``` ---- a/minitorch/operators.py -+++ b/minitorch/operators.py -@@ -81,7 +81,7 @@ def relu(x: float) -> float: - (See https://en.wikipedia.org/wiki/Rectifier_(neural_networks) .) - """ - # TODO: Implement for Task 0.1. -- raise NotImplementedError('Need to implement for Task 0.1') -+ return 1. if x > 0. else 0. -``` - -Once this is done we can run `test` with -a branch and the environment will sync and run. - -```bash -commit0 test minitorch branch=mychange tests/test_operators.py::test_relu -``` - -## Running an Agent - -Next we will see how this can be run with an AI agent system. -We will use [Aider](https://aider.chat/) which is a nice -command-line oriented agent system. - -To setup Aider first set your api key. -We recommend using Claude Sonnet. - -```bash -# Work with Claude 3.5 Sonnet on your repo -export ANTHROPIC_API_KEY=your-key-goes-here -``` - -Once this is setup you can run Aider with the following command. -This will edit the files locally in your branch, but -run the tests inside the environment. - -```bash -aider --model sonnet --file repos/minitorch/operators.py --message "fill in" \ - --auto-test --test \ - --test-cmd 'commit0 test minitorch branch=mychange tests/test_operators.py::test_relu' \ - --yes -``` - -This will run an LLM agent that will try to fill in the -functions in one file of the minitorch library. - -For a full example baseline system that tries to solve -all the tests in the library see the [baseline](baseline) documentation. - - -## Distributed Tests - -One of the main advantages of `commit0` is that it can run -a range of unit tests in distributed environments. - -By default, the library is configured to work with [modal](https://modal.com/). - -```bash -pip install modal -modal token new -``` - -To enable distributed run, first -create a file called `distributed.yaml` - -```yaml -backend: modal -base_dir: repos.dist/ -``` - -You can pass this configuration file as an argumnet to clone. - -```bash -commit0 clone lite --cfg=distributed.yaml -``` - -Next to run tests you can run the standard test command. - -```bash -commit0 test simpy master tests/test_event.py::test_succeed --cfg=distributed.yaml -``` diff --git a/docs/setupdist.md b/docs/setupdist.md new file mode 100644 index 0000000..e04679b --- /dev/null +++ b/docs/setupdist.md @@ -0,0 +1,46 @@ +## Distributed Mode + +Commit0 is a command-line tool that allows you to run unit-tests on a +variety of libraries in isolated environments. + +The defaul tool uses [modal](https://modal.com/) as a distributed +test runner. + +```bash +pip install modal +modal token new +``` + +To get started, run the `setup` command with the dataset +split that youare interested in working with. +We'll start with the `lite` split. + +```bash +commit0 setup lite +``` + +This will clone a set of skeleton libraries in your `repos/` directory. +Commiting changes to branches in this directory is how you send changes +to the test runner. + +Next to run tests you can run the standard test command. +This command will run a reference unit test for the `simpy` repo. + +```bash +commit0 test simpy tests/test_event.py::test_succeed --reference +``` + +To run a test in your codebase you can run with no args. +This one will fail. + +```bash +commit0 test simpy tests/test_event.py::test_succeed +``` + +To run a test in your codebase with a specific branch +you can commit to the branch and call with the --branch command. + + +```bash +commit0 test simpy tests/test_event.py::test_succeed --branch my_branch +``` diff --git a/docs/setuplocal.md b/docs/setuplocal.md new file mode 100644 index 0000000..c238b39 --- /dev/null +++ b/docs/setuplocal.md @@ -0,0 +1,36 @@ +## Local Mode + +To run in local mode you first be sure that you have [docker tools](https://docs.docker.com/desktop/install/mac-install/) +installed. On Debian systems: + +```bash +apt install docker +``` + +To get started, run the `setup` command with the dataset +split that you are interested in working with. +We'll start with the `lite` split. + + +```bash +commit0 setup lite +``` + +This will install a clone the code for subset of libraries to your `repos/` directory. + +Next run the `build` command which will configure Docker containers for +each of the libraries with isolated virtual environments. The command uses the +[uv](https://github.com/astral-sh/uv) library for efficient builds. + +```bash +commit0 build +``` + +The main operation you can do with these enviroments is to run tests. +Here we run [a test](https://github.com/commit-0/simpy/blob/master/tests/test_event.py#L11) in the `simpy` library. + +```bash +commit0 test simpy tests/test_event.py::test_succeed +``` + +See [distributed setup](setupdist) for more commands. diff --git a/mkdocs.yml b/mkdocs.yml index 67cf1fb..f6e64f8 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -3,7 +3,8 @@ site_name: "" docs_dir: docs nav: - Home: index.md - - Setup: setup.md + - Distributed: setupdist.md + - Local: setuplocal.md - Extending: repos.md - About: about.md theme: