Skip to content

Commit

Permalink
moved algos.yaml to algos-2021.yaml (FALCONN-LIB#104)
Browse files Browse the repository at this point in the history
  • Loading branch information
harsha-simhadri committed Apr 25, 2023
1 parent 22c4afa commit 68fa1de
Show file tree
Hide file tree
Showing 5 changed files with 8 additions and 8 deletions.
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,10 @@

## Install

The only prerequisite is Python (tested with 3.6) and Docker. Works with newer versions of Python as well but probably requires an updated `requirements.txt` on the host. (Suggestion: copy `requirements.txt` to `requirements${PYTHON_VERSION}.txt` and remove all fixed versions. `requirements.txt` has to be kept for the docker containers.)
The only prerequisite is Python and Docker.

1. Clone the repo.
2. Run `pip install -r requirements.txt` (Use `requirements_py38.txt` if you have Python 3.8.)
2. Run `pip install -r requirements_py3.10.txt` (Use `requirements_py38.txt` if you have Python 3.8.)
3. Install docker by following instructions [here](https://docs.docker.com/engine/install/ubuntu/).
You might also want to follow the post-install steps for running docker in non-root user mode.
3. Run `python install.py` to build all the libraries inside Docker containers.
Expand Down Expand Up @@ -44,7 +44,7 @@ Run `python run.py --dataset $DS --algorithm $ALGO` where `DS` is the dataset yo
and `ALGO` is the name of the algorithm. (Use `python run.py --list-algorithms`) to get an overview.
`python run.py -h` provides you with further options.

The parameters used by the implementation to build and query the index can be found in `algos.yaml`.
The parameters used by the implementation to build and query the index can be found in `algos-2021.yaml`.

## Running the track 1 baseline
After running the installation, we can evaluate the baseline as follows.
Expand Down
File renamed without changes.
2 changes: 1 addition & 1 deletion benchmark/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ def main():
'--definitions',
metavar='FILE',
help='load algorithm definitions from FILE',
default='algos.yaml')
default='algos-2021.yaml')
parser.add_argument(
'--algorithm',
metavar='NAME',
Expand Down
2 changes: 1 addition & 1 deletion plot.py
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ def mean_y(algo):
'--definitions',
metavar='FILE',
help='load algorithm definitions from FILE',
default='algos.yaml')
default='algos-2021.yaml')
parser.add_argument(
'--limit',
default=-1)
Expand Down
6 changes: 3 additions & 3 deletions t1_t2/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -143,7 +143,7 @@ When you are ready to test on the competition datasets, use the create_dataset.p
```
python create_dataset.py --dataset [sift-1B|bigann-1B|text2image-1B|msturing-1B|msspacev-1B|ssnpp-1B]
```
To benchmark your algorithm, first create an algorithm configuration yaml in your teams directory called *algos.yaml.* This file contains the index build parameters and query parameters that will get passed to your algorithm at run-time. Please look at [algos.yaml](../algos.yaml).
To benchmark your algorithm, first create an algorithm configuration yaml in your teams directory called *algos-2021.yaml.* This file contains the index build parameters and query parameters that will get passed to your algorithm at run-time. Please look at [algos-2021.yaml](../algos-2021.yaml).

If your machine is capable of both building and searching an index, you can benchmark your algorithm using the run.py script.
```
Expand Down Expand Up @@ -242,11 +242,11 @@ diskann-t2 bigann-1B 0.94913
A submission is composed of a pull request to this repo with the following.
* Your algorithm's python class, inheriting from `BaseANN`, placed in the [benchmark/algorithms/](../benchmark/algorithms) directory.
* A Dockerfile in `install/` describing how to retrieve, compile and set up requirements for your algorithm.
* For each dataset you are participating in, add to [algos.yaml](../algos.yaml)
* For each dataset you are participating in, add to [algos-2021.yaml](../algos-2021.yaml)
* 1 index build configuration
* 10 search configuration
* Add an entry to [CI test list](../.github/workflows/benchmarks.yml) for the random-xs dataset, and for the random-range-xs dataset if your algorithm supports range search. We can start working with larger datasets once these tests pass.
* An URL to download any prebuilt indices placed in `algos.yaml`. **This is optional, but strongly encourages.** This would help us evaluate faster, although we would build your index to verify the time limit. Please see `faiss_t1.py` and `diskann-t2.py` for examples. If you are unable to host the index on your own Azure blob storage, please let us know and we can arrange to have it copied to organizer's account.
* An URL to download any prebuilt indices placed in `algos-2021.yaml`. **This is optional, but strongly encourages.** This would help us evaluate faster, although we would build your index to verify the time limit. Please see `faiss_t1.py` and `diskann-t2.py` for examples. If you are unable to host the index on your own Azure blob storage, please let us know and we can arrange to have it copied to organizer's account.

We will run early PRs on organizer's machines to the extent possible and provide any feedback necessary.

Expand Down

0 comments on commit 68fa1de

Please sign in to comment.