Skip to content

Commit

Permalink
update readme
Browse files Browse the repository at this point in the history
  • Loading branch information
wxingda committed Jun 10, 2023
1 parent 9a738fc commit c7febf0
Show file tree
Hide file tree
Showing 7 changed files with 14 additions and 34 deletions.
5 changes: 2 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,15 +38,15 @@ Vearch is a scalable distributed system for efficient similarity search of deep
* https://vearch.readthedocs.io/zh_CN/latest
* [Common QA](https://github.com/vearch/vearch/wiki/Vearch-QA)

## Components

<details>
<summary><b>Vearch Architecture</b></summary>

![arc](docs/img/VearchArch.jpg)

</details>

## Components

<details>
<summary><b>Master</b></summary>
Responsible for schema mananagement, cluster-level metadata, and resource coordination.
Expand All @@ -63,7 +63,6 @@ Vearch is a scalable distributed system for efficient similarity search of deep

Gamma is the core vector search engine implemented based on [faiss](https://github.com/facebookresearch/faiss). It provides the ability of storing, indexing and retrieving the vectors and scalars.
</details>
</br>

## Benchmarks

Expand Down
8 changes: 4 additions & 4 deletions docs/K8sDeploy.md
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,7 @@
## Make Docker Image
* go to $vearch/cloud dir
* you can run `./run_docker.sh`
* docker push image center,such as docker push xx.xx.local/ai/vearch:3.2.0
* docker push image center,such as docker push xx.xx.local/ai/vearch:latest

## Deploy
* new master.yaml start two pod
Expand Down Expand Up @@ -173,7 +173,7 @@ spec:
containers:
- name: vearchmaster
command: ["./bin/start.sh master"]
image: xx.xx.local/ai/vearch:3.2.0
image: xx.xx.local/ai/vearch:latest
resources:
limits:
cpu: 16
Expand Down Expand Up @@ -225,7 +225,7 @@ spec:
containers:
- name: vearchps
command: ["./bin/start.sh ps"]
image: xx.xx.local/ai/vearch:3.2.0
image: xx.xx.local/ai/vearch:latest
resources:
limits:
cpu: 16
Expand Down Expand Up @@ -277,7 +277,7 @@ spec:
containers:
- name: vearchrouter
command: ["./bin/start.sh router"]
image: xx.xx.local/ai/vearch:3.2.0
image: xx.xx.local/ai/vearch:latest
resources:
limits:
cpu: 16
Expand Down
7 changes: 3 additions & 4 deletions docs/SourceCompileDeployment.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,9 @@
2. vearch deploy image address: https://hub.docker.com/r/vearch/vearch/tags

#### Use Vearch Image Deploy
1. take vearch:3.2.0 as an example
2. docker pull vearch/vearch:3.2.0
3. one docker deploy or distributed deployment
1. ```If deploy a docker start vearch,master,ps,router start together: cat vearch/config/config.toml.example > config.toml nohup docker run -p 8817:8817 -p 9001:9001 -v $PWD/config.toml:/vearch/config.toml vearch/vearch:3.2.0 all &```
1. docker pull vearch/vearch:latest
2. one docker deploy or distributed deployment
1. ```If deploy a docker start vearch,master,ps,router start together: cat vearch/config/config.toml.example > config.toml nohup docker run -p 8817:8817 -p 9001:9001 -v $PWD/config.toml:/vearch/config.toml vearch/vearch:latest all &```

2. ```If distributed deploy ,modify vearch/config/config.toml and start separately```
3. ```Modify vearch/config/config.toml ,refer the step 'Local Model'```
Expand Down
7 changes: 3 additions & 4 deletions docs/SourceCompileDeploymentZH_CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,9 @@
2. vearch部署镜像地址: https://hub.docker.com/r/vearch/vearch/tags

#### 使用Vearch镜像部署
1. 以vearch:3.2.0为例
2. docker pull vearch/vearch:3.2.0
3. 一个docker部署或分布式部署
1. ```If deploy a docker start vearch,master,ps,router start together: cat vearch/config/config.toml.example > config.toml nohup docker run -p 8817:8817 -p 9001:9001 -v $PWD/config.toml:/vearch/config.toml vearch/vearch:3.2.0 all &```
1. docker pull vearch/vearch:latest
2. 一个docker部署或分布式部署
1. ```If deploy a docker start vearch,master,ps,router start together: cat vearch/config/config.toml.example > config.toml nohup docker run -p 8817:8817 -p 9001:9001 -v $PWD/config.toml:/vearch/config.toml vearch/vearch:latest all &```

2. ```If distributed deploy ,modify vearch/config/config.toml and start separately```
3. ```Modify vearch/config/config.toml ,refer the step 'Local Model'```
Expand Down
3 changes: 0 additions & 3 deletions engine/AUTHORS

This file was deleted.

16 changes: 1 addition & 15 deletions engine/README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,5 @@
# Gamma
Gamma is the core vector search engine of Vearch. It is a high-performance, concurrent vector search engine, and supports real time indexing vectors and scalars without lock. Differently from general vector search engine, Gamma can store and index a document which contains scalars and vectors, and provides the ablity of quickly indexing and filtering by numeric scalar fields. The work of design and implementation of real time indexing has been publish in [our Middleware paper](https://arxiv.org/abs/1908.07389).
As for the part of similarity search of vectors in Gamma, it is mainly implemented based on faiss which is an open source library developed by Facebook AI Research. Besides faiss, it can easily support other approximate nearest neighbor search(ANN) algorithms or libraries.

## Requirements
* [Faiss](https://github.com/facebookresearch/faiss)
Gamma is the core vector search engine of Vearch. It is a high-performance, concurrent vector search engine, and supports real time indexing vectors and scalars without lock.

## Installation
1. `mkdir build`
Expand All @@ -21,13 +17,3 @@ As for the part of similarity search of vectors in Gamma, it is mainly implement
6. `wget ftp://ftp.irisa.fr/local/texmex/corpus/siftsmall.tar.gz && tar -zxvf siftsmall.tar.gz`
7. `./test_files profile_10k.txt siftsmall/siftsmall_base.fvecs`
Currently we support gamma both on Linux and OSX of x86_64 machines. We have tested on Centos, Ubuntu and Mac os. And we all just tested with gcc on both Linux and OSX.

## Issue Report


## References
Please cite this paper when referencing Gamma.
Jie Li, Haifeng Liu, Chuanghua Gui, Jianyu chen, Zhenyun Ni, Ning Wang, Yuan Chen. [The Design and Implementation of a Real Time Visual Search System on JD E-commerce Platform](https://arxiv.org/abs/1908.07389). In the 19th International ACM Middleware Conference, December 10–14, 2018, Rennes, France.

## License
Licensed under the Apache License, Version 2.0. For detail see LICENSE and NOTICE.
2 changes: 1 addition & 1 deletion engine/search/gamma_engine.cc
Original file line number Diff line number Diff line change
Expand Up @@ -571,7 +571,7 @@ int GammaEngine::CreateTable(TableInfo &table) {

if (!meta_jp) {
utils::JsonParser dump_meta_;
dump_meta_.PutInt("version", 327); // version=3.2.0
dump_meta_.PutInt("version", 327);

utils::JsonParser table_jp;
table_->GetDumpConfig()->ToJson(table_jp);
Expand Down

0 comments on commit c7febf0

Please sign in to comment.