Skip to content

Commit

Permalink
Merge branch 'master' into master
Browse files Browse the repository at this point in the history
  • Loading branch information
SS committed Jan 14, 2022
2 parents 50c629f + 0f475e2 commit 2055ea7
Show file tree
Hide file tree
Showing 117 changed files with 16,391 additions and 10,767 deletions.
5 changes: 4 additions & 1 deletion .github/workflows/deploy_ci.yml
Expand Up @@ -40,9 +40,12 @@ jobs:
wget https://github.com/milvus-io/milvus/releases/download/v2.0.0-rc7/milvus-standalone-docker-compose.yml -O docker-compose.yml
docker-compose up -d
- name: Insatll libsndfile
- name: Install libsndfile
if: github.event.label.name == 'audio_similarity_search/quick_deploy'
run: sudo apt-get install -y libsndfile1
- name: Install ffmpeg
if: github.event.label.name == 'video_similarity_search/object_detection' || github.event.label.name == 'video_similarity_search/quick_deploy'
run: sudo apt-get install -y ffmpeg
- name: Start MySQL
if: contains(github.event.label.name, '/quick_deploy') || contains(github.event.label.name, '/object_detection')
run: docker run -p 3306:3306 -e MYSQL_ROOT_PASSWORD=123456 -d mysql:5.7
Expand Down
43 changes: 43 additions & 0 deletions .github/workflows/docker_compose_ci.yml
@@ -0,0 +1,43 @@
name: Docker Compose CI Test
on:
pull_request:
branches:
- master
types: [ labeled ]
paths:
- '**docker-compose.yaml'

jobs:
Docker-Compose-Test:
if: github.event.label.name == 'docker-compose'
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@master
- name: Setup Python3.7
uses: actions/setup-python@master
with:
python-version: 3.7
- name: Install pip packeages
run: |
python -m pip install --upgrade pip
pip install pytest
pip install gdown
- name: Get all added and modified files as CSV
id: files
uses: Ana06/get-changed-files@v2.0.0
with:
format: 'csv'
- name: Run test with docker-compose
run: |
mapfile -d ',' -t added_modified_files < <(printf '%s,' '${{ steps.files.outputs.added_modified }}')
for added_modified_file in "${added_modified_files[@]}"; do
if [[ ${added_modified_file} = *docker-compose.yaml ]]; then
mkdir -p `dirname ${added_modified_file}`/data
docker-compose -f ${added_modified_file} up -d
cd `dirname ${added_modified_file}`
pip install -r server/requirements.txt
python -m pytest test_docker_compose.py
fi
done
10 changes: 8 additions & 2 deletions .github/workflows/jupyter_ci.yml
Expand Up @@ -22,8 +22,14 @@ jobs:
python -m pip install --upgrade pip
pip install gdown
pip install pytest nbmake
- name: Insatll libsndfile
- name: Install libsndfile
run: sudo apt-get install -y libsndfile1
- name: Install libglu
run: sudo apt-get install -y libglu1-mesa
- name: Install libopengl
run: sudo apt install -y libopengl0

- name: Start Milvus
run: |
wget https://github.com/milvus-io/milvus/releases/download/v2.0.0-rc7/milvus-standalone-docker-compose.yml -O docker-compose.yml
Expand All @@ -43,4 +49,4 @@ jobs:
fi
done
130 changes: 51 additions & 79 deletions solutions/audio_similarity_search/quick_deploy/README.md
@@ -1,92 +1,71 @@
# Audio search system with Milvus
# Audio search system

This project uses [PANNs](https://github.com/qiuqiangkong/audioset_tagging_cnn)(Large-Scale Pretrained Audio Neural Networks) for Audio Pattern Recognition to perform audio tagging and sound event detection, finally obtaining audio embeddings. Then this project uses [Milvus](https://milvus.io/docs/v2.0.0/overview.md) to search for similar audio clips.

## Local Deployment

### Requirements

- [Milvus 2.0](https://milvus.io/docs/v2.0.0/install_standalone-docker.md)
- [MySQL](https://hub.docker.com/r/mysql/mysql-server)
- [Python3](https://www.python.org/downloads/)

### 1. Start Milvus and MySQL
* [Local Deployment](#local-deployment)
* [Deploy with Docker Compose](#deploy-with-docker-compose)
* [Deploy with source code](#deploy-with-source-code)
* [1. Start API Server](#1-start-api-server)
* [2. Start Client](#2-start-client)
* [How to use front-end](#how-to-use-front-end)
* [Code structure](#code--structure)

The system will use Milvus to store and search the feature vector data, and Mysql is used to store the correspondence between the ids returned by Milvus and the questions data set, then you need to start Milvus and Mysql first.

- **Start Milvus v2.0**

First, you are supposed to refer to the Install [Milvus v2.0](https://milvus.io/docs/v2.0.0/install_standalone-docker.md) for how to run Milvus docker.
## Local Deployment

> Note the version of Milvus.
### Deploy with Docker Compose

- **Start MySQL**
The molecular similarity search system requires [**Milvus**](https://milvus.io/docs/v2.0.0/install_standalone-docker.md), MySQL, Webserver and Webclient services. We can start these containers with one click through [docker-compose.yaml](./audiosearch-docker-compose.yaml), so please make sure you have [installed Docker Engine](https://docs.docker.com/engine/install/) and [Docker Compose](https://docs.docker.com/compose/install/) before running.

```bash
$ docker run -p 3306:3306 -e MYSQL_ROOT_PASSWORD=123456 -d mysql:5.7
$ git clone https://github.com/milvus-io/bootcamp.git
$ cd solutions/audio_similarity_search/quick_deploy
$ docker-compose -f audiosearch-docker-compose.yaml up -d
```

### 2. Start API Server

The next step is to start the system server. It provides HTTP backend services, and there are two ways to start: running with Docker or source code.

#### 2.1 Run server with Docker

- **Set parameters**

Modify the parameters according to your own environment. Below are the main parameters that you will need to setup, for more information please refer to [config.py](./server/src/config.py).

| **Parameter** | **Description** | **example** |
| --------------- | ----------------------------------------------------- | ---------------- |
| **EXTERNAL_DATAPATH** | The directory of the audio data. | /data/audio_path |
| **INTERNAL_DATAPATH** | The mount locaiton of the data within the docker container. | /audio_data |
| **MILVUS_HOST** | The IP address of Milvus. Due to containers not being able to access localhost, please use the IP address of the host, you can get it by `ifconfig`. | 172.16.20.10 |
| **MILVUS_PORT** | The port of Milvus. | 19530 |
| **MYSQL_HOST** | The IP address of MySQL, can be the same as MILVUS_HOST if running on the same system. | 172.16.20.10 |
Then you will see the that all containers are created.

```bash
$ export EXTERNAL_DATAPATH='/data/audio_path'
$ export INTERNAL_DATAPATH='/audio_data'
$ export Milvus_HOST='172.16.20.10'
$ export Milvus_PORT='19530'
$ export Mysql_HOST='172.16.20.10'
Creating network "quick_deploy_app_net" with driver "bridge"
Creating milvus-minio ... done
Creating audio-webclient ... done
Creating milvus-etcd ... done
Creating audio-mysql ... done
Creating milvus-standalone ... done
Creating audio-webserver ... done
```

- **Run Docker**

First, build the docker image from the Dockerfile.
And show all containers with `docker ps`, and you can use `docker logs audio-webserver` to get the logs of **server** container.

```bash
$ cd server
$ docker build -t audio-search-webserver .
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a8428e99f49d milvusbootcamp/audio-search-server:2.0 "/bin/sh -c 'python3…" 28 seconds ago Up 24 seconds 0.0.0.0:8002->8002/tcp, :::8002->8002/tcp audio-webserver
5391a8ebc3a0 milvusdb/milvus:v2.0.0-rc8-20211104-d1f4106 "/tini -- milvus run…" 33 seconds ago Up 28 seconds 0.0.0.0:19530->19530/tcp, :::19530->19530/tcp milvus-standalone
1d1f70f98735 minio/minio:RELEASE.2020-12-03T00-03-10Z "/usr/bin/docker-ent…" 38 seconds ago Up 33 seconds (healthy) 9000/tcp milvus-minio
8f4cfeba5953 quay.io/coreos/etcd:v3.5.0 "etcd -advertise-cli…" 38 seconds ago Up 33 seconds 2379-2380/tcp milvus-etcd
209563de4c12 mysql:5.7 "docker-entrypoint.s…" 38 seconds ago Up 29 seconds 0.0.0.0:3306->3306/tcp, :::3306->3306/tcp, 33060/tcp audio-mysql
f4a6b30f5840 milvusbootcamp/audio-search-client:2.0 "/bin/bash -c '/usr/…" 38 seconds ago Up 31 seconds (health: starting) 0.0.0.0:801->80/tcp, :::801->80/tcp audio-webclient
```

### Deploy with source code

```bash
$ docker run -d \
-v ${EXTERNAL_DATAPATH}:${INTERNAL_DATAPATH} \
-p 8002:8002 \
-e "MILVUS_HOST=${Milvus_HOST}" \
-e "MILVUS_PORT=${Milvus_PORT}" \
-e "MYSQL_HOST=${Mysql_HOST}" \
audio-search-webserver
```
Actually we recommend using Docker Compose to deploy the audio similarity search system. If you want to run from source code, you must manually start [Milvus](https://milvus.io/docs/v2.0.0/install_standalone-docker.md) and [Mysql](https://dev.mysql.com/doc/mysql-installation-excerpt/5.7/en/docker-mysql-getting-started.html). Next show you how to run the API server and Client.

Note: The first time you run the container, it may take a while to become usable as models must be downloaded.
#### 1. Start API Server

#### 2.2 Run source code
Then to start the system server, and it provides HTTP backend services.

- **Install the Python packages**

```bash
$ cd server
$ git clone https://github.com/milvus-io/bootcamp.git
$ cd solutions/audio_similarity_search/quick_deploy/server
$ pip install -r requirements.txt
```

- **Set configuration**

```bash
$ vim server/src/config.py
$ vim src/config.py
```

Modify the parameters according to your own environment. Here listing some parameters that need to be set, for more information please refer to [config.py](./server/src/config.py).
Expand All @@ -105,12 +84,12 @@ Modify the parameters according to your own environment. Here listing some param
Then start the server with Fastapi.

```bash
$ python main.py
$ python src/main.py
```

### 2.3 API Docs
- **API Docs**

After starting the service, Please visit 127.0.0.1:8002/docs in your browser to view all the APIs.
After starting the service, Please visit `127.0.0.1:8002/docs` in your browser to view all the APIs.

![](./pic/allapi.png)

Expand Down Expand Up @@ -138,11 +117,9 @@ After starting the service, Please visit 127.0.0.1:8002/docs in your browser to
>
> Drops Milvus and MySQL tables, removing loaded data.
## 3. Start Client
#### 2. Start Client

Next, start the frontend GUI. Like the system server, there are two ways to start the frontend: running with Docker or source code(recommended).

#### 3.1 Run server with Docker
Next, start the frontend GUI.

- **Set parameters**

Expand All @@ -162,39 +139,34 @@ $ export API_PORT='8002'

First, build the docker image from the Dockerfile.

```bash
$ cd client
$ docker build -t audio-search-client .
```

```bash
$ docker run -d \
-p 80:80 \
-e "API_URL=http://${API_HOST}:${API_PORT}" \
audio-search-client
milvusbootcamp/audio-search-client:2.0
```

#### 3.2 Run source code

Refer to the instructions in the [Client Readme](./client/README.md).
> Refer to the instructions in the [Client Readme](./client/README.md).
### 3.3 How to use front-end
## How to use front-end

Navigate to `127.0.0.1:80` in your browser to access the front-end interface.

- Insert data.
- **Insert data**

Download and unzip the [extract .wav sound files](https://drive.google.com/uc?id=1bKu21JWBfcZBuEuzFEvPoAX6PmRrgnUp) to the specified [data directory](./data). Next, enter the path **/audio_data** in the frontend GUI to initiate the upload.

Download and extract .wav sound files to the "EXTERNAL_DATAPATH" directory specified earlier. Next, input the path to the "INTERNAL_DATAPATH" in the frontend GUI to initiate the upload.
> The **data** directory is the path where the data is locally mounted to the webserver docker, and **/audio_data** is the path inside the docker, so we are supposed to fill in the path in the docker, namely `/audio_data`.
![](./pic/insertgui.png)

- Search for similar audio clips.
- **Search for similar audio clips**

Select the magnifying glass icon on the left side of the interface. Then, press the "Default Target Audio File" button and upload a .wav sound file you'd like to search. Results will be displayed.

![](./pic/searchgui.png)

- **Code structure**
## Code structure

If you are interested in our code or would like to contribute code, feel free to learn more about our code structure.

Expand Down
@@ -0,0 +1,107 @@
version: '3.5'

services:
etcd:
container_name: milvus-etcd
image: quay.io/coreos/etcd:v3.5.0
networks:
app_net:
environment:
- ETCD_AUTO_COMPACTION_MODE=revision
- ETCD_AUTO_COMPACTION_RETENTION=1000
- ETCD_QUOTA_BACKEND_BYTES=4294967296
volumes:
- ${DOCKER_VOLUME_DIRECTORY:-.}/volumes/etcd:/etcd
command: etcd -advertise-client-urls=http://127.0.0.1:2379 -listen-client-urls http://0.0.0.0:2379 --data-dir /etcd

minio:
container_name: milvus-minio
image: minio/minio:RELEASE.2020-12-03T00-03-10Z
networks:
app_net:
environment:
MINIO_ACCESS_KEY: minioadmin
MINIO_SECRET_KEY: minioadmin
volumes:
- ${DOCKER_VOLUME_DIRECTORY:-.}/volumes/minio:/minio_data
command: minio server /minio_data
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
interval: 30s
timeout: 20s
retries: 3

standalone:
container_name: milvus-standalone
image: milvusdb/milvus:v2.0.0-rc8-20211104-d1f4106
networks:
app_net:
ipv4_address: 172.16.23.10
command: ["milvus", "run", "standalone"]
environment:
ETCD_ENDPOINTS: etcd:2379
MINIO_ADDRESS: minio:9000
volumes:
- ${DOCKER_VOLUME_DIRECTORY:-.}/volumes/milvus:/var/lib/milvus
ports:
- "19530:19530"
depends_on:
- "etcd"
- "minio"


mysql:
container_name: audio-mysql
image: mysql:5.7
networks:
app_net:
ipv4_address: 172.16.23.11
environment:
- MYSQL_ROOT_PASSWORD=123456
volumes:
- ${DOCKER_VOLUME_DIRECTORY:-.}/volumes/mysql:/var/lib/mysql
ports:
- "3306:3306"

webserver:
container_name: audio-webserver
image: milvusbootcamp/audio-search-server:2.0
networks:
app_net:
ipv4_address: 172.16.23.12
environment:
MILVUS_HOST: '172.16.23.10'
MYSQL_HOST: '172.16.23.11'
volumes:
- ./data:/audio_data
restart: always
depends_on:
- standalone
- mysql
ports:
- "8002:8002"

webclient:
container_name: audio-webclient
image: milvusbootcamp/audio-search-client:2.0
networks:
app_net:
ipv4_address: 172.16.23.13
environment:
API_URL: 'http://127.0.0.1:8002'
ports:
- "801:80"
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost/"]
interval: 30s
timeout: 20s
retries: 3

networks:
app_net:
driver: bridge
ipam:
driver: default
config:
- subnet: 172.16.23.0/24
gateway: 172.16.23.1

0 comments on commit 2055ea7

Please sign in to comment.