Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix errors in .mds #4644

Merged
merged 14 commits into from
Jan 22, 2021
2 changes: 1 addition & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Changelog

Please mark all change in change log and use the issue from GitHub
# Milvus 0.10.5 (TBD)
# Milvus 0.10.5 (2021-01-07)
## Bug
- \#4296,#4554 Fix mishards add name-mismatched pod to read-only group
- \#4307 Specify partition to load for load_collection()
Expand Down
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ The Milvus team members will review your pull requests, and once it is accepted,
Please create a new branch from an up-to-date master on your fork.

1. Fork the repository on GitHub.
2. Clone your fork to your local machine with `git clone git@github.com:<yourname>/milvus-io/milvus.git`.
2. Clone your fork to your local machine with `git clone git@github.com:<yourname>/milvus.git`.
3. Create a branch with `git checkout -b my-topic-branch`.
4. Make your changes, commit, then push to to GitHub with `git push --set-upstream origin my-topic-branch`. You must record your changes in [CHANGELOG.md](CHANGELOG.md) with issue numbers and descriptions.
5. Visit GitHub and make your pull request.
Expand Down
115 changes: 75 additions & 40 deletions INSTALL.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,31 +4,31 @@

- [Build from source](#build-from-source)

- [Requirements](#requirements)
- [Requirements](#requirements)

- [Compilation](#compilation)
- [Compilation](#compilation)

- [Launch Milvus server](#launch-milvus-server)
- [Launch Milvus server](#launch-milvus-server)

- [Compile Milvus on Docker](#compile-milvus-on-docker)
- [Compile Milvus on Docker](#compile-milvus-on-docker)

- [Step 1 Pull Milvus Docker images](#step-1-pull-milvus-docker-images)
- [Step 1 Pull Milvus Docker images](#step-1-pull-milvus-docker-images)

- [Step 2 Start the Docker container](#step-2-start-the-docker-container)
- [Step 2 Start the Docker container](#step-2-start-the-docker-container)

- [Step 3 Download Milvus source code](#step-3-download-milvus-source-code)
- [Step 3 Download Milvus source code](#step-3-download-milvus-source-code)

- [Step 4 Compile Milvus in the container](#step-4-compile-milvus-in-the-container)
- [Step 4 Compile Milvus in the container](#step-4-compile-milvus-in-the-container)

- [Troubleshooting](#troubleshooting)

- [Error message: `protocol https not supported or disabled in libcurl`](#error-message-protocol-https-not-supported-or-disabled-in-libcurl)
- [Error message: `protocol https not supported or disabled in libcurl`](#error-message-protocol-https-not-supported-or-disabled-in-libcurl)

- [Error message: `internal compiler error`](#error-message-internal-compiler-error)
- [Error message: `internal compiler error`](#error-message-internal-compiler-error)

- [Error message: `error while loading shared libraries: libmysqlpp.so.3`](#error-message-error-while-loading-shared-libraries-libmysqlppso3)
- [Error message: `error while loading shared libraries: libmysqlpp.so.3`](#error-message-error-while-loading-shared-libraries-libmysqlppso3)

- [CMake version is not supported](#cmake-version-is-not-supported)
- [CMake version is not supported](#cmake-version-is-not-supported)

<!-- /TOC -->

Expand All @@ -42,64 +42,77 @@

- CentOS 7

> Note: If your Linux operating system does not meet the requirements, we recommend that you pull a Docker image of [Ubuntu 18.04](https://docs.docker.com/install/linux/docker-ce/ubuntu/) or [CentOS 7](https://docs.docker.com/install/linux/docker-ce/centos/) as your compilation environment.
> Note: If your Linux operating system does not meet the requirements, we recommend that you pull a Docker image of [Ubuntu 18.04](https://docs.docker.com/install/linux/docker-ce/ubuntu/) or [CentOS 7](https://docs.docker.com/install/linux/docker-ce/centos/) as your compilation environment.

- GCC 7.0 or higher to support C++ 17

- CMake 3.12 or higher
- CMake 3.14 or higher

- Git

For GPU-enabled version, you will also need:

- CUDA 10.0 or higher
- CUDA 10.x (10.0, 10.1, 10.2)

- NVIDIA driver 418 or higher

### Compilation

#### Step 1 Install dependencies
#### Step 1 Download Milvus source code and specify version

Download Milvus source code, change directory and specify version (for example, 0.10.3):

```shell
$ git clone https://github.com/milvus-io/milvus
$ cd ./milvus/core
$ git checkout 0.10.3
```

#### Step 2 Install dependencies

##### Install in Ubuntu

```shell
$ cd [Milvus root path]/core
$ ./ubuntu_build_deps.sh
```

##### Install in CentOS

```shell
$ cd [Milvus root path]/core
$ ./centos7_build_deps.sh
```

#### Step 2 Build
#### Step 3 Build Milvus source code

If you want to use CPU-only:

run `build.sh`:

```shell
$ cd [Milvus root path]/core
$ ./build.sh -t Debug
$ ./build.sh -t Release
```

or
If you want to use GPU-enabled:

1. Add cuda library path to `LD_LIBRARY_PATH`:

```shell
$ ./build.sh -t Release
$ export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH
```

By default, it will build CPU-only version. To build GPU version, add `-g` option.
2. Add cuda binary path to `PATH`:

```shell
$ ./build.sh -g
$ export PATH=/usr/local/cuda/bin:$PATH
```

If you want to know the complete build options, run the following command.
3. Add a `-g` parameter to run `build.sh`:

```shell
$./build.sh -h
$ ./build.sh -g -t Release
```

When the build is completed, everything that you need in order to run Milvus will be installed under `[Milvus root path]/core/milvus`.
When the build completes, everything that you need to run Milvus is under `[Milvus root path]/core/milvus`.

### Launch Milvus server

Expand Down Expand Up @@ -132,37 +145,54 @@ With the following Docker images, you should be able to compile Milvus on any Li

### Step 1 Pull Milvus Docker images

#### ubuntu18.04

Pull CPU-only image:

```shell
$ docker pull milvusdb/milvus-cpu-build-env:latest
$ docker pull milvusdb/milvus-cpu-build-env:v0.7.0-ubuntu18.04
```

Pull GPU-enabled image:

```shell
$ docker pull milvusdb/milvus-gpu-build-env:latest
$ docker pull milvusdb/milvus-gpu-build-env:v0.7.0-ubuntu18.04
```

#### CentOs7

Pull CPU-only image:

```shell
$ docker pull milvusdb/milvus-cpu-build-env:v0.7.0-centos7
```

Pull GPU-enabled image:

```shell
$ docker pull milvusdb/milvus-gpu-build-env:v0.7.0-centos7
```

### Step 2 Start the Docker container

Start a CPU-only container:

```shell
$ docker run -it -p 19530:19530 -d milvusdb/milvus-cpu-build-env:latest
$ docker run -it -p 19530:19530 -d <milvus_cpu_docker_image>
```

Start a GPU container:

- For nvidia docker 2:

```shell
$ docker run --runtime=nvidia -it -p 19530:19530 -d milvusdb/milvus-gpu-build-env:latest
$ docker run --runtime=nvidia -it -p 19530:19530 -d <milvus_gpu_docker_image>
```

- For nvidia container toolkit:

```shell
docker run --gpus all -it -p 19530:19530 -d milvusdb/milvus-gpu-build-env:latest
docker run --gpus all -it -p 19530:19530 -d <milvus_gpu_docker_image>
```

To enter the container:
Expand All @@ -173,10 +203,9 @@ $ docker exec -it [container_id] bash

### Step 3 Download Milvus source code

Download latest Milvus source code:
Download Milvus source code:

```shell
$ cd /home
$ git clone https://github.com/milvus-io/milvus
```

Expand All @@ -186,6 +215,12 @@ To enter its core directory:
$ cd ./milvus/core
```

Specify version (for example, 0.10.3):

```shell
$ git checkout 0.10.3
```

### Step 4 Compile Milvus in the container

If you are using a CPU-only image:
Expand Down Expand Up @@ -267,16 +302,16 @@ Follow the steps below to solve this problem:
Follow the steps below to install a supported version of CMake:

1. Remove the unsupported version of CMake.
2. Get CMake 3.12 or higher. Here we get CMake 3.12.
2. Get CMake 3.14 or higher. Here we get CMake 3.14.

```shell
$ wget https://cmake.org/files/v3.12/cmake-3.12.2-Linux-x86_64.tar.gz
$ wget https://cmake.org/files/v3.14/cmake-3.14.7-Linux-x86_64.tar.gz
```

3. Extract the file and install CMake.

```shell
$ tar zxvf cmake-3.12.2-Linux-x86_64.tar.gz
$ mv cmake-3.12.2-Linux-x86_64 /opt/cmake-3.12.2
$ ln -sf /opt/cmake-3.12.2/bin/* /usr/bin/
$ tar zxvf cmake-3.14.7-Linux-x86_64.tar.gz
$ mv cmake-3.14.7-Linux-x86_64 /opt/cmake-3.14.7
$ ln -sf /opt/cmake-3.14.7/bin/* /usr/bin/
```
34 changes: 15 additions & 19 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
![Milvuslogo](https://github.com/milvus-io/docs/blob/master/assets/milvus_logo.png)

![Milvuslogo](https://github.com/milvus-io/docs/blob/master/v0.9.1/assets/milvus_logo.png)
[![Slack](https://img.shields.io/badge/Join-Slack-orange)](https://join.slack.com/t/milvusio/shared_invite/zt-e0u4qu3k-bI2GDNys3ZqX1YCJ9OM~GQ)
![GitHub](https://img.shields.io/github/license/milvus-io/milvus)
![Docker pulls](https://img.shields.io/docker/pulls/milvusdb/milvus)
Expand All @@ -11,29 +10,26 @@
[![CodeFactor Grade](https://www.codefactor.io/repository/github/milvus-io/milvus/badge)](https://www.codefactor.io/repository/github/milvus-io/milvus)
[![Codacy Badge](https://api.codacy.com/project/badge/Grade/c4bb2ccfb51b47f99e43bfd1705edd95)](https://app.codacy.com/gh/milvus-io/milvus?utm_source=github.com&utm_medium=referral&utm_content=milvus-io/milvus&utm_campaign=Badge_Grade_Dashboard)

English | [中文版](README_CN.md)

## What is Milvus

As an open source vector similarity search engine, Milvus is easy-to-use, highly reliable, scalable, robust, and blazing fast. Adopted by over 100 organizations and institutions worldwide, Milvus empowers applications in a variety of fields, including image processing, computer vision, natural language processing, voice recognition, recommender systems, drug discovery, etc.

Milvus has the following architecture:
As an open source vector similarity search engine, Milvus is easy-to-use, highly reliable, scalable, robust, and blazing fast. Adopted by over 100 organizations and institutions worldwide, Milvus empowers applications in a variety of fields, including image processing, computer vision, natural language processing, voice recognition, recommender systems, drug discovery, and more.

![arch](https://github.com/milvus-io/docs/blob/v0.7.1/assets/milvus_arch.png)
The following is Milvus architecture:

For more detailed introduction of Milvus and its architecture, see [Milvus overview](https://www.milvus.io/docs/about_milvus/overview.md). Keep up-to-date with newest releases and latest updates by reading Milvus [release notes](https://www.milvus.io/docs/releases/release_notes.md).
![arch](https://github.com/milvus-io/docs/blob/master/v0.9.1/assets/milvus_arch.png)
For more detailed introduction of Milvus and its architecture, see [Milvus overview](https://www.milvus.io/docs/overview.md). See Milvus [release notes](https://www.milvus.io/docs/release_notes.md) to keep up-to-date with its releases and updates.

Milvus is an [LF AI Foundation](https://lfai.foundation/) incubation project. Learn more at [lfai.foundation](https://lfai.foundation/).
Milvus is an incubation-stage project at [LF AI & Data Foundation](https://lfaidata.foundation/).

## Get started

### Install Milvus

See the [Milvus install guide](https://www.milvus.io/docs/guides/get_started/install_milvus/install_milvus.md) to install Milvus using Docker. To install Milvus from source code, see [build from source](INSTALL.md).
See [Milvus install guide](https://www.milvus.io/docs/install_milvus.md) to install Milvus using Docker. To install Milvus from source code, see [build from source](INSTALL.md).

### Try example programs

Try an example program with Milvus using [Python](https://www.milvus.io/docs/guides/get_started/example_code.md), [Java](https://github.com/milvus-io/milvus-sdk-java/tree/master/examples), [Go](https://github.com/milvus-io/milvus-sdk-go/tree/master/examples), or [C++ example code](https://github.com/milvus-io/milvus/tree/master/sdk/examples).
Try an example program with Milvus using [Python](https://www.milvus.io/docs/example_code.md), [Java](https://github.com/milvus-io/milvus-sdk-java/tree/master/examples), [Go](https://github.com/milvus-io/milvus-sdk-go/tree/master/examples), or [C++ example code](https://github.com/milvus-io/milvus/tree/master/sdk/examples).

## Supported clients

Expand All @@ -42,33 +38,33 @@ Try an example program with Milvus using [Python](https://www.milvus.io/docs/gui
- [Java](https://github.com/milvus-io/milvus-sdk-java)
- [C++](https://github.com/milvus-io/milvus/tree/master/sdk)
- [RESTful API](https://github.com/milvus-io/milvus/tree/master/core/src/server/web_impl)
- [Node.js](https://www.npmjs.com/package/@arkie-ai/milvus-client) (Provided by [arkie](https://www.arkie.cn/))
- [Node.js](https://www.npmjs.com/package/@arkie-ai/milvus-client) (Contributed by [arkie](https://www.arkie.cn/))

## Application scenarios

You can use Milvus to build intelligent systems in a variety of AI application scenarios. Refer to [Milvus Scenarios](https://milvus.io/scenarios) for live demos. You can also refer to [Milvus Bootcamp](https://github.com/milvus-io/bootcamp) for detailed solutions and application scenarios.
You can use Milvus to build intelligent systems in a variety of AI application scenarios. See [Milvus Scenarios](https://milvus.io/scenarios) for live demos. You can also see [Milvus Bootcamp](https://github.com/milvus-io/bootcamp) for detailed solutions and application scenarios.

## Benchmark

See our [test reports](https://github.com/milvus-io/milvus/tree/master/docs) for more information about performance benchmarking of different indexes in Milvus.

## Roadmap

To learn what's coming up soon in Milvus, read our [Roadmap](https://github.com/milvus-io/milvus/projects).
To learn what's coming up soon in Milvus, read our [Roadmap](https://github.com/milvus-io/milvus/milestones).

It is a Work in Progress, and is subject to reasonable adjustments when necessary. And we greatly welcome any comments/requirements/suggestions regarding Milvus roadmap.:clap:
It is a Work in Progress, and is subject to reasonable adjustments when necessary. And we greatly appreciate any comments/requirements/suggestions regarding Milvus' roadmap.:clap:

## Contribution guidelines

Contributions are welcomed and greatly appreciated. Please read our [contribution guidelines](CONTRIBUTING.md) for detailed contribution workflow. This project adheres to the [code of conduct](CODE_OF_CONDUCT.md) of Milvus. By participating, you are expected to uphold this code.
Contributions are welcomed and greatly appreciated. Please read our [contribution guidelines](CONTRIBUTING.md) for detailed contribution workflow. This project adheres to the [code of conduct](CODE_OF_CONDUCT.md) of Milvus. You must abide by this code in order to participate.

We use [GitHub issues](https://github.com/milvus-io/milvus/issues) to track issues and bugs. For general questions and public discussions, please join our community.

## Join our community

:heart:To connect with other users and contributors, welcome to join our [Slack channel](https://join.slack.com/t/milvusio/shared_invite/zt-e0u4qu3k-bI2GDNys3ZqX1YCJ9OM~GQ).
:heart: To connect with other users and contributors, welcome to join our [Slack channel](https://join.slack.com/t/milvusio/shared_invite/zt-e0u4qu3k-bI2GDNys3ZqX1YCJ9OM~GQ).

See our [community](https://github.com/milvus-io/community) repository to learn about our governance and access more community resources.
See our [community](https://github.com/milvus-io/community) repository to learn more about our governance and access more community resources.


## Resources
Expand Down