Skip to content
This repository has been archived by the owner on Jan 18, 2023. It is now read-only.

Commit

Permalink
Merge pull request #41 from milvus-io/adapt-rc7
Browse files Browse the repository at this point in the history 
Support rc7
  • Loading branch information
@czhen-zilliz
czhen-zilliz committed Oct 12, 2021
2 parents ba38b97 + 2f6b06d commit 9cea5a9
Show file tree
Hide file tree
Showing 4 changed files with 179 additions and 99 deletions.
116 changes: 73 additions & 43 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -178,18 +178,23 @@ Usage: milvus_cli.py create index [OPTIONS]
Example:
create index -c car -f vector -t IVF_FLAT -m L2 -p nlist:128
milvus_cli > create index
Collection name (car, car2): car2
The name of the field to create an index for (vector): vector
Index type (FLAT, IVF_FLAT, IVF_SQ8, IVF_PQ, RNSG, HNSW, ANNOY):
IVF_FLAT
Index metric type (L2, IP, HAMMING, TANIMOTO): L2
Index params nlist: 2
Timeout []:
Options:
-c, --collection TEXT Collection name.
-f, --field TEXT The name of the field to create an index for.
-t, --index-type TEXT Index type.
-m, --index-metric TEXT Index metric type.
-p, --index-params TEXT Index params, usage is "<Name>:<Value>"
-e, --timeout INTEGER An optional duration of time in seconds to allow
for the RPC. When timeout is set to None, client
waits until server response or error occur.
--help Show this message and exit.
--help Show this message and exit.
```

#### `delete`
Expand Down Expand Up @@ -509,61 +514,86 @@ Usage: milvus_cli.py search [OPTIONS]
Conducts a vector similarity search with an optional boolean expression as
filter.
Example-1:
Example-1(import a csv file):
Collection name: car
Collection name (car, test_collection): car
The vectors of search data, the length of data is number of query (nq),
The vectors of search data(the length of data is number of query (nq),
the dim of every vector in data must be equal to vector field’s of
collection: examples/import_csv/search_vectors.csv
collection. You can also import a csv file with out headers):
examples/import_csv/search_vectors.csv
The vector field used to search of collection []: vector
The vector field used to search of collection (vector): vector
Metric type []: L2
Metric type: L2
The parameters of search(split by "," if multiple) []: nprobe:10
Search parameter nprobe's value: 10
The max number of returned record, also known as topk []: 2
The max number of returned record, also known as topk: 2
The boolean expression used to filter attribute []: id > 0
The names of partitions to search(split by "," if multiple) []: _default
The names of partitions to search(split by "," if multiple) ['_default']
[]: _default
timeout []:
Example-2(collection has index):
Example-2:
Collection name (car, test_collection): car
Collection name: car
The vectors of search data(the length of data is number of query (nq),
the dim of every vector in data must be equal to vector field’s of
collection. You can also import a csv file with out headers):
[[0.71, 0.76, 0.17, 0.13, 0.42, 0.07, 0.15, 0.67, 0.58, 0.02, 0.39,
0.47, 0.58, 0.88, 0.73, 0.31, 0.23, 0.57, 0.33, 0.2, 0.03, 0.43,
0.78, 0.49, 0.17, 0.56, 0.76, 0.54, 0.45, 0.46, 0.05, 0.1, 0.43,
0.63, 0.29, 0.44, 0.65, 0.01, 0.35, 0.46, 0.66, 0.7, 0.88, 0.07,
0.49, 0.92, 0.57, 0.5, 0.16, 0.77, 0.98, 0.1, 0.44, 0.88, 0.82,
0.16, 0.67, 0.63, 0.57, 0.55, 0.95, 0.13, 0.64, 0.43, 0.71, 0.81,
0.43, 0.65, 0.76, 0.7, 0.05, 0.24, 0.03, 0.9, 0.46, 0.28, 0.92,
0.25, 0.97, 0.79, 0.73, 0.97, 0.49, 0.28, 0.64, 0.19, 0.23, 0.51,
0.09, 0.1, 0.53, 0.03, 0.23, 0.94, 0.87, 0.14, 0.42, 0.82, 0.91,
0.11, 0.91, 0.37, 0.26, 0.6, 0.89, 0.6, 0.32, 0.11, 0.98, 0.67,
0.12, 0.66, 0.47, 0.02, 0.15, 0.6, 0.64, 0.57, 0.14, 0.81, 0.75,
0.11, 0.49, 0.78, 0.16, 0.63, 0.57, 0.18]]
The vectors of search data, the length of data is number of query (nq),
the dim of every vector in data must be equal to vector field’s of
collection:
[[0.71, 0.76, 0.17, 0.13, 0.42, 0.07, 0.15, 0.67, 0.58, 0.02, 0.39,
0.47, 0.58, 0.88, 0.73, 0.31, 0.23, 0.57, 0.33, 0.2, 0.03, 0.43,
0.78, 0.49, 0.17, 0.56, 0.76, 0.54, 0.45, 0.46, 0.05, 0.1, 0.43,
0.63, 0.29, 0.44, 0.65, 0.01, 0.35, 0.46, 0.66, 0.7, 0.88, 0.07,
0.49, 0.92, 0.57, 0.5, 0.16, 0.77, 0.98, 0.1, 0.44, 0.88, 0.82,
0.16, 0.67, 0.63, 0.57, 0.55, 0.95, 0.13, 0.64, 0.43, 0.71, 0.81,
0.43, 0.65, 0.76, 0.7, 0.05, 0.24, 0.03, 0.9, 0.46, 0.28, 0.92,
0.25, 0.97, 0.79, 0.73, 0.97, 0.49, 0.28, 0.64, 0.19, 0.23, 0.51,
0.09, 0.1, 0.53, 0.03, 0.23, 0.94, 0.87, 0.14, 0.42, 0.82, 0.91,
0.11, 0.91, 0.37, 0.26, 0.6, 0.89, 0.6, 0.32, 0.11, 0.98, 0.67,
0.12, 0.66, 0.47, 0.02, 0.15, 0.6, 0.64, 0.57, 0.14, 0.81, 0.75,
0.11, 0.49, 0.78, 0.16, 0.63, 0.57, 0.18]]
The vector field used to search of collection (vector): vector
The vector field used to search of collection []: vector
Metric type: L2
Metric type []: L2
Search parameter nprobe's value: 10
The parameters of search(split by "," if multiple) []: nprobe:10
The specified number of decimal places of returned distance [-1]: 5
The max number of returned record, also known as topk []: 2
The max number of returned record, also known as topk: 2
The boolean expression used to filter attribute []: id > 0
The names of partitions to search(split by "," if multiple) []: _default
The names of partitions to search(split by "," if multiple) ['_default']
[]: _default
timeout []:
Example-3(collection has no index):
Collection name (car, car2): car
The vectors of search data(the length of data is number of query (nq),
the dim of every vector in data must be equal to vector field’s of
collection. You can also import a csv file with out headers):
examples/import_csv/search_vectors.csv
The vector field used to search of collection (vector): vector
The specified number of decimal places of returned distance [-1]: 5
The max number of returned record, also known as topk: 2
The boolean expression used to filter attribute []:
The names of partitions to search(split by "," if multiple) ['_default']
[]:
timeout []:
Timeout []:
Options:
--help Show this message and exit.
Expand Down
87 changes: 66 additions & 21 deletions milvus_cli/scripts/milvus_cli.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
from utils import getPackageVersion, readCsvFile
from utils import validateParamsByCustomFunc, validateCollectionParameter, validateIndexParameter, validateSearchParams, validateQueryParams
from utils import ParameterException, ConnectException
from utils import MetricTypes, IndexParams, SearchParams, IndexTypesMap
from utils import MetricTypes, IndexParams, SearchParams, IndexTypesMap, IndexTypes

pass_context = click.make_pass_decorator(PyOrm, ensure=True)

Expand Down Expand Up @@ -304,23 +304,45 @@ def createPartition(obj, collectionName, partition, description):


@createDetails.command('index')
@click.option('-c', '--collection', 'collectionName', help='Collection name.', default='')
@click.option('-f', '--field', 'fieldName', help='The name of the field to create an index for.', default='')
@click.option('-t', '--index-type', 'indexType', help='Index type.', default='')
@click.option('-m', '--index-metric', 'metricType', help='Index metric type.', default='')
@click.option('-p', '--index-params', 'params', help='Index params, usage is "<Name>:<Value>"', default=None, multiple=True)
@click.option('-e', '--timeout', 'timeout', help='An optional duration of time in seconds to allow for the RPC. When timeout is set to None, client waits until server response or error occur.', default=None, type=int)
@click.pass_obj
def createIndex(obj, collectionName, fieldName, indexType, metricType, params, timeout):
def createIndex(obj):
"""
Create index.
Example:
create index -c car -f vector -t IVF_FLAT -m L2 -p nlist:128
milvus_cli > create index
Collection name (car, car2): car2
The name of the field to create an index for (vector): vector
Index type (FLAT, IVF_FLAT, IVF_SQ8, IVF_PQ, RNSG, HNSW, ANNOY): IVF_FLAT
Index metric type (L2, IP, HAMMING, TANIMOTO): L2
Index params nlist: 2
Timeout []:
"""
try:
obj.checkConnection()
collectionName = click.prompt(
'Collection name', type=click.Choice(obj._list_collection_names()))
fieldName = click.prompt(
'The name of the field to create an index for', type=click.Choice(obj._list_field_names(collectionName, showVectorOnly=True)))
indexType = click.prompt(
'Index type', type=click.Choice(IndexTypes))
metricType = click.prompt(
'Index metric type', type=click.Choice(MetricTypes))
index_building_parameters = IndexTypesMap[indexType]['index_building_parameters']
params = []
for param in index_building_parameters:
tmpParam = click.prompt(
f'Index params {param}')
params.append(f'{param}:{tmpParam}')
inputTimeout = click.prompt('Timeout', default='')
timeout = inputTimeout if inputTimeout else None
validateIndexParameter(
indexType, metricType, params)
except ParameterException as pe:
Expand Down Expand Up @@ -428,12 +450,13 @@ def search(obj):
"""
Conducts a vector similarity search with an optional boolean expression as filter.
Example-1:
Example-1(import a csv file):
Collection name (car, test_collection): car
The vectors of search data, the length of data is number of query (nq),
the dim of every vector in data must be equal to vector field’s of collection: examples/import_csv/search_vectors.csv
The vectors of search data(the length of data is number of query (nq),
the dim of every vector in data must be equal to vector field’s of
collection. You can also import a csv file with out headers): examples/import_csv/search_vectors.csv
The vector field used to search of collection (vector): vector
Expand All @@ -447,14 +470,14 @@ def search(obj):
The names of partitions to search(split by "," if multiple) ['_default'] []: _default
Example-2:
Example-2(collection has index):
Collection name (car, test_collection): car
\b
The vectors of search data, the length of data is number of query (nq),
The vectors of search data(the length of data is number of query (nq),
the dim of every vector in data must be equal to vector field’s of
collection:
collection. You can also import a csv file with out headers):
[[0.71, 0.76, 0.17, 0.13, 0.42, 0.07, 0.15, 0.67, 0.58, 0.02, 0.39,
0.47, 0.58, 0.88, 0.73, 0.31, 0.23, 0.57, 0.33, 0.2, 0.03, 0.43,
0.78, 0.49, 0.17, 0.56, 0.76, 0.54, 0.45, 0.46, 0.05, 0.1, 0.43,
Expand All @@ -474,13 +497,35 @@ def search(obj):
Search parameter nprobe's value: 10
The specified number of decimal places of returned distance [-1]: 5
The max number of returned record, also known as topk: 2
The boolean expression used to filter attribute []: id > 0
The names of partitions to search(split by "," if multiple) ['_default'] []: _default
timeout []:
Example-3(collection has no index):
Collection name (car, car2): car
The vectors of search data(the length of data is number of query (nq),
the dim of every vector in data must be equal to vector field’s of
collection. You can also import a csv file with out headers): examples/import_csv/search_vectors.csv
The vector field used to search of collection (vector): vector
The specified number of decimal places of returned distance [-1]: 5
The max number of returned record, also known as topk: 2
The boolean expression used to filter attribute []:
The names of partitions to search(split by "," if multiple) ['_default'] []:
Timeout []:
"""
collectionName = click.prompt(
'Collection name', type=click.Choice(obj._list_collection_names()))
Expand All @@ -489,6 +534,7 @@ def search(obj):
annsField = click.prompt(
'The vector field used to search of collection', type=click.Choice(obj._list_field_names(collectionName, showVectorOnly=True)))
indexDetails = obj._list_index(collectionName)
hasIndex = not not indexDetails
if indexDetails:
index_type = indexDetails['index_type']
search_parameters = IndexTypesMap[index_type]['search_parameters']
Expand All @@ -500,17 +546,16 @@ def search(obj):
paramInput = click.prompt(f'Search parameter {parameter}\'s value')
params += [f"{parameter}:{paramInput}"]
else:
metricType = click.prompt(
'Metric type', default='', type=click.Choice(MetricTypes))
params = click.prompt(
f'The parameters of search(input "<type>:<value>" and split by "," if multiple, type should be one of {SearchParams})', default='')
metricType = ''
params = []
roundDecimal = click.prompt('The specified number of decimal places of returned distance', default=-1, type=int)
limit = click.prompt(
'The max number of returned record, also known as topk', default=None, type=int)
expr = click.prompt(
'The boolean expression used to filter attribute', default='')
partitionNames = click.prompt(
f'The names of partitions to search(split by "," if multiple) {obj._list_partition_names(collectionName)}', default='')
timeout = click.prompt('timeout', default='')
timeout = click.prompt('Timeout', default='')
export, exportPath = False, ''
# if click.confirm('Would you like to export results as a csv File?'):
# export = True
Expand All @@ -520,7 +565,7 @@ def search(obj):
# exportPath = click.prompt('Directory path to csv file')
try:
searchParameters = validateSearchParams(
data, annsField, metricType, params, limit, expr, partitionNames, timeout)
data, annsField, metricType, params, limit, expr, partitionNames, timeout, roundDecimal, hasIndex=hasIndex)
obj.checkConnection()
except ParameterException as pe:
click.echo("Error!\n{}".format(str(pe)))
Expand Down

0 comments on commit 9cea5a9

Please sign in to comment.