Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update client to support v1.10 #134

Merged
merged 29 commits into from
Jun 26, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
c721305
Update gRPC definitions to v1.10.0, add query test and resolve errors…
generall Jun 19, 2024
38bf414
add custom builder for VectorsConfig (#135)
generall Jun 20, 2024
9b66981
Improve crate layout, mark deprecations, resolve many test warnings (…
timvisee Jun 20, 2024
f2eaa01
Deprecate NotA and provide alternatives (#137)
timvisee Jun 20, 2024
57db89d
Deprecate the serde module (#138)
timvisee Jun 20, 2024
27ecdb1
Expose Payload at root, hide obsolete modules, vendor some types alon…
timvisee Jun 21, 2024
44a2f37
Add missing functionality and builder (#140)
JojiiOfficial Jun 21, 2024
a6c2098
Migrate rust client (#141)
JojiiOfficial Jun 21, 2024
c34355c
fix order of the snippets
generall Jun 22, 2024
787688a
Qdrant query conversions (#142)
generall Jun 24, 2024
b8804a3
fix typo
generall Jun 24, 2024
8ba4e76
Add documentation to operation/endpoint methods (#144)
timvisee Jun 24, 2024
37112f3
Fix incorrect docs warning
timvisee Jun 24, 2024
dc95911
Define upsert_points_batch alias in docs
timvisee Jun 24, 2024
2cd6142
Update example in README (#145)
timvisee Jun 25, 2024
3b3ad63
Build documentation with all features
timvisee Jun 25, 2024
6ced3e1
Add docs.rs badge to README (#146)
timvisee Jun 26, 2024
0d775b6
Add query batch interface (#149)
timvisee Jun 26, 2024
9c56948
Expose configuration types, improve configuration interface, add docs…
timvisee Jun 26, 2024
d4e45f6
Simplify JSON to/from Payload conversions (#148)
timvisee Jun 26, 2024
ae105d4
Add builders and conversions for cluster operation types (#151)
timvisee Jun 26, 2024
f248d94
Don't make config optional in client construction (#153)
timvisee Jun 26, 2024
8707fa0
Organize client operations, split into modules, order by CRUD (#152)
timvisee Jun 26, 2024
c5422e8
Mark deprecated gRPC types as deprecated (#154)
timvisee Jun 26, 2024
96e80ca
Make payload (de)serialization transparent
timvisee Jun 26, 2024
e995e31
Link documentation in README
timvisee Jun 26, 2024
af9a860
Review suggestions by @xzfc
timvisee Jun 26, 2024
e45dce6
Fix documentation mistakes in gRPC spec
timvisee Jun 26, 2024
99e51ee
Review suggestions by @xzfc
timvisee Jun 26, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .rusty-hook.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
[hooks]
pre-push = "cargo +nightly fmt --all -- --check"

[logging]
verbose = true
7 changes: 7 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -37,3 +37,10 @@ serde = ["dep:serde", "dep:serde_json"]
[[example]]
name = "search"
required-features = ["serde"]

[[example]]
name = "deprecated_search"
required-features = ["serde"]

[package.metadata.docs.rs]
all-features = true
138 changes: 57 additions & 81 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,27 @@
# rust-client
# Qdrant Rust client

Rust client for Qdrant vector search engine.
The [Qdrant](https://qdrant.tech/) - High-Performance Vector Search at Scale - client for Rust.

[![Crates.io][crates-badge]][crates-url]
[![docs.rs][docs-badge]][docs-url]
[![Apache 2.0 licensed][apache2-badge]][apache2-url]

[crates-badge]: https://img.shields.io/crates/v/qdrant-client.svg

[crates-url]: https://crates.io/crates/qdrant-client

[docs-badge]: https://img.shields.io/docsrs/qdrant-client.svg

[docs-url]: https://docs.rs/qdrant-client

[apache2-badge]: https://img.shields.io/badge/license-apache2-blue.svg

[apache2-url]: https://github.com/qdrant/rust-client/blob/master/LICENSE

Documentation:
- Qdrant documentation: <https://qdrant.tech/documentation/>
- Crate documentation: <https://docs.rs/qdrant-client>

## Installation

```bash
Expand Down Expand Up @@ -60,52 +69,43 @@ cargo add qdrant-client anyhow tonic tokio serde-json --features tokio/rt-multi-
Add search example from [`examples/search.rs`](./examples/search.rs) to your `src/main.rs`:

```rust
use anyhow::Result;
use qdrant_client::prelude::*;
use qdrant_client::qdrant::vectors_config::Config;
use qdrant_client::qdrant::{
Condition, CreateCollection, Filter, SearchPoints, VectorParams, VectorsConfig,
Condition, CreateCollectionBuilder, Distance, Filter, PointStruct, ScalarQuantizationBuilder,
SearchParamsBuilder, SearchPointsBuilder, UpsertPointsBuilder, VectorParamsBuilder,
};
use serde_json::json;
use qdrant_client::{Payload, Qdrant, QdrantError};

#[tokio::main]
async fn main() -> Result<()> {
async fn main() -> Result<(), QdrantError> {
// Example of top level client
// You may also use tonic-generated client from `src/qdrant.rs`
let client = QdrantClient::from_url("http://localhost:6334").build()?;
let client = Qdrant::from_url("http://localhost:6334").build()?;

let collections_list = client.list_collections().await?;
dbg!(collections_list);
// collections_list = ListCollectionsResponse {
// collections: [
// CollectionDescription {
// name: "test",
// },
// ],
// time: 1.78e-6,
// collections_list = {
// "collections": [
// {
// "name": "test"
// }
// ]
// }

let collection_name = "test";
client.delete_collection(collection_name).await?;

client
.create_collection(&CreateCollection {
collection_name: collection_name.into(),
vectors_config: Some(VectorsConfig {
config: Some(Config::Params(VectorParams {
size: 10,
distance: Distance::Cosine.into(),
..Default::default()
})),
}),
..Default::default()
})
.create_collection(
CreateCollectionBuilder::new(collection_name)
.vectors_config(VectorParamsBuilder::new(10, Distance::Cosine))
.quantization_config(ScalarQuantizationBuilder::default()),
)
.await?;

let collection_info = client.collection_info(collection_name).await?;
dbg!(collection_info);

let payload: Payload = json!(
let payload: Payload = serde_json::json!(
{
"foo": "Bar",
"bar": 12,
Expand All @@ -114,60 +114,37 @@ async fn main() -> Result<()> {
}
}
)
.try_into()
.unwrap();
.try_into()
.unwrap();

let points = vec![PointStruct::new(0, vec![12.; 10], payload)];
client
.upsert_points_blocking(collection_name, None, points, None)
.upsert_points(UpsertPointsBuilder::new(collection_name, points))
.await?;

let search_result = client
.search_points(&SearchPoints {
collection_name: collection_name.into(),
vector: vec![11.; 10],
filter: Some(Filter::all([Condition::matches("bar", 12)])),
limit: 10,
with_payload: Some(true.into()),
..Default::default()
})
.search_points(
SearchPointsBuilder::new(collection_name, [11.; 10], 10)
.filter(Filter::all([Condition::matches("bar", 12)]))
.with_payload(true)
.params(SearchParamsBuilder::default().exact(true)),
)
.await?;
dbg!(&search_result);
// search_result = SearchResponse {
// result: [
// ScoredPoint {
// id: Some(
// PointId {
// point_id_options: Some(
// Num(
// 0,
// ),
// ),
// },
// ),
// payload: {
// "bar": Value {
// kind: Some(
// IntegerValue(
// 12,
// ),
// ),
// },
// "foo": Value {
// kind: Some(
// StringValue(
// "Bar",
// ),
// ),
// },
// },
// score: 1.0000001,
// version: 0,
// vectors: None,
// },
// ],
// time: 9.5394e-5,
// }
// search_result = [
// {
// "id": 0,
// "version": 0,
// "score": 1.0000001,
// "payload": {
// "bar": 12,
// "baz": {
// "qux": "quux"
// },
// "foo": "Bar"
// }
// }
// ]

let found_point = search_result.result.into_iter().next().unwrap();
let mut payload = found_point.payload;
Expand Down Expand Up @@ -195,11 +172,10 @@ The client needs to be configured properly to access the service.
- make sure to pass your API KEY

```rust
async fn make_client() -> Result<QdrantClient> {
let client = QdrantClient::from_url("http://xxxxxxxxxx.eu-central.aws.cloud.qdrant.io:6334")
// using an env variable for the API KEY for example
.with_api_key(std::env::var("QDRANT_API_KEY"))
.build()?;
Ok(client)
}
use qdrant_client::Qdrant;

let client = Qdrant::from_url("http://xxxxxxxxxx.eu-central.aws.cloud.qdrant.io:6334")
// Use an environment variable for the API KEY for example
.api_key(std::env::var("QDRANT_API_KEY"))
.build()?;
```
80 changes: 80 additions & 0 deletions build.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
use std::fs;
use std::path::{Path, PathBuf};

fn generate_snippet_test(snippet: &str, name: &str, dir: &Path) {
// Pad each line of the snippet with 8 spaces

let snippet = snippet
.lines()
.map(|line| format!(" {}", line))
.collect::<Vec<_>>()
.join("\n");

let test_snippet = format!(
r#"
#[tokio::test]
async fn test_{name}() {{
async fn {name}() -> Result<(), Box<dyn std::error::Error>> {{
// WARNING: This is a generated test snippet.
// Please, modify the snippet in the `../snippets/{name}.rs` file
{snippet}
Ok(())
}}
let _ = {name}().await;
}}
"#
);

// Write the test snippet to the file
let mut test_snippet_path = PathBuf::from(dir);
test_snippet_path.push(format!("test_{}.rs", name));
fs::write(test_snippet_path, test_snippet).unwrap();
}

fn main() {
println!("cargo:rerun-if-changed=tests/snippets");

// Open all files in the `./tests/snippets` directory and save them to the `./tests/snippets_converted` directory
// Wrap text in the `tests/test_snippets.rs` file into { }
let snippets_dir = Path::new("./tests/snippets");
let tests_output_dir = Path::new("./tests/snippet_tests");

// Create the converted directory if it doesn't exist
if !tests_output_dir.exists() {
fs::create_dir_all(tests_output_dir).unwrap();
}

let mut snippet_names: Vec<_> = fs::read_dir(snippets_dir)
.unwrap()
.filter_map(|entry| {
let entry = entry.unwrap();
let path = entry.path();
if path.is_file() {
let snippet_name = path.file_stem().unwrap().to_str().unwrap().to_string();
Some((path, snippet_name))
} else {
None
}
})
.collect();

snippet_names.sort_unstable_by_key(|(_, name)| name.clone());

for (path, name) in &snippet_names {
let content = fs::read_to_string(path).unwrap();
generate_snippet_test(&content, name, tests_output_dir);
}

// Generate `tests/snippet_tests/mod.rs` file
// For each file in `./tests/snippet_tests` directory, generate a line `mode {file_name};`

let mod_file = snippet_names
.iter()
.map(|(_, name)| format!("mod test_{};", name))
.collect::<Vec<_>>()
.join("\n");

let mut mod_file_path = PathBuf::from(tests_output_dir);
mod_file_path.push("mod.rs");
fs::write(mod_file_path, mod_file).unwrap();
}
Loading
Loading