From 9acd5181dfcfcca415105e1251d5f58d086267ba Mon Sep 17 00:00:00 2001 From: Robert Shelton Date: Tue, 1 Apr 2025 11:52:11 -0400 Subject: [PATCH 1/2] update correct this time --- .../develop/get-started/vector-database.md | 2 +- content/integrate/amazon-bedrock/_index.md | 2 +- content/integrate/redisvl/overview/install.md | 75 +++++ .../redisvl/user-guide/json-v-hashes.md | 306 ++++++++++++++++++ 4 files changed, 383 insertions(+), 2 deletions(-) create mode 100644 content/integrate/redisvl/overview/install.md create mode 100644 content/integrate/redisvl/user-guide/json-v-hashes.md diff --git a/content/develop/get-started/vector-database.md b/content/develop/get-started/vector-database.md index 15411c7cc5..d3f10323e5 100644 --- a/content/develop/get-started/vector-database.md +++ b/content/develop/get-started/vector-database.md @@ -281,7 +281,7 @@ From the description, this bike is an excellent match for younger children, and 1. You can learn more about the query options, such as filters and vector range queries, by reading the [vector reference documentation]({{< relref "/develop/interact/search-and-query/advanced-concepts/vectors" >}}). 2. The complete [Redis Query Engine documentation]({{< relref "/develop/interact/search-and-query/" >}}) might be interesting for you. -3. If you want to follow the code examples more interactively, then you can use the [Jupyter notebook](https://github.com/RedisVentures/redis-vss-getting-started/blob/main/vector_similarity_with_redis.ipynb) that inspired this quick start guide. +3. If you want to follow the code examples more interactively, then you can use the [Jupyter notebook](https://github.com/redis-developer/redis-ai-resources/blob/main/python-recipes/vector-search/00_redispy.ipynb) that inspired this quick start guide. 4. If you want to see more advanced examples of a Redis vector database in action, visit the [Redis AI Resources](https://github.com/redis-developer/redis-ai-resources) page on GitHub. ## Continue learning with Redis University diff --git a/content/integrate/amazon-bedrock/_index.md b/content/integrate/amazon-bedrock/_index.md index 2bab818933..97657767bf 100644 --- a/content/integrate/amazon-bedrock/_index.md +++ b/content/integrate/amazon-bedrock/_index.md @@ -36,4 +36,4 @@ To fully set up Bedrock with Redis Cloud, you will need to do the following: ## More info - [Amazon Bedrock integration blog post](https://redis.io/blog/amazon-bedrock-integration-with-redis-enterprise/) -- [Detailed steps](https://github.com/RedisVentures/aws-redis-bedrock-stack/blob/main/README.md) +- [Detailed steps](https://github.com/redis-applied-ai/aws-redis-bedrock-stack/blob/main/README.md) diff --git a/content/integrate/redisvl/overview/install.md b/content/integrate/redisvl/overview/install.md new file mode 100644 index 0000000000..64bdbbc048 --- /dev/null +++ b/content/integrate/redisvl/overview/install.md @@ -0,0 +1,75 @@ +--- +description: How to install RedisVL +title: Install RedisVL +type: integration +--- +There are a few ways to install RedisVL. The easiest way is to use Python's `pip` command. + +## Install RedisVL with pip + +Install `redisvl` into your Python (>=3.8) environment using `pip`: + +```bash +$ pip install -U redisvl +``` + +RedisVL comes with a few dependencies that are automatically installed. However, a few dependencies +are optional and can be installed separately if needed: + +```bash +$ pip install redisvl[all] # install vectorizer dependencies +$ pip install redisvl[dev] # install dev dependencies +``` + +If you use Zsh, remember to escape the brackets: + +```bash +$ pip install redisvl\[all\] +``` + +This library supports the use of [hiredis](https://redis.com/lp/hiredis/), so you can also install RedisVL by running: + +```bash +pip install redisvl[hiredis] +``` + +## Install RedisVL from source + +To install RedisVL from source, clone the repository and install the package using `pip`: + +```bash +$ git clone git@github.com:redis/redis-vl-python.git && cd redis-vl-python +$ pip install . + +# or for an editable installation (for developers of RedisVL) +$ pip install -e . +``` + +## Install Redis + +RedisVL requires a distribution of Redis that supports the [search and query](https://redis.com/modules/redis-search/) capability, of which there are three: + +1. [Redis Cloud](https://redis.com/try-free), a fully managed cloud offering that you can try for free. +2. [Redis Stack]({{< relref "/operate/oss_and_stack/install/install-stack/docker" >}}), a local docker image for testing and development. +3. [Redis Enterprise](https://redis.com/redis-enterprise/), a commercial self-hosted offering. + +### Redis Cloud + +Redis Cloud is the easiest way to get started with RedisVL. You can sign up for a free account [here](https://redis.com/try-free). Make sure to have the **Search and Query** capability enabled when creating your database. + +### Redis Stack (local development) + +For local development and testing, Redis Stack and be used. We recommend running Redis +in a docker container. To do so, run the following command: + +```bash +docker run -d --name redis-stack -p 6379:6379 -p 8001:8001 redis/redis-stack:latest +``` + +This will also start the [Redis Insight application](https://redis.com/redis-enterprise/redis-insight/) at `http://localhost:8001`. + +### Redis Enterprise (self-hosted) + +Redis Enterprise is a commercial offering that can be self-hosted. You can download the latest version [here](https://redis.com/redis-enterprise-software/download-center/software/). + +If you are considering a self-hosted Redis Enterprise deployment on Kubernetes, there is the [Redis Enterprise Operator](https://docs.redis.com/latest/kubernetes/) for Kubernetes. This will allow you to easily deploy and manage a Redis Enterprise cluster on Kubernetes. \ No newline at end of file diff --git a/content/integrate/redisvl/user-guide/json-v-hashes.md b/content/integrate/redisvl/user-guide/json-v-hashes.md new file mode 100644 index 0000000000..f6b1ee23d9 --- /dev/null +++ b/content/integrate/redisvl/user-guide/json-v-hashes.md @@ -0,0 +1,306 @@ +--- +description: Storing JSON and hashes with RedisVL +linkTitle: JSON vs. hash storage +title: JSON vs. hash storage +type: integration +weight: 6 +--- + +Out of the box, Redis provides a [variety of data structures](https://redis.com/redis-enterprise/data-structures/) that can be used for your domain specific applications and use cases. +In this document, you will learn how to use RedisVL with both [hash]({{< relref "/develop/data-types/hashes" >}}) and [JSON]({{< relref "/develop/data-types/json/" >}}) data. + +{{< note >}} +This document is a converted form of [this Jupyter notebook](https://github.com/redis/redis-vl-python/blob/main/docs/user_guide/05_hash_vs_json.ipynb). +{{< /note >}} + +Before beginning, be sure of the following: + +1. You have installed RedisVL and have that environment activated. +1. You have a running Redis instance with the search and query capability. + +```python +# import necessary modules +import pickle + +from redisvl.redis.utils import buffer_to_array +from jupyterutils import result_print, table_print +from redisvl.index import SearchIndex + +# load in the example data and printing utils +data = pickle.load(open("hybrid_example_data.pkl", "rb")) +``` + +```python +table_print(data) +``` + +
useragejobcredit_scoreoffice_locationuser_embedding
john18engineerhigh-122.4194,37.7749b'\xcd\xcc\xcc=\xcd\xcc\xcc=\x00\x00\x00?'
derrick14doctorlow-122.4194,37.7749b'\xcd\xcc\xcc=\xcd\xcc\xcc=\x00\x00\x00?'
nancy94doctorhigh-122.4194,37.7749b'333?\xcd\xcc\xcc=\x00\x00\x00?'
tyler100engineerhigh-122.0839,37.3861b'\xcd\xcc\xcc=\xcd\xcc\xcc>\x00\x00\x00?'
tim12dermatologisthigh-122.0839,37.3861b'\xcd\xcc\xcc>\xcd\xcc\xcc>\x00\x00\x00?'
taimur15CEOlow-122.0839,37.3861b'\x9a\x99\x19?\xcd\xcc\xcc=\x00\x00\x00?'
joe35dentistmedium-122.0839,37.3861b'fff?fff?\xcd\xcc\xcc='
+ + +## Hash or JSON - how to choose? + +Both storage options offer a variety of features and tradeoffs. Below, you will work through a dummy dataset to learn when and how to use both data types. + +### Working with hashes + +Hashes in Redis are simple collections of field-value pairs. Think of it like a mutable, single-level dictionary that contains multiple "rows": + +```python +{ + "model": "Deimos", + "brand": "Ergonom", + "type": "Enduro bikes", + "price": 4972, +} +``` + +Hashes are best suited for use cases with the following characteristics: + +- Performance (speed) and storage space (memory consumption) are top concerns. +- Data can be easily normalized and modeled as a single-level dictionary. + +> Hashes are typically the default recommendation. + +```python +# define the hash index schema +hash_schema = { + "index": { + "name": "user-hash", + "prefix": "user-hash-docs", + "storage_type": "hash", # default setting -- HASH + }, + "fields": [ + {"name": "user", "type": "tag"}, + {"name": "credit_score", "type": "tag"}, + {"name": "job", "type": "text"}, + {"name": "age", "type": "numeric"}, + {"name": "office_location", "type": "geo"}, + { + "name": "user_embedding", + "type": "vector", + "attrs": { + "dims": 3, + "distance_metric": "cosine", + "algorithm": "flat", + "datatype": "float32" + } + } + ], +} +``` + +```python +# construct a search index from the hash schema +hindex = SearchIndex.from_dict(hash_schema) + +# connect to local redis instance +hindex.connect("redis://localhost:6379") + +# create the index (no data yet) +hindex.create(overwrite=True) +``` + +```python +# show the underlying storage type +hindex.storage_type + + +``` + +#### Vectors as byte strings + +One nuance when working with hashes in Redis is that all vectorized data must be passed as a byte string (for efficient storage, indexing, and processing). An example of this can be seen below: + + +```python +# show a single entry from the data that will be loaded +data[0] + + {'user': 'john', + 'age': 18, + 'job': 'engineer', + 'credit_score': 'high', + 'office_location': '-122.4194,37.7749', + 'user_embedding': b'\xcd\xcc\xcc=\xcd\xcc\xcc=\x00\x00\x00?'} +``` + +```python +# load hash data +keys = hindex.load(data) +``` + +```python +$ rvl stats -i user-hash + + Statistics: + ╭─────────────────────────────┬─────────────╮ + │ Stat Key │ Value │ + ├─────────────────────────────┼─────────────┤ + │ num_docs │ 7 │ + │ num_terms │ 6 │ + │ max_doc_id │ 7 │ + │ num_records │ 44 │ + │ percent_indexed │ 1 │ + │ hash_indexing_failures │ 0 │ + │ number_of_uses │ 1 │ + │ bytes_per_record_avg │ 3.40909 │ + │ doc_table_size_mb │ 0.000767708 │ + │ inverted_sz_mb │ 0.000143051 │ + │ key_table_size_mb │ 0.000248909 │ + │ offset_bits_per_record_avg │ 8 │ + │ offset_vectors_sz_mb │ 8.58307e-06 │ + │ offsets_per_term_avg │ 0.204545 │ + │ records_per_doc_avg │ 6.28571 │ + │ sortable_values_size_mb │ 0 │ + │ total_indexing_time │ 0.587 │ + │ total_inverted_index_blocks │ 18 │ + │ vector_index_sz_mb │ 0.0202332 │ + ╰─────────────────────────────┴─────────────╯ +``` + +#### Performing queries + +Once the index is created and data is loaded into the right format, you can run queries against the index: + +```python +from redisvl.query import VectorQuery +from redisvl.query.filter import Tag, Text, Num + +t = (Tag("credit_score") == "high") & (Text("job") % "enginee*") & (Num("age") > 17) + +v = VectorQuery([0.1, 0.1, 0.5], + "user_embedding", + return_fields=["user", "credit_score", "age", "job", "office_location"], + filter_expression=t) + + +results = hindex.query(v) +result_print(results) + +``` + +
vector_distanceusercredit_scoreagejoboffice_location
0johnhigh18engineer-122.4194,37.7749
0.109129190445tylerhigh100engineer-122.0839,37.3861
+ +```python +# clean up +hindex.delete() +``` + +### Working with JSON + +Redis also supports native **JSON** objects. These can be multi-level (nested) objects, with full [JSONPath]({{< relref "/develop/data-types/json/" >}}path/) support for retrieving and updating sub-elements: + +```python +{ + "name": "bike", + "metadata": { + "model": "Deimos", + "brand": "Ergonom", + "type": "Enduro bikes", + "price": 4972, + } +} +``` + +JSON is best suited for use cases with the following characteristics: + +- Ease of use and data model flexibility are top concerns. +- Application data is already native JSON. +- Replacing another document storage/database solution. + +#### Full JSON Path support + +Because Redis enables full JSONPath support, when creating an index schema, elements need to be indexed and selected by their path with the desired `name` and `path` that points to where the data is located within the objects. + +{{< note >}} +By default, RedisVL will assume the path as `$.{name}` if not provided in JSON fields schema. +{{< /note >}} + +```python +# define the json index schema +json_schema = { + "index": { + "name": "user-json", + "prefix": "user-json-docs", + "storage_type": "json", # JSON storage type + }, + "fields": [ + {"name": "user", "type": "tag"}, + {"name": "credit_score", "type": "tag"}, + {"name": "job", "type": "text"}, + {"name": "age", "type": "numeric"}, + {"name": "office_location", "type": "geo"}, + { + "name": "user_embedding", + "type": "vector", + "attrs": { + "dims": 3, + "distance_metric": "cosine", + "algorithm": "flat", + "datatype": "float32" + } + } + ], +} +``` + +```python +# construct a search index from the JSON schema +jindex = SearchIndex.from_dict(json_schema) + +# connect to a local redis instance +jindex.connect("redis://localhost:6379") + +# create the index (no data yet) +jindex.create(overwrite=True) +``` + +```python +# note the multiple indices in the same database +$ rvl index listall + + 20:23:08 [RedisVL] INFO Indices: + 20:23:08 [RedisVL] INFO 1. user-json + +#### Vectors as float arrays + +Vectorized data stored in JSON must be stored as a pure array (e.g., a Python list) of floats. Modify your sample data to account for this below: + +```python +import numpy as np + +json_data = data.copy() + +for d in json_data: + d['user_embedding'] = buffer_to_array(d['user_embedding'], dtype=np.float32) +``` + +```python +# inspect a single JSON record +json_data[0] +``` + + {'user': 'john', + 'age': 18, + 'job': 'engineer', + 'credit_score': 'high', + 'office_location': '-122.4194,37.7749', + 'user_embedding': [0.10000000149011612, 0.10000000149011612, 0.5]} + + +```python +keys = jindex.load(json_data) +``` + +```python +# we can now run the exact same query as above +result_print(jindex.query(v)) +``` + +
vector_distanceusercredit_scoreagejoboffice_location
0johnhigh18engineer-122.4194,37.7749
0.109129190445tylerhigh100engineer-122.0839,37.3861
+ +## Cleanup + +```python +jindex.delete() +``` From 9bbffc49a8f371c70e102751e3423946421b25b1 Mon Sep 17 00:00:00 2001 From: Robert Shelton Date: Tue, 1 Apr 2025 11:56:13 -0400 Subject: [PATCH 2/2] remove dangling files --- content/integrate/redisvl/overview/install.md | 75 ----- .../redisvl/user-guide/json-v-hashes.md | 306 ------------------ 2 files changed, 381 deletions(-) delete mode 100644 content/integrate/redisvl/overview/install.md delete mode 100644 content/integrate/redisvl/user-guide/json-v-hashes.md diff --git a/content/integrate/redisvl/overview/install.md b/content/integrate/redisvl/overview/install.md deleted file mode 100644 index 64bdbbc048..0000000000 --- a/content/integrate/redisvl/overview/install.md +++ /dev/null @@ -1,75 +0,0 @@ ---- -description: How to install RedisVL -title: Install RedisVL -type: integration ---- -There are a few ways to install RedisVL. The easiest way is to use Python's `pip` command. - -## Install RedisVL with pip - -Install `redisvl` into your Python (>=3.8) environment using `pip`: - -```bash -$ pip install -U redisvl -``` - -RedisVL comes with a few dependencies that are automatically installed. However, a few dependencies -are optional and can be installed separately if needed: - -```bash -$ pip install redisvl[all] # install vectorizer dependencies -$ pip install redisvl[dev] # install dev dependencies -``` - -If you use Zsh, remember to escape the brackets: - -```bash -$ pip install redisvl\[all\] -``` - -This library supports the use of [hiredis](https://redis.com/lp/hiredis/), so you can also install RedisVL by running: - -```bash -pip install redisvl[hiredis] -``` - -## Install RedisVL from source - -To install RedisVL from source, clone the repository and install the package using `pip`: - -```bash -$ git clone git@github.com:redis/redis-vl-python.git && cd redis-vl-python -$ pip install . - -# or for an editable installation (for developers of RedisVL) -$ pip install -e . -``` - -## Install Redis - -RedisVL requires a distribution of Redis that supports the [search and query](https://redis.com/modules/redis-search/) capability, of which there are three: - -1. [Redis Cloud](https://redis.com/try-free), a fully managed cloud offering that you can try for free. -2. [Redis Stack]({{< relref "/operate/oss_and_stack/install/install-stack/docker" >}}), a local docker image for testing and development. -3. [Redis Enterprise](https://redis.com/redis-enterprise/), a commercial self-hosted offering. - -### Redis Cloud - -Redis Cloud is the easiest way to get started with RedisVL. You can sign up for a free account [here](https://redis.com/try-free). Make sure to have the **Search and Query** capability enabled when creating your database. - -### Redis Stack (local development) - -For local development and testing, Redis Stack and be used. We recommend running Redis -in a docker container. To do so, run the following command: - -```bash -docker run -d --name redis-stack -p 6379:6379 -p 8001:8001 redis/redis-stack:latest -``` - -This will also start the [Redis Insight application](https://redis.com/redis-enterprise/redis-insight/) at `http://localhost:8001`. - -### Redis Enterprise (self-hosted) - -Redis Enterprise is a commercial offering that can be self-hosted. You can download the latest version [here](https://redis.com/redis-enterprise-software/download-center/software/). - -If you are considering a self-hosted Redis Enterprise deployment on Kubernetes, there is the [Redis Enterprise Operator](https://docs.redis.com/latest/kubernetes/) for Kubernetes. This will allow you to easily deploy and manage a Redis Enterprise cluster on Kubernetes. \ No newline at end of file diff --git a/content/integrate/redisvl/user-guide/json-v-hashes.md b/content/integrate/redisvl/user-guide/json-v-hashes.md deleted file mode 100644 index f6b1ee23d9..0000000000 --- a/content/integrate/redisvl/user-guide/json-v-hashes.md +++ /dev/null @@ -1,306 +0,0 @@ ---- -description: Storing JSON and hashes with RedisVL -linkTitle: JSON vs. hash storage -title: JSON vs. hash storage -type: integration -weight: 6 ---- - -Out of the box, Redis provides a [variety of data structures](https://redis.com/redis-enterprise/data-structures/) that can be used for your domain specific applications and use cases. -In this document, you will learn how to use RedisVL with both [hash]({{< relref "/develop/data-types/hashes" >}}) and [JSON]({{< relref "/develop/data-types/json/" >}}) data. - -{{< note >}} -This document is a converted form of [this Jupyter notebook](https://github.com/redis/redis-vl-python/blob/main/docs/user_guide/05_hash_vs_json.ipynb). -{{< /note >}} - -Before beginning, be sure of the following: - -1. You have installed RedisVL and have that environment activated. -1. You have a running Redis instance with the search and query capability. - -```python -# import necessary modules -import pickle - -from redisvl.redis.utils import buffer_to_array -from jupyterutils import result_print, table_print -from redisvl.index import SearchIndex - -# load in the example data and printing utils -data = pickle.load(open("hybrid_example_data.pkl", "rb")) -``` - -```python -table_print(data) -``` - -
useragejobcredit_scoreoffice_locationuser_embedding
john18engineerhigh-122.4194,37.7749b'\xcd\xcc\xcc=\xcd\xcc\xcc=\x00\x00\x00?'
derrick14doctorlow-122.4194,37.7749b'\xcd\xcc\xcc=\xcd\xcc\xcc=\x00\x00\x00?'
nancy94doctorhigh-122.4194,37.7749b'333?\xcd\xcc\xcc=\x00\x00\x00?'
tyler100engineerhigh-122.0839,37.3861b'\xcd\xcc\xcc=\xcd\xcc\xcc>\x00\x00\x00?'
tim12dermatologisthigh-122.0839,37.3861b'\xcd\xcc\xcc>\xcd\xcc\xcc>\x00\x00\x00?'
taimur15CEOlow-122.0839,37.3861b'\x9a\x99\x19?\xcd\xcc\xcc=\x00\x00\x00?'
joe35dentistmedium-122.0839,37.3861b'fff?fff?\xcd\xcc\xcc='
- - -## Hash or JSON - how to choose? - -Both storage options offer a variety of features and tradeoffs. Below, you will work through a dummy dataset to learn when and how to use both data types. - -### Working with hashes - -Hashes in Redis are simple collections of field-value pairs. Think of it like a mutable, single-level dictionary that contains multiple "rows": - -```python -{ - "model": "Deimos", - "brand": "Ergonom", - "type": "Enduro bikes", - "price": 4972, -} -``` - -Hashes are best suited for use cases with the following characteristics: - -- Performance (speed) and storage space (memory consumption) are top concerns. -- Data can be easily normalized and modeled as a single-level dictionary. - -> Hashes are typically the default recommendation. - -```python -# define the hash index schema -hash_schema = { - "index": { - "name": "user-hash", - "prefix": "user-hash-docs", - "storage_type": "hash", # default setting -- HASH - }, - "fields": [ - {"name": "user", "type": "tag"}, - {"name": "credit_score", "type": "tag"}, - {"name": "job", "type": "text"}, - {"name": "age", "type": "numeric"}, - {"name": "office_location", "type": "geo"}, - { - "name": "user_embedding", - "type": "vector", - "attrs": { - "dims": 3, - "distance_metric": "cosine", - "algorithm": "flat", - "datatype": "float32" - } - } - ], -} -``` - -```python -# construct a search index from the hash schema -hindex = SearchIndex.from_dict(hash_schema) - -# connect to local redis instance -hindex.connect("redis://localhost:6379") - -# create the index (no data yet) -hindex.create(overwrite=True) -``` - -```python -# show the underlying storage type -hindex.storage_type - - -``` - -#### Vectors as byte strings - -One nuance when working with hashes in Redis is that all vectorized data must be passed as a byte string (for efficient storage, indexing, and processing). An example of this can be seen below: - - -```python -# show a single entry from the data that will be loaded -data[0] - - {'user': 'john', - 'age': 18, - 'job': 'engineer', - 'credit_score': 'high', - 'office_location': '-122.4194,37.7749', - 'user_embedding': b'\xcd\xcc\xcc=\xcd\xcc\xcc=\x00\x00\x00?'} -``` - -```python -# load hash data -keys = hindex.load(data) -``` - -```python -$ rvl stats -i user-hash - - Statistics: - ╭─────────────────────────────┬─────────────╮ - │ Stat Key │ Value │ - ├─────────────────────────────┼─────────────┤ - │ num_docs │ 7 │ - │ num_terms │ 6 │ - │ max_doc_id │ 7 │ - │ num_records │ 44 │ - │ percent_indexed │ 1 │ - │ hash_indexing_failures │ 0 │ - │ number_of_uses │ 1 │ - │ bytes_per_record_avg │ 3.40909 │ - │ doc_table_size_mb │ 0.000767708 │ - │ inverted_sz_mb │ 0.000143051 │ - │ key_table_size_mb │ 0.000248909 │ - │ offset_bits_per_record_avg │ 8 │ - │ offset_vectors_sz_mb │ 8.58307e-06 │ - │ offsets_per_term_avg │ 0.204545 │ - │ records_per_doc_avg │ 6.28571 │ - │ sortable_values_size_mb │ 0 │ - │ total_indexing_time │ 0.587 │ - │ total_inverted_index_blocks │ 18 │ - │ vector_index_sz_mb │ 0.0202332 │ - ╰─────────────────────────────┴─────────────╯ -``` - -#### Performing queries - -Once the index is created and data is loaded into the right format, you can run queries against the index: - -```python -from redisvl.query import VectorQuery -from redisvl.query.filter import Tag, Text, Num - -t = (Tag("credit_score") == "high") & (Text("job") % "enginee*") & (Num("age") > 17) - -v = VectorQuery([0.1, 0.1, 0.5], - "user_embedding", - return_fields=["user", "credit_score", "age", "job", "office_location"], - filter_expression=t) - - -results = hindex.query(v) -result_print(results) - -``` - -
vector_distanceusercredit_scoreagejoboffice_location
0johnhigh18engineer-122.4194,37.7749
0.109129190445tylerhigh100engineer-122.0839,37.3861
- -```python -# clean up -hindex.delete() -``` - -### Working with JSON - -Redis also supports native **JSON** objects. These can be multi-level (nested) objects, with full [JSONPath]({{< relref "/develop/data-types/json/" >}}path/) support for retrieving and updating sub-elements: - -```python -{ - "name": "bike", - "metadata": { - "model": "Deimos", - "brand": "Ergonom", - "type": "Enduro bikes", - "price": 4972, - } -} -``` - -JSON is best suited for use cases with the following characteristics: - -- Ease of use and data model flexibility are top concerns. -- Application data is already native JSON. -- Replacing another document storage/database solution. - -#### Full JSON Path support - -Because Redis enables full JSONPath support, when creating an index schema, elements need to be indexed and selected by their path with the desired `name` and `path` that points to where the data is located within the objects. - -{{< note >}} -By default, RedisVL will assume the path as `$.{name}` if not provided in JSON fields schema. -{{< /note >}} - -```python -# define the json index schema -json_schema = { - "index": { - "name": "user-json", - "prefix": "user-json-docs", - "storage_type": "json", # JSON storage type - }, - "fields": [ - {"name": "user", "type": "tag"}, - {"name": "credit_score", "type": "tag"}, - {"name": "job", "type": "text"}, - {"name": "age", "type": "numeric"}, - {"name": "office_location", "type": "geo"}, - { - "name": "user_embedding", - "type": "vector", - "attrs": { - "dims": 3, - "distance_metric": "cosine", - "algorithm": "flat", - "datatype": "float32" - } - } - ], -} -``` - -```python -# construct a search index from the JSON schema -jindex = SearchIndex.from_dict(json_schema) - -# connect to a local redis instance -jindex.connect("redis://localhost:6379") - -# create the index (no data yet) -jindex.create(overwrite=True) -``` - -```python -# note the multiple indices in the same database -$ rvl index listall - - 20:23:08 [RedisVL] INFO Indices: - 20:23:08 [RedisVL] INFO 1. user-json - -#### Vectors as float arrays - -Vectorized data stored in JSON must be stored as a pure array (e.g., a Python list) of floats. Modify your sample data to account for this below: - -```python -import numpy as np - -json_data = data.copy() - -for d in json_data: - d['user_embedding'] = buffer_to_array(d['user_embedding'], dtype=np.float32) -``` - -```python -# inspect a single JSON record -json_data[0] -``` - - {'user': 'john', - 'age': 18, - 'job': 'engineer', - 'credit_score': 'high', - 'office_location': '-122.4194,37.7749', - 'user_embedding': [0.10000000149011612, 0.10000000149011612, 0.5]} - - -```python -keys = jindex.load(json_data) -``` - -```python -# we can now run the exact same query as above -result_print(jindex.query(v)) -``` - -
vector_distanceusercredit_scoreagejoboffice_location
0johnhigh18engineer-122.4194,37.7749
0.109129190445tylerhigh100engineer-122.0839,37.3861
- -## Cleanup - -```python -jindex.delete() -```