Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions snippets/destination_connectors/redis.sh.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,10 @@ unstructured-ingest \
--partition-endpoint $UNSTRUCTURED_API_URL \
--additional-partition-args="{\"split_pdf_page\":\"true\", \"split_pdf_allow_failed\":\"true\", \"split_pdf_concurrency_level\": 15}" \
redis \
--uri $REDIS_URI
--uri $REDIS_URI,
--no-ssl \
--key-prefix $REDIS_KEY_PREFIX \
--batch-size 100

# Use Redis connection properties:
unstructured-ingest \
Expand All @@ -30,4 +33,6 @@ unstructured-ingest \
--database 0 \
--username $REDIS_USERNAME \
--password $REDIS_PASSWORD \
--no-ssl
--no-ssl \
--key-prefix $REDIS_KEY_PREFIX \
--batch-size 100
5 changes: 4 additions & 1 deletion snippets/destination_connectors/redis.v2.py.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -57,5 +57,8 @@ if __name__ == "__main__":
username=os.getenv("REDIS_USERNAME"),
ssl=False
),
uploader_config=RedisUploaderConfig(batch_size=100)
uploader_config=RedisUploaderConfig(
batch_size=100
key_prefix=os.getenv("REDIS_KEY_PREFIX")
)
).run()
1 change: 1 addition & 0 deletions snippets/destination_connectors/redis_rest_create.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ curl --request 'POST' --location \
"database": <database>,
"ssl": <true|false>,
"batch_size": <batch-size>,
"key_prefix": "<key-prefix>",

# For URI authentication:
"uri": "<uri>"
Expand Down
1 change: 1 addition & 0 deletions snippets/destination_connectors/redis_sdk.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ with UnstructuredClient(api_key_auth=os.getenv("UNSTRUCTURED_API_KEY")) as clien
database="<database>",
ssl=<True|False>,
batch_size=<batch-size>,
key_prefix="<key-prefix>",

# For URI authentication:
uri="<uri>"
Expand Down
1 change: 1 addition & 0 deletions snippets/general-shared-text/redis-api-placeholders.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,5 @@
- `<password>` (_required_ for password authentication): The database user's password.
- `<database>`: The number (index) for the target database. The default is `0` unless otherwise specified.
- For `ssl`, `true` to enable SSL encryption for the connection. Set to `true` only if SSL is already set up and enabled for the target database. The default is `true` unless otherwise specified.
- `<key-prefix>`: A string to prepend to each element's `element_id`. This is useful for distinguishing between different data sets in the same Redis instance. The default is an empty string.
- `<batch-size>`: The maximum number of records to upload in a single batch. The default is `100` unless otherwise specified.
3 changes: 2 additions & 1 deletion snippets/general-shared-text/redis-cli-api.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -22,4 +22,5 @@ The following environment variables:
- `REDIS_PORT` - The database's port number, represented by `--port` (CLI) or `port` (Python).
- `REDIS_DATABASE` - The database number of the target database, represented by `--database` (CLI) or `database` (Python).
- `REDIS_USERNAME` - The username for the database, represented by `--username` (CLI) or `username` (Python).
- `REDIS_PASSWORD` - The user's password, represented by `--password` (CLI) or `password` (Python).
- `REDIS_PASSWORD` - The user's password, represented by `--password` (CLI) or `password` (Python).
- `REDIS_KEY_PREFIX` - A string to prepend to each element's `element_id`, represented by `--key-prefix` (CLI) or `key_prefix` (Python).
1 change: 1 addition & 0 deletions snippets/general-shared-text/redis-platform.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,5 @@ Fill in the following fields:
- **Password** (_required_ if **Password** is selected for **Authentication Method**): The database user's password.
- **Database Index**: The number (index) for the target database. The default is `0` unless otherwise specified.
- **SSL**: Check this box to enable SSL encryption for the connection. Check this box only if SSL is already set up and enabled for the target database. This box is checked by default.
- **Key Prefix**: A string to prepend to each element's `element_id`. This is useful for distinguishing between different data sets in the same Redis instance. The default is an empty string.
- **Batch Size**: The maximum number of records to upload in a single batch. The default is `100` unless otherwise specified.