Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions platform-api/api/destinations/motherduck.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,12 @@ import MotherDuckPrerequisites from '/snippets/general-shared-text/motherduck.md

To create or change a MotherDuck destination connector, see the following examples.

import MotherDuckSDK from '/snippets/destination_connectors/motherduck_sdk.mdx';
import MotherDuckAPIRESTCreate from '/snippets/destination_connectors/motherduck_rest_create.mdx';
import MotherDuckAPIRESTChange from '/snippets/destination_connectors/motherduck_rest_change.mdx';

<CodeGroup>
<MotherDuckSDK />
<MotherDuckAPIRESTCreate />
<MotherDuckAPIRESTChange />
</CodeGroup>
Expand Down
42 changes: 21 additions & 21 deletions platform-api/api/destinations/overview.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -18,25 +18,25 @@ To use the [Unstructured Platform Workflow Endpoint](/platform-api/api/overview)
To create or update a destination connector, you must also provide settings that are specific to that connector.
For the list of specific settings, see:

- [Astra DB](/platform-api/api/destinations/astradb) (`astradb`)
- [Azure AI Search](/platform-api/api/destinations/azure-ai-search) (`azure_ai_search`)
- [Couchbase](/platform-api/api/destinations/couchbase) (`couchbase`)
- [Databricks Volumes](/platform-api/api/destinations/databricks-volumes) (`databricks_volumes`)
- [Delta Tables in Amazon S3](/platform-api/api/destinations/delta-table) (`delta_table`)
- [Delta Tables in Databricks](/platform-api/api/destinations/databricks-delta-table) (`databricks_volume_delta_tables`)
- [Elasticsearch](/platform-api/api/destinations/elasticsearch) (`elasticsearch`)
- [Google Cloud Storage](/platform-api/api/destinations/google-cloud) (`gcs`)
- [Kafka](/platform-api/api/destinations/kafka) (`kafka-cloud`)
- [Milvus](/platform-api/api/destinations/milvus) (`milvus`)
- [MongoDB](/platform-api/api/destinations/mongodb) (`mongodb`)
- [MotherDuck](/platform-api/api/destinations/motherduck) (`motherduck`)
- [Neo4j](/platform-api/api/destinations/neo4j) (`neo4j`)
- [OneDrive](/platform-api/api/destinations/onedrive) (`onedrive`)
- [Pinecone](/platform-api/api/destinations/pinecone) (`pinecone`)
- [PostgreSQL](/platform-api/api/destinations/postgresql) (`postgres`)
- [Qdrant](/platform-api/api/destinations/qdrant) (`qdrant-cloud`)
- [Redis](/platform-api/api/destinations/redis) (`redis`)
- [Snowflake](/platform-api/api/destinations/snowflake) (`snowflake`)
- [S3](/platform-api/api/destinations/s3) (`s3`)
- [Weaviate](/platform-api/api/destinations/weaviate) (`weaviate`)
- [Astra DB](/platform-api/api/destinations/astradb) (`ASTRADB` for the Python SDK or `astradb` for `curl` or Postman)
- [Azure AI Search](/platform-api/api/destinations/azure-ai-search) (`AZURE_AI_SEARCH` for the Python SDK or `azure_ai_search` for `curl` or Postman)
- [Couchbase](/platform-api/api/destinations/couchbase) (`COUCHBASE` for the Python SDK or `couchbase` for `curl` or Postman)
- [Databricks Volumes](/platform-api/api/destinations/databricks-volumes) (`DATABRICKS_VOLUMES` for the Python SDK or `databricks_volumes` for `curl` or Postman)
- [Delta Tables in Amazon S3](/platform-api/api/destinations/delta-table) (`DELTA_TABLE` for the Python SDK or `delta_table` for `curl` or Postman)
- [Delta Tables in Databricks](/platform-api/api/destinations/databricks-delta-table) (`DATABRICKS_VOLUME_DELTA_TABLES` for the Python SDK or `databricks_volume_delta_tables` for `curl` or Postman)
- [Elasticsearch](/platform-api/api/destinations/elasticsearch) (`ELASTICSEARCH` for the Python SDK or `elasticsearch` for `curl` or Postman)
- [Google Cloud Storage](/platform-api/api/destinations/google-cloud) (`GCS` for the Python SDK or `gcs` for `curl` or Postman)
- [Kafka](/platform-api/api/destinations/kafka) (`KAFKA_CLOUD` for the Python SDK or `kafka-cloud` for `curl` or Postman)
- [Milvus](/platform-api/api/destinations/milvus) (`MILVUS` for the Python SDK or `milvus` for `curl` or Postman)
- [MongoDB](/platform-api/api/destinations/mongodb) (`MONGODB` for the Python SDK or `mongodb` for `curl` or Postman)
- [MotherDuck](/platform-api/api/destinations/motherduck) (`MOTHERDUCK` for the Python SDK or `motherduck` for `curl` or Postman)
- [Neo4j](/platform-api/api/destinations/neo4j) (`NEO4J` for the Python SDK or `neo4j` for `curl` or Postman)
- [OneDrive](/platform-api/api/destinations/onedrive) (`ONEDRIVE` for the Python SDK or `onedrive` for `curl` or Postman)
- [Pinecone](/platform-api/api/destinations/pinecone) (`PINECONE` for the Python SDK or `pinecone` for `curl` or Postman)
- [PostgreSQL](/platform-api/api/destinations/postgresql) (`POSTGRES` for the Python SDK or `postgres` for `curl` or Postman)
- [Qdrant](/platform-api/api/destinations/qdrant) (`QDRANT_CLOUD` for the Python SDK or `qdrant-cloud` for `curl` or Postman)
- [Redis](/platform-api/api/destinations/redis) (`REDIS` for the Python SDK or `redis` for `curl` or Postman)
- [Snowflake](/platform-api/api/destinations/snowflake) (`SNOWFLAKE` for the Python SDK or `snowflake` for `curl` or Postman)
- [S3](/platform-api/api/destinations/s3) (`S3` for the Python SDK or `s3` for `curl` or Postman)
- [Weaviate](/platform-api/api/destinations/weaviate) (`WEAVIATE` for the Python SDK or `weaviate` for `curl` or Postman)

36 changes: 18 additions & 18 deletions platform-api/api/sources/overview.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -18,23 +18,23 @@ To use the [Unstructured Platform Workflow Endpoint](/platform-api/api/overview)
To create or update a source connector, you must also provide settings that are specific to that connector.
For the list of specific settings, see:

- [Azure](/platform-api/api/sources/azure-blob-storage) (`azure`)
- [Box](/platform-api/api/sources/box) (`box`)
- [Confluence](/platform-api/api/sources/confluence) (`confluence`)
- [Couchbase](/platform-api/api/sources/couchbase) (`couchbase`)
- [Databricks Volumes](/platform-api/api/sources/databricks-volumes) (`databricks_volumes`)
- [Dropbox](/platform-api/api/sources/dropbox) (`dropbox`)
- [Elasticsearch](/platform-api/api/sources/elasticsearch) (`elasticsearch`)
- [Google Cloud Storage](/platform-api/api/sources/google-cloud) (`gcs`)
- [Google Drive](/platform-api/api/sources/google-drive) (`google_drive`)
- [Kafka](/platform-api/api/sources/kafka) (`kafka-cloud`)
- [MongoDB](/platform-api/api/sources/mongodb) (`mongodb`)
- [OneDrive](/platform-api/api/sources/onedrive) (`onedrive`)
- [Outlook](/platform-api/api/sources/outlook) (`outlook`)
- [PostgreSQL](/platform-api/api/sources/postgresql) (`postgres`)
- [S3](/platform-api/api/sources/s3) (`s3`)
- [Salesforce](/platform-api/api/sources/salesforce) (`salesforce`)
- [SharePoint](/platform-api/api/sources/sharepoint) (`sharepoint`)
- [Snowflake](/platform-api/api/sources/snowflake) (`snowflake`)
- [Azure](/platform-api/api/sources/azure-blob-storage) (`AZURE` for the Python SDK or `azure` for `curl` and Postman)
- [Box](/platform-api/api/sources/box) (`BOX` for the Python SDK or `box` for `curl` and Postman)
- [Confluence](/platform-api/api/sources/confluence) (`CONFLUENCE` for the Python SDK or `confluence` for `curl` and Postman)
- [Couchbase](/platform-api/api/sources/couchbase) (`COUCHBASE` for the Python SDK or `couchbase` for `curl` and Postman)
- [Databricks Volumes](/platform-api/api/sources/databricks-volumes) (`DATABRICKS_VOLUMES` for the Python SDK or `databricks_volumes` for `curl` and Postman)
- [Dropbox](/platform-api/api/sources/dropbox) (`DROPBOX` for the Python SDK or `dropbox` for `curl` and Postman)
- [Elasticsearch](/platform-api/api/sources/elasticsearch) (`ELASTICSEARCH` for the Python SDK or `elasticsearch` for `curl` and Postman)
- [Google Cloud Storage](/platform-api/api/sources/google-cloud) (`GCS` for the Python SDK or `gcs` for `curl` and Postman)
- [Google Drive](/platform-api/api/sources/google-drive) (`GOOGLE_DRIVE` for the Python SDK or `google_drive` for `curl` and Postman)
- [Kafka](/platform-api/api/sources/kafka) (`KAFKA_CLOUD` for the Python SDK or `kafka-cloud` for `curl` and Postman)
- [MongoDB](/platform-api/api/sources/mongodb) (`MONGODB` for the Python SDK or `mongodb` for `curl` and Postman)
- [OneDrive](/platform-api/api/sources/onedrive) (`ONEDRIVE` for the Python SDK or `onedrive` for `curl` and Postman)
- [Outlook](/platform-api/api/sources/outlook) (`OUTLOOK` for the Python SDK or `outlook` for `curl` and Postman)
- [PostgreSQL](/platform-api/api/sources/postgresql) (`POSTGRES` for the Python SDK or `postgres` for `curl` and Postman)
- [S3](/platform-api/api/sources/s3) (`S3` for the Python SDK or `s3` for `curl` and Postman)
- [Salesforce](/platform-api/api/sources/salesforce) (`SALESFORCE` for the Python SDK or `salesforce` for `curl` and Postman)
- [SharePoint](/platform-api/api/sources/sharepoint) (`SHAREPOINT` for the Python SDK or `sharepoint` for `curl` and Postman)
- [Snowflake](/platform-api/api/sources/snowflake) (`SNOWFLAKE` for the Python SDK or `snowflake` for `curl` and Postman)


22 changes: 13 additions & 9 deletions snippets/destination_connectors/astradb_sdk.mdx
Original file line number Diff line number Diff line change
@@ -1,17 +1,21 @@
```python Python SDK
# ...
from unstructured_client.models.shared import <Create|Update>DestinationConnector
from unstructured_client.models.shared import (
<Create|Update>DestinationConnector,
DestinationConnectorType,
AstraDBConnectorConfigInput
)
# ...
destination_connector = <Create|Update>DestinationConnector(
name="<name>", # Create only.
type="astradb", # Create only.
config={
"token": "<token>",
"api_endpoint": "<api-endpoint>",
"collection_name": "<collection-name>",
"keyspace": "<keyspace>",
"batch_size": <batch-size>
}
type=DestinationConnectorType.ASTRADB, # Create only.
config=AstraDBConnectorConfigInput(
token="<token>",
api_endpoint="<api-endpoint>",
collection_name="<collection-name>",
keyspace="<keyspace>",
batch_size=<batch-size>
)
)
# ...
```
18 changes: 11 additions & 7 deletions snippets/destination_connectors/azure_ai_search_sdk.mdx
Original file line number Diff line number Diff line change
@@ -1,15 +1,19 @@
```python Python SDK
# ...
from unstructured_client.models.shared import <Create|Update>DestinationConnector
from unstructured_client.models.shared import (
<Create|Update>DestinationConnector,
DestinationConnectorType,
AzureAISearchConnectorConfigInput
)
# ...
destination_connector = <Create|Update>DestinationConnector(
name="<name>", # Create only.
type="azure_ai_search", # Create only.
config={
"endpoint": "<endpoint>",
"index": "<index>",
"azure_ai_search_key": "<azure-ai-search-key>"
}
type=DestinationConnectorType.AZURE_AI_SEARCH, # Create only.
config=AzureAISearchConnectorConfigInput(
endpoint="<endpoint>",
index="<index>",
key="<azure-ai-search-key>"
)
)
# ...
```
3 changes: 1 addition & 2 deletions snippets/destination_connectors/couchbase_rest_change.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,7 @@ curl --request 'PUT' --location \
"scope": "<scope>",
"collection": "<collection>",
"password": "<password>",
"batch_size": <batch-size>,
"collection_id": "<collection-id>"
"batch_size": <batch-size>
}
}'
```
3 changes: 1 addition & 2 deletions snippets/destination_connectors/couchbase_rest_create.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,7 @@ curl --request 'POST' --location \
"scope": "<scope>",
"collection": "<collection>",
"password": "<password>",
"batch_size": <batch-size>,
"collection_id": "<collection-id>"
"batch_size": <batch-size>
}
}'
```
27 changes: 15 additions & 12 deletions snippets/destination_connectors/couchbase_sdk.mdx
Original file line number Diff line number Diff line change
@@ -1,20 +1,23 @@
```python Python SDK
# ...
from unstructured_client.models.shared import <Create|Update>DestinationConnector
from unstructured_client.models.shared import (
<Create|Update>DestinationConnector,
DestinationConnectorType,
CouchbaseDestinationConnectorConfigInput
)
# ...
destination_connector = <Create|Update>DestinationConnector(
name="<name>", # Create only.
type="couchbase", # Create only.
config={
"username": "<username>",
"bucket": "<bucket>",
"connection_string": "<connection-string>",
"scope": "<scope>",
"collection": "<collection>",
"password": "<password>",
"batch_size": <batch-size>,
"collection_id": "<collection-id>"
}
type=estinationConnectorType.COUCHBASE, # Create only.
config=CouchbaseDestinationConnectorConfigInput(
username="<username>",
bucket="<bucket>",
connection_string="<connection-string>",
scope="<scope>",
collection="<collection>",
password="<password>",
batch_size=<batch-size>
)
)
# ...
```
34 changes: 19 additions & 15 deletions snippets/destination_connectors/databricks_delta_table_sdk.mdx
Original file line number Diff line number Diff line change
@@ -1,23 +1,27 @@
```python Python SDK
# ...
from unstructured_client.models.shared import <Create|Update>DestinationConnector
from unstructured_client.models.shared import (
<Create|Update>DestinationConnector,
DestinationConnectorType,
DatabricksVDTDestinationConnectorConfigInput
)
# ...
destination_connector = <Create|Update>DestinationConnector(
name="<name>", # Create only.
type="databricks_volume_delta_tables", # Create only.
config={
"server_hostname": "<server-hostname>",
"http_path": "<http-path>",
"token": "<token>",
"client_id": "<client-id>",
"client_secret": "<client-secret>",
"volume": "<volume>",
"catalog": "<catalog>",
"volume_path": "<volume-path>",
"schema": "<schema>",
"database": "<database>",
"table_name": "<table-name>"
}
type=DestinationConnectorType.DATABRICKS_VOLUME_DELTA_TABLES, # Create only.
config=DatabricksVDTDestinationConnectorConfigInput(
server_hostname="<server-hostname>",
http_path="<http-path>",
token="<token>",
client_id="<client-id>",
client_secret="<client-secret>",
volume="<volume>",
catalog="<catalog>",
volume_path="<volume-path>",
schema="<schema>",
database="<database>",
table_name="<table-name>"
)
)
# ...
```
26 changes: 15 additions & 11 deletions snippets/destination_connectors/databricks_volumes_sdk.mdx
Original file line number Diff line number Diff line change
@@ -1,19 +1,23 @@
```python Python SDK
# ...
from unstructured_client.models.shared import <Create|Update>DestinationConnector
from unstructured_client.models.shared import (
<Create|Update>DestinationConnector,
DestinationConnectorType,
DatabricksVolumesConnectorConfigInput
)
# ...
destination_connector = <Create|Update>DestinationConnector(
name="<name>", # Create only.
type="databricks_volumes", # Create only.
config={
"host": "<host>",
"catalog": "<catalog>",
"schema": "<schema>",
"volume": "<volume>",
"volume_path": "<volume-path>",
"client_secret": "<client-secret>",
"client_id": "<client-id>"
}
type=DestinationConnectorType.DATABRICKS_VOLUMES, # Create only.
config=DatabricksVolumesConnectorConfigInput(
host="<host>",
catalog="<catalog>",
schema="<schema>",
volume="<volume>",
volume_path="<volume-path>",
client_secret="<client-secret>",
client_id="<client-id>"
)
)
# ...
```
20 changes: 12 additions & 8 deletions snippets/destination_connectors/delta_table_sdk.mdx
Original file line number Diff line number Diff line change
@@ -1,16 +1,20 @@
```python Python SDK
# ...
from unstructured_client.models.shared import <Create|Update>DestinationConnector
from unstructured_client.models.shared import (
<Create|Update>DestinationConnector,
DestinationConnectorType,
DeltaTableConnectorConfigInput
)
# ...
destination_connector = <Create|Update>DestinationConnector(
name="<name>", # Create only.
type="delta_table", # Create only.
config={
"aws_region": "<aws-region>",
"table_uri": "<table-uri>",
"aws_access_key_id": "<aws-access-key-id>",
"aws_secret_access_key": "<aws-secret-access-key>"
}
type=DestinationConnectorType.DELTA_TABLE, # Create only.
config=DeltaTableConnectorConfigInput(
aws_region="<aws-region>",
table_uri="<table-uri>",
aws_access_key_id="<aws-access-key-id>",
aws_secret_access_key="<aws-secret-access-key>"
)
)
# ...
```
18 changes: 11 additions & 7 deletions snippets/destination_connectors/elasticsearch_sdk.mdx
Original file line number Diff line number Diff line change
@@ -1,15 +1,19 @@
```python Python SDK
# ...
from unstructured_client.models.shared import <Create|Update>DestinationConnector
from unstructured_client.models.shared import (
<Create|Update>DestinationConnector,
DestinationConnectorType,
ElasticsearchConnectorConfigInput
)
# ...
destination_connector = <Create|Update>DestinationConnector(
name="<name>", # Create only.
type="elasticsearch", # Create only.
config={
"hosts": ["<host-url>"],
"es_api_key": "<es-api-key>",
"index_name": "<index-name>"
}
type=DestinationConnectorType.ELASTICSEARCH, # Create only.
config=ElasticsearchConnectorConfigInput(
hosts=["<host-url>"],
es_api_key="<es-api-key>",
index_name="<index-name>"
)
)
# ...
```
16 changes: 10 additions & 6 deletions snippets/destination_connectors/gcs_sdk.mdx
Original file line number Diff line number Diff line change
@@ -1,14 +1,18 @@
```python Python SDK
# ...
from unstructured_client.models.shared import <Create|Update>DestinationConnector
from unstructured_client.models.shared import (
<Create|Update>DestinationConnector,
DestinationConnectorType,
GCSDestinationConnectorConfigInput
)
# ...
destination_connector = <Create|Update>DestinationConnector(
name="<name>", # Create only.
type="gcs", # Create only.
config={
"remote_url": "<remote-url>",
"service_account_key": "<service-account-key>"
}
type=DestinationConnectorType.GCS, # Create only.
config=GCSDestinationConnectorConfigInput(
remote_url="<remote-url>",
service_account_key="<service-account-key>"
)
)
# ...
```
Loading