Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document Snowflake Data Connector #229

Merged
merged 2 commits into from
May 6, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion spiceaidocs/docs/data-connectors/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ Currently supported Data Connectors include:
| `s3` | S3 | Alpha | Parquet | `full` | ❌ |
| `dremio` | Dremio | Alpha | Arrow Flight SQL | `full` | ❌ |
| `flightsql` | FlightSQL | Alpha | Arrow Flight SQL | `full` | ❌ |
| `snowflake` | Snowflake | Coming soon! | Arrow Flight SQL | `full` | ❌ |
| `snowflake` | Snowflake | Alpha | Arrow data via REST | `full` | ❌ |
| `bigquery` | BigQuery | Coming soon! | Arrow Flight SQL | `full` | ❌ |
| `mysql` | MySQL | Alpha | | `full` | ❌ |
| `clickhouse` | Clickhouse | Alpha | | `full` | ❌ |
Expand Down
123 changes: 123 additions & 0 deletions spiceaidocs/docs/data-connectors/snowflake.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,123 @@
---
title: 'Snowflake Data Connector'
sidebar_label: 'Snowflake Data Conector'
description: 'Snowflake Data Conector Documentation'
pagination_prev: null
---

import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';

The Snowflake Data Connector enables federated SQL queries across datasets in the [Snowflake Cloud Data Warehouse](https://www.snowflake.com/).

```yaml
datasets:
- from: snowflake:database.schema.table
name: table
params:
snowflake_warehouse: COMPUTE_WH
snowflake_role: accountadmin
```

### Parameters
- `from`: a Snowflake fully qualified table name (database.schema.table). For instance `snowflake:snowflake_sample_data.tpch_sf1.lineitem` or `snowflake:TAXI_DATA."2024".TAXI_TRIPS`
- `snowflake_warehouse`: optional, specifies the [Snowflake Warehouse](https://docs.snowflake.com/en/user-guide/warehouses-tasks) to use
- `snowflake_role`: optional, specifies the role to use for accessing Snowflake data

### Auth

The connector supports Snowflake basic authentication (username and password) that must be configured using `spice login snowflake` or using [Secrets Stores](/secret-stores). Login requires the account identifier ('orgname-account_name' format) - use [Finding the organization and account name for an account](https://docs.snowflake.com/en/user-guide/admin-account-identifier#finding-the-organization-and-account-name-for-an-account) instructions.

<img width="800" src="/img/snowflake/ui-snowsight-account-identifier.png" />

<Tabs>
<TabItem value="local" label="Local" default>
```bash
spice login snowflake -a <account-identifier> -u <username> -p <password>
```

Learn more about [File Secret Store](/secret-stores/file).
</TabItem>
<TabItem value="env" label="Env">
```bash
SPICE_SECRET_SNOWFLAKE_ACCOUNT=<account-identifier> \
SPICE_SECRET_SNOWFLAKE_USERNAME=<username> \
SPICE_SECRET_SNOWFLAKE_PASSWORD=<password> \
spice run
```

`spicepod.yaml`
```yaml
version: v1beta1
kind: Spicepod
name: spice-app

secrets:
store: env

# <...>
```

Learn more about [Env Secret Store](/secret-stores/env).
</TabItem>
<TabItem value="k8s" label="Kubernetes">
```bash
kubectl create secret generic snowflake \
--from-literal=account='<account-identifier>' \
--from-literal=username='<username>' \
--from-literal=password='<password>'
```

`spicepod.yaml`
```yaml
version: v1beta1
kind: Spicepod
name: spice-app

secrets:
store: kubernetes

# <...>
```

Learn more about [Kubernetes Secret Store](/secret-stores/kubernetes).
</TabItem>
<TabItem value="keyring" label="Keyring">
Add new keychain entry (macOS), with user and password in JSON string

```bash
security add-generic-password -l "Snowflake Secret" \
-a spiced -s spice_secret_snowflake\
-w $(echo -n '{"account":"<account-identifier>", "username": "<username>", "password": "<password>"}')
```

`spicepod.yaml`
```yaml
version: v1beta1
kind: Spicepod
name: spice-app

secrets:
store: keyring

# <...>
```

Learn more about [Keyring Secret Store](/secret-stores/keyring).
</TabItem>
</Tabs>

## Example

```yaml
datasets:
- from: snowflake:snowflake_sample_data.tpch_sf1.lineitem
name: lineitem
params:
snowflake_warehouse: COMPUTE_WH
snowflake_role: accountadmin
```

## Limitations
1. Account identifier does not support the [Legacy account locator in a region format](https://docs.snowflake.com/en/user-guide/admin-account-identifier#format-2-legacy-account-locator-in-a-region). Use [Snowflake preferred name in organization format](https://docs.snowflake.com/en/user-guide/admin-account-identifier#format-1-preferred-account-name-in-your-organization).
1. Auth: Snowflake basic authentication is only supported.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.