Skip to content

Commit

Permalink
docs: update docs with information on azure blob storage
Browse files Browse the repository at this point in the history
  • Loading branch information
Victor Benichoux committed Dec 9, 2021
1 parent d45aa20 commit f9e4d61
Show file tree
Hide file tree
Showing 4 changed files with 15 additions and 2 deletions.
2 changes: 1 addition & 1 deletion docs/assets/environment.md
Expand Up @@ -10,7 +10,7 @@ The parameters necessary to instantiate an `AssetsManager` can all be read from

| Environment variable | Default value | Parameter | Notes |
| --- | --- | --- | --- |
| `MODELKIT_STORAGE_PROVIDER` | `gcs` | `storage_provider` | `gcs` (default), `s3` or `local` |
| `MODELKIT_STORAGE_PROVIDER` | `gcs` | `storage_provider` | `gcs` (default), `s3`, `az` or `local` |
| `MODELKIT_STORAGE_BUCKET` | None | `bucket` | Bucket in which data is stored |
| `MODELKIT_STORAGE_PREFIX` | `modelkit-assets` | `prefix` | Objects prefix |
| `MODELKIT_STORAGE_TIMEOUT_S` | `300` | `timeout_s` | max time when retrying storage downloads |
Expand Down
12 changes: 12 additions & 0 deletions docs/assets/storage_provider.md
Expand Up @@ -65,6 +65,18 @@ By default, the GCS client use the credentials setup up on the machine.

If `GOOGLE_APPLICATION_CREDENTIALS` is provided, it should point to a local JSON service account file, which we use to instantiate the client with `google.cloud.storage.Client.from_service_account_json`

### Using Azure blob storage

Use `STORAGE_PROVIDER=az` to connect to Azure blob storage.

We use [azure-storage-blobl](https://docs.microsoft.com/en-us/azure/storage/blobs/storage-quickstart-blobs-python) under the hood.

The client is created by passing the authentication information to `BlobServiceClient.from_connection_string`:

| Environment variable | Note |
| ----------------------- | ----------------------- |
| `AZURE_STORAGE_CONNECTION_STRING` | azure connection string |


### `local` mode

Expand Down
2 changes: 1 addition & 1 deletion docs/assets/store_organization.md
Expand Up @@ -12,7 +12,7 @@ Remote assets are stored in object stores, referenced as:

In this "path":

- `provider` is `s3` or `gcs` or `file` depending on the storage driver (value of `MODELKIT_STORAGE_PROVIDER`)
- `provider` is `s3`, `azfs` or `gcs` or `file` depending on the storage driver (value of `MODELKIT_STORAGE_PROVIDER`)
- `bucket` is the remote container name (`MODELKIT_STORAGE_BUCKET`)

The rest of the "path" is the remote object's name and consists of
Expand Down
1 change: 1 addition & 0 deletions docs/configuration.md
Expand Up @@ -31,6 +31,7 @@ These variables are necessary to set a remote storage from which to retrieve ass
pointing to a service account credentials JSON file (this is not necessary on dev
machines)
- for `MODELKIT_STORAGE_PROVIDER=s3`, you need to instantiate `AWS_PROFILE`
- for `MODELKIT_STORAGE_PROVIDER=az`, you need to instantiate `AZURE_STORAGE_CONNECTION_STRING` with a connection string

### Assets versioning related environment variable

Expand Down

0 comments on commit f9e4d61

Please sign in to comment.