Docs: Improve documentation for Azure CLI authentication

[kvedes]

Description

It is unclear how to configure deltalake to authenticate using Azure CLI credentials. It seems like it should be supported (rust documentation). However, the delta lake documentation doesn't show how it is done. It only shows key based examples for AWS.

All the other options in the rust documentation are key-value based, but the UseCLI doesn't seem to be, so what do you put, when you are supposed to pass a dictionary?

I have tried to figure it out myself, but without any success:

> az login
> python
>>> from deltalake import DeltaTable
>>> dt = DeltaTable(abfs_path, storage_options={
    "AZURE_TENANT_ID": tenant_id,
    "AZURE_USE_AZURE_CLI": "true"
})
PyDeltaTableError: Failed to read delta log object: Generic MicrosoftAzure error: At least one authorization option must be specified

Use Case
For data scientists/data analysts who are not knowledgeable about authentication it is easy to sign into the CLI but hard to manually featch tokens or manage service principal credentials for storage_options.

Related Issue(s)
This might be related, if the mentioned bug is also affecting CLI auth: #1153

[roeap]

Hi @kvedes,

Our docs link to the latest config for object store, but the current python release pre-dates the azure cli support. That said, we are looking to cut a new release soon, which will then support azure cli auth.

Your configuration is already correct, personally I would set if up with keys a bit more pleasing to the eye, but thats just cosmetics 😆.

dt = DeltaTable(abfs_path, storage_options={
    "tenant_id": "<tenant_id>",
    "use_azure_cli": "true"
})

[kvedes]

Hi @roeap

Thanks for your swift reply. Will the new release also enable SAS key auth? - It's a little unclear which ones will be supported and which ones I'm just not configuring correctly:

key = "sp=r&st=2023-03-08T08:15:31Z&se=2023-03-08T16:15:31Z&spr=https&sv=2021-12-02&sr=c&sig=SECRETVALUE"
url = "https://MY_STORAGE_ACCOUNT.blob.core.windows.net/MY_CONTAINER?sp=r&st=2023-03-08T08:15:31Z&se=2023-03-08T16:15:31Z&spr=https&sv=2021-12-02&sr=c&sig=SECRETVALUE"
dt = DeltaTable(url, storage_options={
    "sas_key": key,
})

Gives error:

PyDeltaTableError: Failed to read delta log object: Generic MicrosoftAzure error: Container name must be specified

The container is already a part of the URL, so it is unclear from the error, where the container name is missing. Also there doesn't seem to be an option that lets you point at a specific table when using the SAS key, so I suppose it has to be added to the URL in some manner?

[kvedes]

So I managed to figure it out for the SAS token. Instead of using the https path given in the portal when creating the token, one has to do the following:

key = "sp=r&st=2023-03-08T08:15:31Z&se=2023-03-08T16:15:31Z&spr=https&sv=2021-12-02&sr=c&sig=SECRETVALUE"
container_name = "my_container" 
storage_account_name = "my_storage_account" 
table_name = "my_table" 
abfs_path = f"abfs://{container_name}@{storage_account_name}.dfs.core.windows.net/{table_name}/"

dt = DeltaTable(abfs_path, storage_options={
    "sas_key": key,
})

Might do a PR to add these auth options to the python docs at some point.