pyiceberg azure should support token

[djouallah]

Feature Request / Improvement

this works in delta

from deltalake import write_deltalake
import duckdb
duckdb.sql(f""" CREATE or replace SECRET onelake ( TYPE AZURE, PROVIDER ACCESS_TOKEN, ACCESS_TOKEN '{os.getenv('azure_storage_token')}')   """)
write_deltalake(
        f"abfss://iceberg@onelake.dfs.fabric.microsoft.com/external.Lakehouse/Files/xxxx",
            df,
            mode ='overwrite'
    )

this should works too in pyiceberg

from pyiceberg.catalog.sql import SqlCatalog
catalog = SqlCatalog(
      "default",
      **{
          "uri": "sqlite:///:memory:",
          "adls.account-name"         : 'onelake' ,
          "token"  : token ,
          "legacy-current-snapshot-id": True ,
          "warehouse"                 : "abfss://iceberg@onelake.dfs.fabric.microsoft.com/external.Lakehouse/Files"
      },
                        )
db = "aemo"
catalog.create_namespace_if_not_exists(db)
calendar = catalog.create_table_if_not_exists(db+"."+"calendar",schema=df.schema)
calendar.overwrite(df)

[kevinjqliu]

Similar to my comment here #2326 (comment)

can you use

    "adls.credential": credential,

with

credential = DefaultAzureCredential()

[kevinjqliu]

let me see if fsspec's AzureBlobFileSystem also supports token

[kevinjqliu]

looks like the best way is to wrap the token in a class like so.

Azure/azure-sdk-for-python#9075 (comment)

[kevinjqliu]

maybe we should allow setting token...
i dont see it as an option on the java side
https://github.com/apache/iceberg/blob/54f8e58fc845cb82af70750d748ac7b3dee25365/azure/src/main/java/org/apache/iceberg/azure/AzureProperties.java#L126-L137

[kevinjqliu]

Duckdb azure allows setting access token (which is what is used in the example)
https://github.com/duckdb/duckdb-azure/blob/main/README.md#other-authentication-methods
(note this isnt mentioned in https://duckdb.org/docs/stable/core_extensions/azure.html#authentication)

[kevinjqliu]

i created apache/iceberg#13818 to add adls.token on the java implementation.

I think it makes sense to add it for pyiceberg too

[djouallah]

credential = DefaultAzureCredential() does not works inside Fabric notebook, we had the same problem with delta_rs till they added support for adls.token

[kevinjqliu]

Heres a workaround to make Entra/AAD token work like DefaultAzureCredential inside Fabric notebook env

from azure.core.credentials import AccessToken, TokenCredential
from datetime import datetime, timedelta

class UserTokenCredential(TokenCredential):
    def __init__(self, token: str, expires_on: datetime = None):
        self._token = token
        self._expires_on = expires_on or (datetime.utcnow() + timedelta(hours=1))

    def get_token(self, *scopes, **kwargs):
        return AccessToken(self._token, int(self._expires_on.timestamp()))

token = notebookutils.credentials.getToken('storage')
token_credential = UserTokenCredential(token)

# Example: passing to a BlobServiceClient
from azure.storage.blob import BlobServiceClient
client = BlobServiceClient(account_url="https://<account>.blob.core.windows.net", credential=token_credential)