The key for the expiration of an Azure token is different between Polaris and Iceberg #2766
Description
Describe the bug
Using Iceberg 1.9.2, I tried to have Polaris vend Azure credentials. The flow is good until Iceberg asserts that a map Polaris has given it contains a certain key. The information it is looking for is there but under a different String.
I traced the code and saw it make a call to my Polaris instance here but although the call seems to return successfully with a token, the key in the config map for expiration is "expiration-time". This comes from Polaris here.
However, Iceberg asserts here that the key should be "adls.sas-token-expires-at-ms.XXX" and consequently stops me from proceeding.
When I changed this code such that Polaris and Iceberg agree, I'm able to write to Azure blob storage via Spark (3.5.0).
Note that I first described this bug in the Iceberg repo but @flyrain asked me to move it here so I can create a PR.
To Reproduce
- Configure Polaris to use an Azure warehouse.
- Using Spark/Iceberg, attempt to write to that warehouse using a credential Polaris has vended.
Actual Behavior
Iceberg fails on an assertion (see link above).
Expected Behavior
Iceberg uses the token timeout that Polaris gives it.
Additional context
No response
System information
Iceberg 1.9.2
Spark 3.5.0
Polaris 1.0.1-incubating
$ uname -a
Linux thinkpad 6.8.0-85-generic #85~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Fri Sep 19 16:18:59 UTC 2 x86_64 x86_64 x86_64 GNU/Linux