Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Azure SAS signatures ending in "=" don't work #1003

Closed
damiondoesthings opened this issue Dec 8, 2022 · 2 comments · Fixed by #1007
Closed

Azure SAS signatures ending in "=" don't work #1003

damiondoesthings opened this issue Dec 8, 2022 · 2 comments · Fixed by #1007
Labels
bug Something isn't working

Comments

@damiondoesthings
Copy link
Contributor

Environment

Delta-rs version: 0.6.4

Binding: Python

Environment:

  • Cloud provider: Azure
  • OS: MacOs
  • Other:

Bug

What happened: SAS tokens with signatures ending in "=" passed via storage_options get truncated and won't work.

What you expected to happen: SAS token authenticates all requests to Azure

How to reproduce it:

DeltaTable(
    "abfs://<container_name>",
    storage_options={
        "AZURE_STORAGE_ACCOUNT_NAME": "<account_name>",
        "sas_key": '?sv=2019-07-07&sr=c&sig=ain2DZplwGRn0RHuxrhJSwXl2bkCe0gyVMMJKTMAtFk=&st=2022-12-01T00:00:00Z&se=2022-12-02T00:00:00Z&sp=rl',
    },
)

Results in

PyDeltaTableError: Failed to load checkpoint: Failed to read checkpoint content: Generic MicrosoftAzure error: Error performing get request _delta_log/_last_checkpoint: response error "<?xml version="1.0" encoding="utf-8"?><Error><Code>AuthenticationFailed</Code><Message>Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
Time:2022-12-08T14:26:32.3390349Z</Message><AuthenticationErrorDetail>Signature fields not well formed.</AuthenticationErrorDetail></Error>", after 0 retries: HTTP status client error (403 Forbidden) for url (https://<account_name>.blob.core.windows.net/<container_name>/_delta_log/_last_checkpoint?sv=2019-07-07&sr=c&sig=ain2DZplwGRn0RHuxrhJSwXl2bkCe0gyVMMJKTMAtFk&st=2022-12-01T00:00:00Z&se=2022-12-02T00:00:00Z&sp=rl)

Note that the signature got truncated:
original sig: sig=ain2DZplwGRn0RHuxrhJSwXl2bkCe0gyVMMJKTMAtFk=
request url: sig=ain2DZplwGRn0RHuxrhJSwXl2bkCe0gyVMMJKTMAtFk

More details:
This might originate from the split function here:

fn split_sas(sas: &str) -> Result<Vec<(String, String)>, BuilderError> {

@damiondoesthings damiondoesthings added the bug Something isn't working label Dec 8, 2022
@roeap
Copy link
Collaborator

roeap commented Dec 8, 2022

thanks @damiondoesthings for reporting, and you already identified the root cause :). Good news is, there is an easy fix, we just have to replace the split a few lines below with split_once - any chance you are interested in opening a PR? If not I can probably get to it this weekend.

@damiondoesthings
Copy link
Contributor Author

Thanks for confirming @roeap. I think I should be able to make a PR this weekend.

roeap pushed a commit that referenced this issue Dec 12, 2022
# Description
This fixes the truncation of Azure SAS signatures ending in "=", i.e.
when using non-url-encoded SAS tokens.

# Related Issue(s)
<!---
For example:

- closes #1003
--->
closes #1003

Co-authored-by: Damion Werner <damion.werner@arisense.live>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants