Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Set-AzStorageFileContent corrupting files when using OAuth #24289

Closed
billings7 opened this issue Feb 29, 2024 · 2 comments
Closed

Set-AzStorageFileContent corrupting files when using OAuth #24289

billings7 opened this issue Feb 29, 2024 · 2 comments
Assignees
@blueww
Labels
bug This issue requires a change to an existing behavior in the product in order to be resolved. customer-reported Storage

Comments

@billings7
Copy link

Description

Using Set-AzStorageFileContent to upload a file when using OAuth authentication (a storage context with -UseConnectedAccount -EnableFileBackupRequestIntent) can cause the uploaded file to be corrupt. Seems to happen consistently for files that are over 8MiB in size. If I swap to using a storage key, it works fine.

Having a look at the source, it looks like a buffer is being reused as the file is uploaded in blocks concurrently. So as the one block is being uploaded, the buffer is being overwritten with the next block from the file.

Issue script & Debug output

$StorageAccountName = '...'
$ShareName = '...'

# Path to a file >8MiB
$srcPath = 'C:\Temp\example.file'
$destPath = 'C:\Temp\example-downloaded.file'

$context = New-AzStorageContext -StorageAccountName $StorageAccountName -UseConnectedAccount -EnableFileBackupRequestIntent

Set-AzStorageFileContent `
    -Context $context `
    -ShareName $ShareName `
    -Path 'example.file' `
    -Source $srcPath `
    -PreserveSMBAttribute `
    -Force

Get-AzStorageFileContent `
    -Context $context `
    -ShareName $ShareName `
    -Path 'example.file' `
    -Destination $destPath `
    -PreserveSMBAttribute `
    -Force

$expectedHash = (Get-FileHash -Path $srcPath -Algorithm SHA256).Hash
$actualHash = (Get-FileHash -Path $destPath -Algorithm SHA256).Hash

Write-Host "Expected: $expectedHash"
Write-Host "Actual  : $actualHash"

PS> $srcPath = 'C:\Temp\example.file'
PS> $destPath = 'C:\Temp\example-downloaded.file'
PS> $context = New-AzStorageContext -StorageAccountName $StorageAccountName -UseConnectedAccount -EnableFileBackupRequestIntent
DEBUG: 18:41:01 - NewAzureStorageContext begin processing with ParameterSet 'OAuthAccount'.
DEBUG: 18:41:01 - [SilentAuthenticator] Calling SharedTokenCacheCredential.GetTokenAsync - TenantId:'00000000-0000-0000-0000-000000000000', Scopes:'https://storage.azure.com/.default', AuthorityHost:'https://login.microsoftonline.com/', UserId:'me'
DEBUG: SharedTokenCacheCredential.GetToken invoked. Scopes: [ https://storage.azure.com/.default ] ParentRequestId:
DEBUG: False MSAL 4.56.0.0 MSAL.CoreCLR .NET 8.0.1 Microsoft Windows 10.0.19045 [2024-02-29 18:41:01Z - 32eebd34-3177-4aec-ab10-f4d400a80f72] IsLegacyAdalCacheEnabled: yes
DEBUG: False MSAL 4.56.0.0 MSAL.CoreCLR .NET 8.0.1 Microsoft Windows 10.0.19045 [2024-02-29 18:41:01Z - 32eebd34-3177-4aec-ab10-f4d400a80f72] [Region discovery] Not using a regional authority.
DEBUG: False MSAL 4.56.0.0 MSAL.CoreCLR .NET 8.0.1 Microsoft Windows 10.0.19045 [2024-02-29 18:41:01Z - 32eebd34-3177-4aec-ab10-f4d400a80f72] [Region discovery] Not using a regional authority.
DEBUG: False MSAL 4.56.0.0 MSAL.CoreCLR .NET 8.0.1 Microsoft Windows 10.0.19045 [2024-02-29 18:41:01Z - 32eebd34-3177-4aec-ab10-f4d400a80f72] IsLegacyAdalCacheEnabled: yes
DEBUG: False MSAL 4.56.0.0 MSAL.CoreCLR .NET 8.0.1 Microsoft Windows 10.0.19045 [2024-02-29 18:41:01Z - 32eebd34-3177-4aec-ab10-f4d400a80f72] IsLegacyAdalCacheEnabled: yes
DEBUG: False MSAL 4.56.0.0 MSAL.CoreCLR .NET 8.0.1 Microsoft Windows 10.0.19045 [2024-02-29 18:41:01Z] Found 1 cache accounts and 0 broker accounts
DEBUG: False MSAL 4.56.0.0 MSAL.CoreCLR .NET 8.0.1 Microsoft Windows 10.0.19045 [2024-02-29 18:41:01Z] Returning 1 accounts
DEBUG: False MSAL 4.56.0.0 MSAL.CoreCLR .NET 8.0.1 Microsoft Windows 10.0.19045 [2024-02-29 18:41:01Z - 72047caa-2794-4a4b-bb03-4aa07adb953e] MSAL MSAL.CoreCLR with assembly version '4.56.0.0'. CorrelationId(72047caa-2794-4a4b-bb03-4aa07adb953e)
DEBUG: False MSAL 4.56.0.0 MSAL.CoreCLR .NET 8.0.1 Microsoft Windows 10.0.19045 [2024-02-29 18:41:01Z - 72047caa-2794-4a4b-bb03-4aa07adb953e] === AcquireTokenSilent Parameters ===
DEBUG: False MSAL 4.56.0.0 MSAL.CoreCLR .NET 8.0.1 Microsoft Windows 10.0.19045 [2024-02-29 18:41:01Z - 72047caa-2794-4a4b-bb03-4aa07adb953e] LoginHint provided: False
DEBUG: False MSAL 4.56.0.0 MSAL.CoreCLR .NET 8.0.1 Microsoft Windows 10.0.19045 [2024-02-29 18:41:01Z - 72047caa-2794-4a4b-bb03-4aa07adb953e] Account provided: True
DEBUG: False MSAL 4.56.0.0 MSAL.CoreCLR .NET 8.0.1 Microsoft Windows 10.0.19045 [2024-02-29 18:41:01Z - 72047caa-2794-4a4b-bb03-4aa07adb953e] ForceRefresh: False
DEBUG: False MSAL 4.56.0.0 MSAL.CoreCLR .NET 8.0.1 Microsoft Windows 10.0.19045 [2024-02-29 18:41:01Z - 72047caa-2794-4a4b-bb03-4aa07adb953e]
=== Request Data ===
Authority Provided? - True
Scopes - https://storage.azure.com/.default
Extra Query Params Keys (space separated) -
ApiId - AcquireTokenSilent
IsConfidentialClient - False
SendX5C - False
LoginHint ? False
IsBrokerConfigured - False
HomeAccountId - False
CorrelationId - 72047caa-2794-4a4b-bb03-4aa07adb953e
UserAssertion set: False
LongRunningOboCacheKey set: False
Region configured:

DEBUG: False MSAL 4.56.0.0 MSAL.CoreCLR .NET 8.0.1 Microsoft Windows 10.0.19045 [2024-02-29 18:41:01Z - 72047caa-2794-4a4b-bb03-4aa07adb953e] === Token Acquisition (SilentRequest) started:
         Scopes: https://storage.azure.com/.default
        Authority Host: login.microsoftonline.com
DEBUG: False MSAL 4.56.0.0 MSAL.CoreCLR .NET 8.0.1 Microsoft Windows 10.0.19045 [2024-02-29 18:41:01Z - 72047caa-2794-4a4b-bb03-4aa07adb953e] [Region discovery] Not using a regional authority.
DEBUG: False MSAL 4.56.0.0 MSAL.CoreCLR .NET 8.0.1 Microsoft Windows 10.0.19045 [2024-02-29 18:41:01Z - 72047caa-2794-4a4b-bb03-4aa07adb953e] Access token is not expired. Returning the found cache entry. [Current time (02/29/2024 18:41:01) - Expiration Time (02/29/2024 18:54:37 +00:00) - Extended Expiration Time (02/29/2024 18:54:37 +00:00)]
DEBUG: False MSAL 4.56.0.0 MSAL.CoreCLR .NET 8.0.1 Microsoft Windows 10.0.19045 [2024-02-29 18:41:01Z - 72047caa-2794-4a4b-bb03-4aa07adb953e] Returning access token found in cache. RefreshOn exists ? False
DEBUG: False MSAL 4.56.0.0 MSAL.CoreCLR .NET 8.0.1 Microsoft Windows 10.0.19045 [2024-02-29 18:41:01Z - 72047caa-2794-4a4b-bb03-4aa07adb953e] [Region discovery] Not using a regional authority.
DEBUG: False MSAL 4.56.0.0 MSAL.CoreCLR .NET 8.0.1 Microsoft Windows 10.0.19045 [2024-02-29 18:41:01Z - 72047caa-2794-4a4b-bb03-4aa07adb953e]
        === Token Acquisition finished successfully:
DEBUG: False MSAL 4.56.0.0 MSAL.CoreCLR .NET 8.0.1 Microsoft Windows 10.0.19045 [2024-02-29 18:41:01Z - 72047caa-2794-4a4b-bb03-4aa07adb953e]  AT expiration time: 29/02/2024 18:54:37 +00:00, scopes: https://storage.azure.com/user_impersonation https://storage.azure.com/.default. source: Cache
DEBUG: SharedTokenCacheCredential.GetToken succeeded. Scopes: [ https://storage.azure.com/.default ] ParentRequestId:  ExpiresOn: 2024-02-29T18:54:37.0000000+00:00
DEBUG: 18:41:01 - [SilentAuthenticator] Calling SharedTokenCacheCredential.GetTokenAsync - TenantId:'00000000-0000-0000-0000-000000000000', Scopes:'https://storage.azure.com/.default', AuthorityHost:'https://login.microsoftonline.com/', UserId:'me'
DEBUG: SharedTokenCacheCredential.GetToken invoked. Scopes: [ https://storage.azure.com/.default ] ParentRequestId:
DEBUG: False MSAL 4.56.0.0 MSAL.CoreCLR .NET 8.0.1 Microsoft Windows 10.0.19045 [2024-02-29 18:41:01Z - 243f4334-c917-495a-8212-fb7e277041c9] IsLegacyAdalCacheEnabled: yes
DEBUG: False MSAL 4.56.0.0 MSAL.CoreCLR .NET 8.0.1 Microsoft Windows 10.0.19045 [2024-02-29 18:41:01Z - 243f4334-c917-495a-8212-fb7e277041c9] [Region discovery] Not using a regional authority.
DEBUG: False MSAL 4.56.0.0 MSAL.CoreCLR .NET 8.0.1 Microsoft Windows 10.0.19045 [2024-02-29 18:41:01Z - 243f4334-c917-495a-8212-fb7e277041c9] [Region discovery] Not using a regional authority.
DEBUG: False MSAL 4.56.0.0 MSAL.CoreCLR .NET 8.0.1 Microsoft Windows 10.0.19045 [2024-02-29 18:41:01Z - 243f4334-c917-495a-8212-fb7e277041c9] IsLegacyAdalCacheEnabled: yes
DEBUG: False MSAL 4.56.0.0 MSAL.CoreCLR .NET 8.0.1 Microsoft Windows 10.0.19045 [2024-02-29 18:41:01Z - 243f4334-c917-495a-8212-fb7e277041c9] IsLegacyAdalCacheEnabled: yes
DEBUG: False MSAL 4.56.0.0 MSAL.CoreCLR .NET 8.0.1 Microsoft Windows 10.0.19045 [2024-02-29 18:41:01Z] Found 1 cache accounts and 0 broker accounts
DEBUG: False MSAL 4.56.0.0 MSAL.CoreCLR .NET 8.0.1 Microsoft Windows 10.0.19045 [2024-02-29 18:41:01Z] Returning 1 accounts
DEBUG: False MSAL 4.56.0.0 MSAL.CoreCLR .NET 8.0.1 Microsoft Windows 10.0.19045 [2024-02-29 18:41:01Z - 5a16727a-71e6-42cd-9b9b-d3da607beb54] MSAL MSAL.CoreCLR with assembly version '4.56.0.0'. CorrelationId(5a16727a-71e6-42cd-9b9b-d3da607beb54)
DEBUG: False MSAL 4.56.0.0 MSAL.CoreCLR .NET 8.0.1 Microsoft Windows 10.0.19045 [2024-02-29 18:41:01Z - 5a16727a-71e6-42cd-9b9b-d3da607beb54] === AcquireTokenSilent Parameters ===
DEBUG: False MSAL 4.56.0.0 MSAL.CoreCLR .NET 8.0.1 Microsoft Windows 10.0.19045 [2024-02-29 18:41:01Z - 5a16727a-71e6-42cd-9b9b-d3da607beb54] LoginHint provided: False
DEBUG: False MSAL 4.56.0.0 MSAL.CoreCLR .NET 8.0.1 Microsoft Windows 10.0.19045 [2024-02-29 18:41:01Z - 5a16727a-71e6-42cd-9b9b-d3da607beb54] Account provided: True
DEBUG: False MSAL 4.56.0.0 MSAL.CoreCLR .NET 8.0.1 Microsoft Windows 10.0.19045 [2024-02-29 18:41:01Z - 5a16727a-71e6-42cd-9b9b-d3da607beb54] ForceRefresh: False
DEBUG: False MSAL 4.56.0.0 MSAL.CoreCLR .NET 8.0.1 Microsoft Windows 10.0.19045 [2024-02-29 18:41:01Z - 5a16727a-71e6-42cd-9b9b-d3da607beb54]
=== Request Data ===
Authority Provided? - True
Scopes - https://storage.azure.com/.default
Extra Query Params Keys (space separated) -
ApiId - AcquireTokenSilent
IsConfidentialClient - False
SendX5C - False
LoginHint ? False
IsBrokerConfigured - False
HomeAccountId - False
CorrelationId - 5a16727a-71e6-42cd-9b9b-d3da607beb54
UserAssertion set: False
LongRunningOboCacheKey set: False
Region configured:

DEBUG: False MSAL 4.56.0.0 MSAL.CoreCLR .NET 8.0.1 Microsoft Windows 10.0.19045 [2024-02-29 18:41:01Z - 5a16727a-71e6-42cd-9b9b-d3da607beb54] === Token Acquisition (SilentRequest) started:
         Scopes: https://storage.azure.com/.default
        Authority Host: login.microsoftonline.com
DEBUG: False MSAL 4.56.0.0 MSAL.CoreCLR .NET 8.0.1 Microsoft Windows 10.0.19045 [2024-02-29 18:41:01Z - 5a16727a-71e6-42cd-9b9b-d3da607beb54] [Region discovery] Not using a regional authority.
DEBUG: False MSAL 4.56.0.0 MSAL.CoreCLR .NET 8.0.1 Microsoft Windows 10.0.19045 [2024-02-29 18:41:01Z - 5a16727a-71e6-42cd-9b9b-d3da607beb54] Access token is not expired. Returning the found cache entry. [Current time (02/29/2024 18:41:01) - Expiration Time (02/29/2024 18:54:37 +00:00) - Extended Expiration Time (02/29/2024 18:54:37 +00:00)]
DEBUG: False MSAL 4.56.0.0 MSAL.CoreCLR .NET 8.0.1 Microsoft Windows 10.0.19045 [2024-02-29 18:41:01Z - 5a16727a-71e6-42cd-9b9b-d3da607beb54] Returning access token found in cache. RefreshOn exists ? False
DEBUG: False MSAL 4.56.0.0 MSAL.CoreCLR .NET 8.0.1 Microsoft Windows 10.0.19045 [2024-02-29 18:41:01Z - 5a16727a-71e6-42cd-9b9b-d3da607beb54] [Region discovery] Not using a regional authority.
DEBUG: False MSAL 4.56.0.0 MSAL.CoreCLR .NET 8.0.1 Microsoft Windows 10.0.19045 [2024-02-29 18:41:01Z - 5a16727a-71e6-42cd-9b9b-d3da607beb54]
        === Token Acquisition finished successfully:
DEBUG: False MSAL 4.56.0.0 MSAL.CoreCLR .NET 8.0.1 Microsoft Windows 10.0.19045 [2024-02-29 18:41:01Z - 5a16727a-71e6-42cd-9b9b-d3da607beb54]  AT expiration time: 29/02/2024 18:54:37 +00:00, scopes: https://storage.azure.com/user_impersonation https://storage.azure.com/.default. source: Cache
DEBUG: SharedTokenCacheCredential.GetToken succeeded. Scopes: [ https://storage.azure.com/.default ] ParentRequestId:  ExpiresOn: 2024-02-29T18:54:37.0000000+00:00
DEBUG: AzureQoSEvent:  Module: Az.Storage:6.1.1; CommandName: New-AzStorageContext; PSVersion: 7.4.1; IsSuccess: True; Duration: 00:00:00.0550859
DEBUG: 18:41:01 - NewAzureStorageContext end processing.


PS> Set-AzStorageFileContent `
>>     -Context $context `
>>     -ShareName $ShareName `
>>     -Path 'example.file' `
>>     -Source $srcPath `
>>     -PreserveSMBAttribute `
>>     -Force
DEBUG: 18:41:07 - Init Operation Context for 'SetAzureStorageFileContent' with client request id  Azure-Storage-PowerShell-7962f7f7-6182-4b1b-befa-6228c3c903f5. If you want to get more details, please add "-Debug" to your command.
DEBUG: 18:41:07 - SetAzureStorageFileContent begin processing with ParameterSet 'ShareName'.
DEBUG: 18:41:07 - Use storage account 'storageaccountname' from storage context.
DEBUG: [29/02/2024 18:41:07] GetTokenAsync: token
DEBUG: [29/02/2024 18:41:08] GetTokenAsync: token
DEBUG: Request [53ed0024-bc7e-44f8-a0c4-83c2be14c061] GET https://storageaccountname.file.core.windows.net/sharename/example.file?restype=directory
x-ms-allow-trailing-dot:REDACTED
x-ms-version:2023-08-03
x-ms-file-request-intent:REDACTED
Accept:application/xml
User-Agent:AzurePowershell/v1.0.0,azsdk-net-Storage.Files.Shares/12.16.0 (.NET 8.0.1; Microsoft Windows 10.0.19045)
x-ms-client-request-id:53ed0024-bc7e-44f8-a0c4-83c2be14c061
x-ms-return-client-request-id:true
Authorization:REDACTED
client assembly: Azure.Storage.Files.Shares
DEBUG: Error response [53ed0024-bc7e-44f8-a0c4-83c2be14c061] 404 The specified resource does not exist. (00.2s)
Server:Windows-Azure-File/1.0,Microsoft-HTTPAPI/2.0
x-ms-request-id:021bfbe5-201a-0022-543e-6b10a7000000
x-ms-client-request-id:53ed0024-bc7e-44f8-a0c4-83c2be14c061
x-ms-version:2023-08-03
x-ms-error-code:ResourceNotFound
Date:Thu, 29 Feb 2024 18:41:07 GMT
Content-Length:223
Content-Type:application/xml

DEBUG: Request [413bf205-7128-4db4-9bdf-afbc59481c9a] PUT https://storageaccountname.file.core.windows.net/sharename/example.file
x-ms-allow-trailing-dot:REDACTED
x-ms-version:2023-08-03
x-ms-content-length:9306096
x-ms-type:file
x-ms-file-permission:REDACTED
x-ms-file-attributes:None
x-ms-file-creation-time:Now
x-ms-file-last-write-time:Now
x-ms-file-request-intent:REDACTED
Accept:application/xml
User-Agent:AzurePowershell/v1.0.0,azsdk-net-Storage.Files.Shares/12.16.0 (.NET 8.0.1; Microsoft Windows 10.0.19045)
x-ms-client-request-id:413bf205-7128-4db4-9bdf-afbc59481c9a
x-ms-return-client-request-id:true
Authorization:REDACTED
client assembly: Azure.Storage.Files.Shares
DEBUG: [29/02/2024 18:41:08] GetTokenAsync: token
DEBUG: [29/02/2024 18:41:08] GetTokenAsync: token
DEBUG: [29/02/2024 18:41:08] GetTokenAsync: token
DEBUG: Response [413bf205-7128-4db4-9bdf-afbc59481c9a] 201 Created (00.0s)
ETag:"0x8DC3955FE37E3BC"
Server:Windows-Azure-File/1.0,Microsoft-HTTPAPI/2.0
x-ms-request-id:021bfbed-201a-0022-583e-6b10a7000000
x-ms-client-request-id:413bf205-7128-4db4-9bdf-afbc59481c9a
x-ms-version:2023-08-03
x-ms-file-change-time:2024-02-29T18:41:07.9846844Z
x-ms-file-last-write-time:2024-02-29T18:41:07.9846844Z
x-ms-file-creation-time:2024-02-29T18:41:07.9846844Z
x-ms-file-permission-key:REDACTED
x-ms-file-attributes:Archive
x-ms-file-id:12970426300454928384
x-ms-file-parent-id:0
x-ms-request-server-encrypted:true
Date:Thu, 29 Feb 2024 18:41:07 GMT
Content-Length:0
Last-Modified:Thu, 29 Feb 2024 18:41:07 GMT

DEBUG: Request [dded9634-6241-4c6e-b56f-24345e5c8afc] PUT https://storageaccountname.file.core.windows.net/sharename/example.file?comp=range
x-ms-range:bytes=0-4194303
x-ms-write:update
x-ms-version:2023-08-03
x-ms-allow-trailing-dot:REDACTED
x-ms-file-request-intent:REDACTED
Accept:application/xml
Content-Length:4194304
Content-Type:application/octet-stream
User-Agent:AzurePowershell/v1.0.0,azsdk-net-Storage.Files.Shares/12.16.0 (.NET 8.0.1; Microsoft Windows 10.0.19045)
x-ms-client-request-id:dded9634-6241-4c6e-b56f-24345e5c8afc
x-ms-return-client-request-id:true
Authorization:REDACTED
client assembly: Azure.Storage.Files.Shares
DEBUG: Request [4202bf86-0ee0-4bcd-bb86-0b4692e6a7f6] PUT https://storageaccountname.file.core.windows.net/sharename/example.file?comp=range
x-ms-range:bytes=4194304-8388607
x-ms-write:update
x-ms-version:2023-08-03
x-ms-allow-trailing-dot:REDACTED
x-ms-file-request-intent:REDACTED
Accept:application/xml
Content-Length:4194304
Content-Type:application/octet-stream
User-Agent:AzurePowershell/v1.0.0,azsdk-net-Storage.Files.Shares/12.16.0 (.NET 8.0.1; Microsoft Windows 10.0.19045)
x-ms-client-request-id:4202bf86-0ee0-4bcd-bb86-0b4692e6a7f6
x-ms-return-client-request-id:true
Authorization:REDACTED
client assembly: Azure.Storage.Files.Shares
DEBUG: Request [b096868f-75cb-4533-8cf3-cb4deacaa60e] PUT https://storageaccountname.file.core.windows.net/sharename/example.file?comp=range
x-ms-range:bytes=8388608-9306095
x-ms-write:update
x-ms-version:2023-08-03
x-ms-allow-trailing-dot:REDACTED
x-ms-file-request-intent:REDACTED
Accept:application/xml
Content-Length:917488
Content-Type:application/octet-stream
User-Agent:AzurePowershell/v1.0.0,azsdk-net-Storage.Files.Shares/12.16.0 (.NET 8.0.1; Microsoft Windows 10.0.19045)
x-ms-client-request-id:b096868f-75cb-4533-8cf3-cb4deacaa60e
x-ms-return-client-request-id:true
Authorization:REDACTED
client assembly: Azure.Storage.Files.Shares
DEBUG: Response [b096868f-75cb-4533-8cf3-cb4deacaa60e] 201 Created (00.5s)
ETag:"0x8DC3955FE8A42A9"
Server:Windows-Azure-File/1.0,Microsoft-HTTPAPI/2.0
x-ms-request-id:85f39a48-501a-0007-053e-6bb9db000000
x-ms-client-request-id:b096868f-75cb-4533-8cf3-cb4deacaa60e
x-ms-version:2023-08-03
x-ms-file-last-write-time:2024-02-29T18:41:08.5245097Z
x-ms-request-server-encrypted:true
Date:Thu, 29 Feb 2024 18:41:08 GMT
Content-Length:0
Content-MD5:NeDq/5CfGvqdHsKyueW6Ww==
Last-Modified:Thu, 29 Feb 2024 18:41:08 GMT

DEBUG: Response [dded9634-6241-4c6e-b56f-24345e5c8afc] 201 Created (00.6s)
ETag:"0x8DC3955FE95B36A"
Server:Windows-Azure-File/1.0,Microsoft-HTTPAPI/2.0
x-ms-request-id:021bfbee-201a-0022-593e-6b10a7000000
x-ms-client-request-id:dded9634-6241-4c6e-b56f-24345e5c8afc
x-ms-version:2023-08-03
x-ms-file-last-write-time:2024-02-29T18:41:08.5994858Z
x-ms-request-server-encrypted:true
Date:Thu, 29 Feb 2024 18:41:08 GMT
Content-Length:0
Content-MD5:9BoFN0yJvMpMSe49S62YZw==
Last-Modified:Thu, 29 Feb 2024 18:41:08 GMT

DEBUG: Response [4202bf86-0ee0-4bcd-bb86-0b4692e6a7f6] 201 Created (00.9s)
ETag:"0x8DC3955FEC6354B"
Server:Windows-Azure-File/1.0,Microsoft-HTTPAPI/2.0
x-ms-request-id:393b3632-401a-0069-5f3e-6becf4000000
x-ms-client-request-id:4202bf86-0ee0-4bcd-bb86-0b4692e6a7f6
x-ms-version:2023-08-03
x-ms-file-last-write-time:2024-02-29T18:41:08.9173835Z
x-ms-request-server-encrypted:true
Date:Thu, 29 Feb 2024 18:41:08 GMT
Content-Length:0
Content-MD5:85z2NxQrk4Em1vscA2bhbw==
Last-Modified:Thu, 29 Feb 2024 18:41:08 GMT

DEBUG: Request [96ac76f5-8c2b-4b30-b116-b1eeb1e8adfd] PUT https://storageaccountname.file.core.windows.net/sharename/example.file?comp=properties
x-ms-version:2023-08-03
x-ms-content-md5:NUMWLLmFpdiaJA758lng2Q==
x-ms-file-permission:REDACTED
x-ms-file-attributes:Archive
x-ms-file-creation-time:2024-02-29T18:28:38.7751152Z
x-ms-file-last-write-time:2024-02-01T03:39:15.0000000Z
x-ms-allow-trailing-dot:REDACTED
x-ms-file-request-intent:REDACTED
Accept:application/xml
User-Agent:AzurePowershell/v1.0.0,azsdk-net-Storage.Files.Shares/12.16.0 (.NET 8.0.1; Microsoft Windows 10.0.19045)
x-ms-client-request-id:96ac76f5-8c2b-4b30-b116-b1eeb1e8adfd
x-ms-return-client-request-id:true
Authorization:REDACTED
client assembly: Azure.Storage.Files.Shares
DEBUG: Response [96ac76f5-8c2b-4b30-b116-b1eeb1e8adfd] 200 OK (00.0s)
ETag:"0x8DC3955FECC2846"
Server:Windows-Azure-File/1.0,Microsoft-HTTPAPI/2.0
x-ms-request-id:393b3637-401a-0069-603e-6becf4000000
x-ms-client-request-id:96ac76f5-8c2b-4b30-b116-b1eeb1e8adfd
x-ms-version:2023-08-03
x-ms-file-change-time:2024-02-29T18:41:08.9563718Z
x-ms-file-last-write-time:2024-02-01T03:39:15.0000000Z
x-ms-file-creation-time:2024-02-29T18:28:38.7751152Z
x-ms-file-id:12970426300454928384
x-ms-file-parent-id:0
x-ms-file-permission-key:REDACTED
x-ms-file-attributes:Archive
x-ms-request-server-encrypted:true
Date:Thu, 29 Feb 2024 18:41:08 GMT
Content-Length:0
Last-Modified:Thu, 29 Feb 2024 18:41:08 GMT

DEBUG: 18:41:09 - SetAzureStorageFileContent end processing, Start 0 remote calls. Finish 0 remote calls. Elapsed time 57912472.69 ms. Client operation id: Azure-Storage-PowerShell-7962f7f7-6182-4b1b-befa-6228c3c903f5.
DEBUG: AzureQoSEvent:  Module: Az.Storage:6.1.1; CommandName: Set-AzStorageFileContent; PSVersion: 7.4.1; IsSuccess: True; Duration: 00:00:01.1648962
DEBUG: 18:41:09 - SetAzureStorageFileContent end processing.



PS> Get-AzStorageFileContent `
>>     -Context $context `
>>     -ShareName $ShareName `
>>     -Path 'example.file' `
>>     -Destination $destPath `
>>     -PreserveSMBAttribute `
>>     -Force
DEBUG: 18:41:14 - Init Operation Context for 'GetAzureStorageFileContent' with client request id  Azure-Storage-PowerShell-7962f7f7-6182-4b1b-befa-6228c3c903f5. If you want to get more details, please add "-Debug" to your command.
DEBUG: 18:41:14 - GetAzureStorageFileContent begin processing with ParameterSet 'ShareName'.
DEBUG: 18:41:14 - Use storage account 'storageaccountname' from storage context.
DEBUG: [29/02/2024 18:41:14] GetTokenAsync: token
DEBUG: Request [c038d535-0d90-4995-894c-d32f991054f0] HEAD https://storageaccountname.file.core.windows.net/sharename/example.file
x-ms-allow-trailing-dot:REDACTED
x-ms-version:2023-08-03
x-ms-file-request-intent:REDACTED
Accept:application/xml
User-Agent:AzurePowershell/v1.0.0,azsdk-net-Storage.Files.Shares/12.16.0 (.NET 8.0.1; Microsoft Windows 10.0.19045)
x-ms-client-request-id:c038d535-0d90-4995-894c-d32f991054f0
x-ms-return-client-request-id:true
Authorization:REDACTED
client assembly: Azure.Storage.Files.Shares
DEBUG: Response [c038d535-0d90-4995-894c-d32f991054f0] 200 OK (00.0s)
ETag:"0x8DC3955FECC2846"
Server:Windows-Azure-File/1.0,Microsoft-HTTPAPI/2.0
x-ms-request-id:393b3683-401a-0069-6b3e-6becf4000000
x-ms-client-request-id:c038d535-0d90-4995-894c-d32f991054f0
x-ms-version:2023-08-03
x-ms-type:File
x-ms-server-encrypted:true
x-ms-lease-status:unlocked
x-ms-lease-state:available
x-ms-file-change-time:2024-02-29T18:41:08.9563718Z
x-ms-file-last-write-time:2024-02-01T03:39:15.0000000Z
x-ms-file-creation-time:2024-02-29T18:28:38.7751152Z
x-ms-file-id:12970426300454928384
x-ms-file-parent-id:0
x-ms-file-permission-key:REDACTED
x-ms-file-attributes:Archive
Date:Thu, 29 Feb 2024 18:41:14 GMT
Content-Length:9306096
Content-MD5:NUMWLLmFpdiaJA758lng2Q==
Last-Modified:Thu, 29 Feb 2024 18:41:08 GMT

DEBUG: Request [5b6e82ec-fc09-4931-bb9b-3daac84e8037] GET https://storageaccountname.file.core.windows.net/sharename/example.file
x-ms-allow-trailing-dot:REDACTED
x-ms-version:2023-08-03
x-ms-range:bytes=0-4194303
x-ms-file-request-intent:REDACTED
Accept:application/xml
User-Agent:AzurePowershell/v1.0.0,azsdk-net-Storage.Files.Shares/12.16.0 (.NET 8.0.1; Microsoft Windows 10.0.19045)
x-ms-client-request-id:5b6e82ec-fc09-4931-bb9b-3daac84e8037
x-ms-return-client-request-id:true
Authorization:REDACTED
client assembly: Azure.Storage.Files.Shares
DEBUG: Response [5b6e82ec-fc09-4931-bb9b-3daac84e8037] 206 Partial Content (00.1s)
Accept-Ranges:bytes
ETag:"0x8DC3955FECC2846"
Server:Windows-Azure-File/1.0,Microsoft-HTTPAPI/2.0
x-ms-request-id:393b3684-401a-0069-6c3e-6becf4000000
x-ms-client-request-id:5b6e82ec-fc09-4931-bb9b-3daac84e8037
x-ms-version:2023-08-03
x-ms-content-md5:NUMWLLmFpdiaJA758lng2Q==
x-ms-type:File
x-ms-server-encrypted:true
x-ms-lease-status:unlocked
x-ms-lease-state:available
x-ms-file-change-time:2024-02-29T18:41:08.9563718Z
x-ms-file-last-write-time:2024-02-01T03:39:15.0000000Z
x-ms-file-creation-time:2024-02-29T18:28:38.7751152Z
x-ms-file-id:12970426300454928384
x-ms-file-parent-id:0
x-ms-file-permission-key:REDACTED
x-ms-file-attributes:Archive
Date:Thu, 29 Feb 2024 18:41:14 GMT
Content-Length:4194304
Content-Range:bytes 0-4194303/9306096
Last-Modified:Thu, 29 Feb 2024 18:41:08 GMT

DEBUG: Request [714ec64a-b78f-499c-afb9-bbdd0c26bec6] GET https://storageaccountname.file.core.windows.net/sharename/example.file
x-ms-allow-trailing-dot:REDACTED
x-ms-version:2023-08-03
x-ms-range:bytes=4194304-8388607
x-ms-file-request-intent:REDACTED
Accept:application/xml
User-Agent:AzurePowershell/v1.0.0,azsdk-net-Storage.Files.Shares/12.16.0 (.NET 8.0.1; Microsoft Windows 10.0.19045)
x-ms-client-request-id:714ec64a-b78f-499c-afb9-bbdd0c26bec6
x-ms-return-client-request-id:true
Authorization:REDACTED
client assembly: Azure.Storage.Files.Shares
DEBUG: Response [714ec64a-b78f-499c-afb9-bbdd0c26bec6] 206 Partial Content (00.1s)
Accept-Ranges:bytes
ETag:"0x8DC3955FECC2846"
Server:Windows-Azure-File/1.0,Microsoft-HTTPAPI/2.0
x-ms-request-id:393b368a-401a-0069-703e-6becf4000000
x-ms-client-request-id:714ec64a-b78f-499c-afb9-bbdd0c26bec6
x-ms-version:2023-08-03
x-ms-content-md5:NUMWLLmFpdiaJA758lng2Q==
x-ms-type:File
x-ms-server-encrypted:true
x-ms-lease-status:unlocked
x-ms-lease-state:available
x-ms-file-change-time:2024-02-29T18:41:08.9563718Z
x-ms-file-last-write-time:2024-02-01T03:39:15.0000000Z
x-ms-file-creation-time:2024-02-29T18:28:38.7751152Z
x-ms-file-id:12970426300454928384
x-ms-file-parent-id:0
x-ms-file-permission-key:REDACTED
x-ms-file-attributes:Archive
Date:Thu, 29 Feb 2024 18:41:14 GMT
Content-Length:4194304
Content-Range:bytes 4194304-8388607/9306096
Last-Modified:Thu, 29 Feb 2024 18:41:08 GMT

DEBUG: Request [28a24655-fbb2-4510-880a-d194e2a201cb] GET https://storageaccountname.file.core.windows.net/sharename/example.file
x-ms-allow-trailing-dot:REDACTED
x-ms-version:2023-08-03
x-ms-range:bytes=8388608-9306095
x-ms-file-request-intent:REDACTED
Accept:application/xml
User-Agent:AzurePowershell/v1.0.0,azsdk-net-Storage.Files.Shares/12.16.0 (.NET 8.0.1; Microsoft Windows 10.0.19045)
x-ms-client-request-id:28a24655-fbb2-4510-880a-d194e2a201cb
x-ms-return-client-request-id:true
Authorization:REDACTED
client assembly: Azure.Storage.Files.Shares
DEBUG: Response [28a24655-fbb2-4510-880a-d194e2a201cb] 206 Partial Content (00.1s)
Accept-Ranges:bytes
ETag:"0x8DC3955FECC2846"
Server:Windows-Azure-File/1.0,Microsoft-HTTPAPI/2.0
x-ms-request-id:393b368e-401a-0069-723e-6becf4000000
x-ms-client-request-id:28a24655-fbb2-4510-880a-d194e2a201cb
x-ms-version:2023-08-03
x-ms-content-md5:NUMWLLmFpdiaJA758lng2Q==
x-ms-type:File
x-ms-server-encrypted:true
x-ms-lease-status:unlocked
x-ms-lease-state:available
x-ms-file-change-time:2024-02-29T18:41:08.9563718Z
x-ms-file-last-write-time:2024-02-01T03:39:15.0000000Z
x-ms-file-creation-time:2024-02-29T18:28:38.7751152Z
x-ms-file-id:12970426300454928384
x-ms-file-parent-id:0
x-ms-file-permission-key:REDACTED
x-ms-file-attributes:Archive
Date:Thu, 29 Feb 2024 18:41:14 GMT
Content-Length:917488
Content-Range:bytes 8388608-9306095/9306096
Last-Modified:Thu, 29 Feb 2024 18:41:08 GMT

DEBUG: 18:41:15 - GetAzureStorageFileContent end processing, Start 0 remote calls. Finish 0 remote calls. Elapsed time 57918430.80 ms. Client operation id: Azure-Storage-PowerShell-7962f7f7-6182-4b1b-befa-6228c3c903f5.
DEBUG: AzureQoSEvent:  Module: Az.Storage:6.1.1; CommandName: Get-AzStorageFileContent; PSVersion: 7.4.1; IsSuccess: True; Duration: 00:00:00.4750089
DEBUG: 18:41:15 - GetAzureStorageFileContent end processing.

PS> $expectedHash = (Get-FileHash -Path $srcPath -Algorithm SHA256).Hash
PS> $actualHash = (Get-FileHash -Path $destPath -Algorithm SHA256).Hash
PS>
PS> Write-Host "Expected: $expectedHash"
Expected: 84EA3821ACEE25A4489DD428CFBF1C8A38599F29F36EC1A3356CA219041424DA
PS> Write-Host "Actual  : $actualHash"
Actual  : 48B9CE1AC58926139958D0C26CC6B1826E6320BAB907AC81AD83F64BC6D8D05C

Environment data

Name                           Value
----                           -----
PSVersion                      7.4.1
PSEdition                      Core
GitCommitId                    7.4.1
OS                             Microsoft Windows 10.0.19045
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

Module versions

ModuleType Version    PreRelease Name                                ExportedCommands
---------- -------    ---------- ----                                ----------------
Script     2.15.1                Az.Accounts                         {Add-AzEnvironment, Clear-AzConfig, Clear-AzContext, Clear-AzDefault…}
Script     6.1.1                 Az.Storage                          {Add-AzRmStorageContainerLegalHold, Add-AzStorageAccountManagementPolicyAction, Add-AzStorageAccountNetworkRule, Close-AzStorageFileHandle…}

Error output

No response
@billings7 billings7 added bug This issue requires a change to an existing behavior in the product in order to be resolved. needs-triage This is a new issue that needs to be triaged to the appropriate team. labels Feb 29, 2024
@microsoft-github-policy-service microsoft-github-policy-service bot added customer-reported needs-triage This is a new issue that needs to be triaged to the appropriate team. and removed needs-triage This is a new issue that needs to be triaged to the appropriate team. labels Feb 29, 2024
blueww added a commit to wastoresh/azure-powershell that referenced this issue Mar 1, 2024
@blueww
Fix issue Azure#24289
4232afb
@blueww blueww mentioned this issue Mar 1, 2024
Merged
6 tasks
@blueww
Copy link
Member

blueww commented Mar 1, 2024 •
edited

@billings7

Thanks for raising this issue!
I have raised a fix PR this this issue: #24291

I have tested file upload on different size (from 1KB up to 2GB) with the fix, and they all work well (upload and download the file, file hash is same.)

@blueww blueww self-assigned this Mar 1, 2024
@blueww blueww added Storage and removed needs-triage This is a new issue that needs to be triaged to the appropriate team. labels Mar 1, 2024
blueww added a commit to wastoresh/azure-powershell that referenced this issue Mar 4, 2024
@blueww
Fix issue Azure#24289
d44e27d
vidai-msft pushed a commit that referenced this issue Mar 6, 2024
@blueww
[Storage] Fix file upload with Oauth issue (#24291)
d520e15 
* Fix issue #24289

* Respect the actualy bytes number read from file stream.
@blueww
Copy link
Member

blueww commented Apr 3, 2024

Close as the fix is already released in latest Az.Storage 6.1.3 (Az 11.5.0).

@blueww blueww closed this as completed Apr 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@blueww blueww

Labels
bug This issue requires a change to an existing behavior in the product in order to be resolved. customer-reported Storage
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@blueww @billings7