Cannot set access policy

[hongbin]

Bug Report

• import path of package in question, e.g. .../services/compute/mgmt/2018-06-01/compute

github.com/Azure/azure-sdk-for-go/sdk/storage/azblob

• SDK version e.g. master, latest, 18.1.0

master

• output of go version

go version go1.18.1 linux/amd64

• What happened?

Cannot set access policy. Azure returned 400. According to the SDK log, the "Start" and "Expiry" field in the request's body is incorrect.

For example, the SDK sends <Start>2022-07-28T08:39:46.191396313Z</Start>. It has nine-digit millisecond. However, Azure expected seven-digit millisecond. See here:

The Start and Expiry fields must be expressed as UTC times and must adhere to a valid ISO 8061 format. Supported ISO 8061 formats include the following:

YYYY-MM-DD

YYYY-MM-DDThh:mmTZD

YYYY-MM-DDThh:mm:ssTZD

YYYY-MM-DDThh:mm:ss.fffffffTZD

$ AZURE_SDK_GO_LOGGING=all go run test_set_access_policy.go
[Jul 28 08:39:46.191541] Retry:
=====> Try=1 PUT https://REDACTED_STORAGE_ACCOUNT.blob.core.windows.net/REDACTED_CONTAINER?comp=acl&restype=container
[Jul 28 08:39:46.191619] Request: ==> OUTGOING REQUEST (Try=1)
   PUT https://REDACTED_STORAGE_ACCOUNT.blob.core.windows.net/REDACTED_CONTAINER?comp=REDACTED&restype=REDACTED
   Accept: application/xml
   Authorization: REDACTED
   Content-Length: 295
   Content-Type: application/xml
   User-Agent: azsdk-go-azblob/v0.4.1 (go1.18.1; linux)
   X-Ms-Date: REDACTED
   X-Ms-Version: REDACTED
   --------------------------------------------------------------------------------
<?xml version="1.0" encoding="UTF-8"?>
<SignedIdentifiers><SignedIdentifier><AccessPolicy><Permission>racwdl</Permission><Expiry>2027-07-28T08:39:46.191396313Z</Expiry><Start>2022-07-28T08:39:46.191396313Z</Start></AccessPolicy><Id>1816763986191396313</Id></SignedIdentifier></SignedIdentifiers>
   --------------------------------------------------------------------------------

[Jul 28 08:39:46.248047] Response: ==> REQUEST/RESPONSE (Try=1/56.329139ms, OpTime=56.38834ms) -- RESPONSE RECEIVED
   PUT https://REDACTED_STORAGE_ACCOUNT.blob.core.windows.net/REDACTED_CONTAINER?comp=REDACTED&restype=REDACTED
   Accept: application/xml
   Authorization: REDACTED
   Content-Length: 295
   Content-Type: application/xml
   User-Agent: azsdk-go-azblob/v0.4.1 (go1.18.1; linux)
   X-Ms-Date: REDACTED
   X-Ms-Version: REDACTED
   --------------------------------------------------------------------------------
   RESPONSE Status: 400 XML specified is not syntactically valid.
   Content-Length: 294
   Content-Type: application/xml
   Date: Thu, 28 Jul 2022 08:39:45 GMT
   Server: Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0
   X-Ms-Error-Code: REDACTED
   X-Ms-Request-Id: ecf57a31-f01e-0025-7b5d-a270d7000000
   X-Ms-Version: REDACTED
   --------------------------------------------------------------------------------
<?xml version="1.0" encoding="utf-8"?><Error><Code>InvalidXmlDocument</Code><Message>XML specified is not syntactically valid.
RequestId:ecf57a31-f01e-0025-7b5d-a270d7000000
Time:2022-07-28T08:39:46.2483859Z</Message><LineNumber>0</LineNumber><LinePosition>0</LinePosition><Reason /></Error>
   --------------------------------------------------------------------------------

[Jul 28 08:39:46.248093] Retry: response 400
error: cannot set access policy ===== RESPONSE ERROR (ErrorCode=InvalidXmlDocument) =====
Description=XML specified is not syntactically valid.
RequestId:ecf57a31-f01e-0025-7b5d-a270d7000000
Time:2022-07-28T08:39:46.2483859Z, Details:
   LineNumber: 0
   LinePosition: 0

• What did you expect or want to happen?

Azure returned 200

• How can we reproduce it?

I used the code below:

package main

import (
	"fmt"
	"context"
	"time"

	"github.com/Azure/azure-sdk-for-go/sdk/storage/azblob"
	"github.com/Azure/go-autorest/autorest/azure"
	"github.com/Azure/azure-sdk-for-go/sdk/azcore/policy"
)

var (
	Cloud = "AzurePublicCloud"
	StorageAccount = "REDACTED"
	StorageAccountKey = "REDACTED"
	Container = "REDACTED"
)

func test() error {
	credential, aerr := azblob.NewSharedKeyCredential(StorageAccount, StorageAccountKey)
	if aerr != nil {
		return fmt.Errorf("cannot create shared key credential %w", aerr)
	}

	cloudEnvironment, aerr := azure.EnvironmentFromName(Cloud)
	if aerr != nil {
		return fmt.Errorf("cannot get cloud environment %w", aerr)
	}
	url := fmt.Sprintf("https://%s.blob.%s/%s",
		StorageAccount, cloudEnvironment.StorageEndpointSuffix, Container)

	clientOptions := &azblob.ClientOptions{Logging: policy.LogOptions{IncludeBody: true}}
	containerClient, aerr := azblob.NewContainerClientWithSharedKey(url, credential, clientOptions)
	
	// set policies
	start := time.Now()
	expiry := start.AddDate(5, 0, 0)
	policyIdStr := fmt.Sprintf("%d", expiry.UnixNano())
	permission := azblob.AccessPolicyPermission{
		Read:   true,
		Add:    true,
		Create: true,
		Write:  true,
		Delete: true,
		List:   true,
	}.String()
	policies := make([]*azblob.SignedIdentifier, 0)
	policies = append(policies, &azblob.SignedIdentifier{
		ID: &policyIdStr,
		AccessPolicy: &azblob.AccessPolicy{
			Start:      &start,
			Expiry:     &expiry,
			Permission: &permission,
		},
	})
	_, aerr = containerClient.SetAccessPolicy(context.TODO(), &azblob.ContainerSetAccessPolicyOptions{
		ContainerACL: policies,
	})
	if aerr != nil {
		return fmt.Errorf("cannot set access policy %w", aerr)
	}
	
	return nil
}

func main() {
	err := test()
	if err != nil {
		fmt.Printf("error: %s\n", err)
	} else {
		fmt.Printf("succeeded\n")
	}
}

• Anything we should know about your environment.

N/A

[hongbin]

ping for update.

[hongbin]

Update?

[siminsavani-msft]

Hi @hongbin ! This seems like an outdated version of the Go SDK, please try the latest version (v0.5.1) and let us know if you are still having problems with SetAccessPolicy.

[hongbin]

@siminsavani-msft I upgrade to v0.5.1. The problem is still there:

$ AZURE_SDK_GO_LOGGING=all go run test_set_access_policy.go
[Nov  4 13:24:00.803211] Retry: =====> Try=1
[Nov  4 13:24:00.803257] Request: ==> OUTGOING REQUEST (Try=1)
   PUT https://hongbintest20221104.blob.core.windows.net/testcontainer?comp=REDACTED&restype=REDACTED
   Accept: application/xml
   Authorization: REDACTED
   Content-Length: 295
   Content-Type: application/xml
   User-Agent: azsdk-go-azblob/v0.5.1 (go1.18.1; linux)
   X-Ms-Date: REDACTED
   x-ms-version: REDACTED
   --------------------------------------------------------------------------------
<?xml version="1.0" encoding="UTF-8"?>
<SignedIdentifiers><SignedIdentifier><AccessPolicy><Permission>racwdl</Permission><Expiry>2027-11-04T13:24:00.803119935Z</Expiry><Start>2022-11-04T13:24:00.803119935Z</Start></AccessPolicy><Id>1825334640803119935</Id></SignedIdentifier></SignedIdentifiers>
   --------------------------------------------------------------------------------

[Nov  4 13:24:00.884437] Response: ==> REQUEST/RESPONSE (Try=1/81.094812ms, OpTime=81.128612ms) -- RESPONSE RECEIVED
   PUT https://hongbintest20221104.blob.core.windows.net/testcontainer?comp=REDACTED&restype=REDACTED
   Accept: application/xml
   Authorization: REDACTED
   Content-Length: 295
   Content-Type: application/xml
   User-Agent: azsdk-go-azblob/v0.5.1 (go1.18.1; linux)
   X-Ms-Date: REDACTED
   x-ms-version: REDACTED
   --------------------------------------------------------------------------------
   RESPONSE Status: 400 XML specified is not syntactically valid.
   Content-Length: 294
   Content-Type: application/xml
   Date: Fri, 04 Nov 2022 13:24:00 GMT
   Server: Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0
   X-Ms-Error-Code: REDACTED
   X-Ms-Request-Id: 7c4d1fbb-001e-0055-4450-f0558c000000
   X-Ms-Version: REDACTED
   --------------------------------------------------------------------------------
<?xml version="1.0" encoding="utf-8"?><Error><Code>InvalidXmlDocument</Code><Message>XML specified is not syntactically valid.
RequestId:7c4d1fbb-001e-0055-4450-f0558c000000
Time:2022-11-04T13:24:00.8821186Z</Message><LineNumber>0</LineNumber><LinePosition>0</LinePosition><Reason /></Error>
   --------------------------------------------------------------------------------

[Nov  4 13:24:00.884488] Retry: response 400
[Nov  4 13:24:00.884494] Retry: exit due to non-retriable status code
error: cannot set access policy PUT https://hongbintest20221104.blob.core.windows.net/testcontainer
--------------------------------------------------------------------------------
RESPONSE 400: 400 XML specified is not syntactically valid.
ERROR CODE: InvalidXmlDocument
--------------------------------------------------------------------------------
<?xml version="1.0" encoding="utf-8"?><Error><Code>InvalidXmlDocument</Code><Message>XML specified is not syntactically valid.
RequestId:7c4d1fbb-001e-0055-4450-f0558c000000
Time:2022-11-04T13:24:00.8821186Z</Message><LineNumber>0</LineNumber><LinePosition>0</LinePosition><Reason /></Error>
--------------------------------------------------------------------------------

[souravgupta-msft]

Hi @hongbin. Thanks for reporting this. We have identified this as a bug and have a PR #19565 open for this. We will keep you posted when it is fixed.

[souravgupta-msft]

The PR has been merged. You can import from the main branch and validate.
This fix will be part of the next release. Feel free to reopen if the issue persists.