storage: PostPolicyV4Options.SignBytes is not compatible with Credentials API

[apstndb]

Client

Cloud Storage

Environment

Reproduced in local and remote environment

Go Environment

$ go version
go version go1.16.5 darwin/amd64

Code

The definition of PostPolicyV4Options.SignBytes and SignedURLOptions.SignBytes differ.

The input of PostPolicyV4Options.SignBytes is hashedBytes.

google-cloud-go/storage/post_policy_v4.go

Lines 221 to 233 in 27c3ed0

	var signingFn func(hashedBytes []byte) ([]byte, error)
	switch {
	case opts.SignBytes != nil:
	signingFn = opts.SignBytes
	case len(opts.PrivateKey) != 0:
	parsedRSAPrivKey, err := parseKey(opts.PrivateKey)
	if err != nil {
	return nil, err
	}
	signingFn = func(hashedBytes []byte) ([]byte, error) {
	return rsa.SignPKCS1v15(rand.Reader, parsedRSAPrivKey, crypto.SHA256, hashedBytes)
	}

google-cloud-go/storage/post_policy_v4.go

Lines 310 to 311 in 27c3ed0

	shaSum := sha256.Sum256([]byte(b64Policy))
	signature, err := signingFn(shaSum[:])

The input of SignedURLOptions.SignBytes is raw bytes.

google-cloud-go/storage/storage.go

Lines 699 to 715 in 27c3ed0

	signBytes := opts.SignBytes
	if opts.PrivateKey != nil {
	key, err := parseKey(opts.PrivateKey)
	if err != nil {
	return "", err
	}
	signBytes = func(b []byte) ([]byte, error) {
	sum := sha256.Sum256(b)
	return rsa.SignPKCS1v15(
	rand.Reader,
	key,
	crypto.SHA256,
	sum[:],
	)
	}
	}
	b, err := signBytes(signBuf.Bytes())

This difference seems to make PostPolicyV4Options.SignBytes incompatible with projects.serviceAccounts.signBlob of Service Account Credentials API.

Code

email := os.Getenv("IMPERSONATE_SERVICE_ACCOUNT")
var signBytesFn func(b []byte) ([]byte, error)
var privateKey []byte

if email != "" {
	credCli, err := credentials.NewIamCredentialsClient(ctx)
	if err != nil {
		 panic(err)
	}
	defer credCli.Close()

	signBytesFn = func(b []byte) ([]byte, error) {
		 req := &credentialspb.SignBlobRequest{
			 Payload: b,
			 Name:    email,
		 }
		 resp, err := credCli.SignBlob(ctx, req)
		 if err != nil {
			 panic(err)
		 }
		 return resp.SignedBlob, err
	}
} else {
	cred, _ := google.FindDefaultCredentials(ctx)
	jwtConfig, _ := google.JWTConfigFromJSON(cred.JSON)
	email = jwtConfig.Email
	privateKey = jwtConfig.PrivateKey
}

opts := &storage.PostPolicyV4Options{
	GoogleAccessID: email,
	SignBytes:      signBytesFn,
	PrivateKey:     privateKey,
	Expires:        time.Now().Add(10 * time.Minute),
	Fields: &storage.PolicyV4Fields{
		 Metadata: map[string]string{},
	},
}

policy, err := storage.GenerateSignedPostPolicyV4(bucket, object, opts)

Expected behavior

Signed policy document are valid in both cases of IMPERSONATE_SERVICE_ACCOUNT or GOOGLE_APPLICATION_CREDENTIALS.

Actual behavior

Signed policy document only valid in case of GOOGLE_APPLICATION_CREDENTIALS.

Additional context

Possible patch
storage/v1.16.1...apstndb:fix-policy-document

[BrennaEpp]

Hi @apstndb, thanks for filing this issue! We looked into this and are coming up with a solution for this. I'll update you later this week on the progress.

[BrennaEpp]

Update: I have a PR to implement a fix for this. To avoid breaking people depending on the current behaviour, we added an extra field to PostPolicyV4Options called SignRawBytes that is compatible with the credentials api and does not expect hashed bytes as input.
Once that is merged and released, this code should work with the replacement of SignBytes -> SignRawBytes.