Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expected fix for LeaseID missing not present in v0.12.0 #236

Closed
eccles opened this issue Dec 14, 2020 · 17 comments
Closed

Expected fix for LeaseID missing not present in v0.12.0 #236

eccles opened this issue Dec 14, 2020 · 17 comments
Assignees

Comments

@eccles
Copy link

eccles commented Dec 14, 2020

Which version of the SDK was used?

0.12.0

Which platform are you using? (ex: Windows, Linux, Debian)

Linux

What problem was encountered?

Incorrect version tagged upstream - expected fix #230 not implemented

How can we reproduce the problem in the simplest way?

go get v0.12.0 and inspect chunkwriting.go - expecetd fix from #230 not present in line 187

Have you found a mitigation/solution?

No

Some explanation

Thanks for the release - however the tagging has gone awry - I change my go.mod file to:

github.com/Azure/azure-storage-blob-go v0.12.0

and inspection of my go cache:

vi /opt/go/pkg/mod/github.com/!azure/azure-storage-blob-go@v0.12.0/azblob/chunkwriting.go

shows the change has not been applied.

187 _, err := c.to.StageBlock(c.ctx, chunk.id, bytes.NewReader(chunk.buffer), LeaseAccessConditions{}, nil, c.o.ClientProvidedKeyOptions)

If I pull repo directly using git clone and inspect that all looks good:

paul@paul-Precision-7530:~/git/azure-storage-blob-go$ git log
commit 6df5d9a (HEAD -> master, tag: v0.12.0, origin/master, origin/HEAD)
Merge: 456ab47 559b75b
Author: Mohit Sharma 65536214+mohsha-msft@users.noreply.github.com
Date: Wed Dec 9 17:40:48 2020 +0530

Merge pull request #235 from Azure/dev

Release v0.12.0
commit 559b75b (origin/dev)
Author: Mohit Sharma 65536214+mohsha-msft@users.noreply.github.com
Date: Wed Dec 9 12:49:08 2020 +0530

v0.12.0 Release change logs (#231)

Fixed #230 (#232)
and the fix is in:

186 _, err := c.to.StageBlock(c.ctx, chunk.id, bytes.NewReader(chunk.buffer), c.o.AccessConditions.LeaseAccessConditions, nil, c.o.ClientProvidedKeyOptions)

@eccles
Copy link
Author

eccles commented Dec 14, 2020

Speculation: the merge for the addition of c.o.ClientProvidedKeyOptions overwrote the change for c.o.AccessConditions.LeaseAccessConditions

@mohsha-msft
Copy link
Contributor

mohsha-msft commented Dec 14, 2020

Hey @eccles ,

Thanks for reaching out!
Please take a look here

Can you please try to do a go get github.com/Azure/azure-storage-blob-go again followed by go mod tidy?

@eccles
Copy link
Author

eccles commented Dec 16, 2020

Tried it but still no change - the code in /opt/go/pkg/mod/github.com/!azure/azure-storage-blob-go@v0.12.0/azblob/chunkwriting.go (which is in my gocache) is still incorrect.

@mohsha-msft
Copy link
Contributor

hey @eccles ,

The changes which you mentioned has already been merged. Can you explain what issue you are facing now?

@eccles
Copy link
Author

eccles commented Dec 16, 2020

To summaru=ise the state of the repo looks correct but when I do a go get I get a versuion without the #230 change which I need.
I have no explanation of why this is so

I have tried @dev which gives me the v0.10.1-0.20201209071908-559b75bbc31d which is no good.
I have tried @6df5d9af221d67aa1173ad090 which has the same problem..

@eccles
Copy link
Author

eccles commented Dec 16, 2020

go list -m -json github.com/Azure/azure-storage-blob-go@v0.12.0
{
"Path": "github.com/Azure/azure-storage-blob-go",
"Version": "v0.12.0",
"Time": "2020-11-21T18:08:49Z",
"Dir": "/opt/go/pkg/mod/github.com/!azure/azure-storage-blob-go@v0.12.0",
"GoMod": "/opt/go/pkg/mod/cache/download/github.com/!azure/azure-storage-blob-go/@v/v0.12.0.mod",
"GoVersion": "1.15"
}

The time looks a little suspicious... or maybe not

@h-no
Copy link

h-no commented Dec 16, 2020

I was curious and had a look myself.

I checkout out the repo.

❯ git show
commit 6df5d9af221d67aa1173ad09073ef95c872beda0 (HEAD -> master, tag: v0.12.0, origin/master, origin/HEAD)
Merge: 456ab47 559b75b
Author: Mohit Sharma <65536214+mohsha-msft@users.noreply.github.com>
Date:   Wed Dec 9 17:40:48 2020 +0530

    Merge pull request #235 from Azure/dev

    Release v0.12.0

Then diffed the entire tree using Meld with my go mod cache.
Ended up with the following diff:

❯ diff ~/.go/pkg/mod/github.com/\!azure/azure-storage-blob-go@v0.12.0/azblob/chunkwriting.go /github.com/Azure/azure-storage-blob-go/azblob/chunkwriting.go
186,187c186
<
< 	_, err := c.to.StageBlock(c.ctx, chunk.id, bytes.NewReader(chunk.buffer), LeaseAccessConditions{}, nil, c.o.ClientProvidedKeyOptions)
---
> 	_, err := c.to.StageBlock(c.ctx, chunk.id, bytes.NewReader(chunk.buffer), c.o.AccessConditions.LeaseAccessConditions, nil, c.o.ClientProvidedKeyOptions)

So the bytes that gets downloaded by go is not what's in the repo.

Could this be the reason why I got the checksum mismatch in this issue that you closed: #237

@h-no
Copy link

h-no commented Dec 16, 2020

So to summarize, @eccles I experience the same as you.

@bcmills
Copy link

bcmills commented Dec 16, 2020

It looks like the v0.12.0 tag was moved at some point:

$ GOPROXY=direct go get -d github.com/Azure/azure-storage-blob-go@6df5d9af221d67aa1173ad09073ef95c87
2beda0
go: downloading github.com/Azure/azure-storage-blob-go v0.12.0
go get github.com/Azure/azure-storage-blob-go@6df5d9af221d67aa1173ad09073ef95c872beda0: github.com/Azure/azure-storage-blob-go@v0.12.0: verifying module: checksum mismatch
        downloaded: h1:LEu0WO1uxAkeJFuNTZR+xxUpIoRUwOXgA2ipS798Ycc=
        sum.golang.org: h1:7bFXA1QB+lOK2/ASWHhp6/vnxjaeeZq6t8w1Jyp0Iaw=
SECURITY ERROR
This download does NOT match the one reported by the checksum server.
The bits may have been replaced on the origin server, or an attacker may
have intercepted the download attempt.
For more information, see 'go help module-auth'.

As a workaround, you can instead refer to that commit by the pseudo-version v0.11.1-0.20201209121048-6df5d9af221d, which is what it would have been called prior to the erroneous tag being applied:

$ go get -d github.com/Azure/azure-storage-blob-go@v0.11.1-0.20201209121048-6df5d9af221d
go: downloading github.com/Azure/azure-storage-blob-go v0.11.1-0.20201209121048-6df5d9af221d

$

@bcmills
Copy link

bcmills commented Dec 16, 2020

Probably the best fix going forward would be to retract v0.12.0 (see golang/go#24031 (comment)) and tag a new v0.12.1 at the desired commit.

@eccles
Copy link
Author

eccles commented Dec 17, 2020

@bcmills thanx - @mohsha-msft ?

stanhu added a commit to stanhu/go-cloud that referenced this issue Dec 18, 2020
The tag for this module got yanked and now checksums are failing. Pin
the version to the pseudo-version as described in
Azure/azure-storage-blob-go#236 (comment).
stanhu added a commit to stanhu/go-cloud that referenced this issue Dec 18, 2020
The tag for this module got yanked and now checksums are failing. Pin
the version to the pseudo-version as described in
Azure/azure-storage-blob-go#236 (comment).
stanhu added a commit to stanhu/go-cloud that referenced this issue Dec 18, 2020
The tag for this module got yanked and now checksums are failing. Pin
the version to the pseudo-version as described in
Azure/azure-storage-blob-go#236 (comment).
stanhu added a commit to stanhu/go-cloud that referenced this issue Dec 18, 2020
The tag for this module got yanked and now checksums are failing. Pin
the version to the pseudo-version as described in
Azure/azure-storage-blob-go#236 (comment).
@vangent
Copy link

vangent commented Dec 22, 2020

@mohsha-msft any chance for a 12.1 tag soon?

@eccles
Copy link
Author

eccles commented Jan 5, 2021

This issue has become a matter of urgency for us - any chance of a 0.12.1 release which is properly tagged?

@eccles
Copy link
Author

eccles commented Jan 6, 2021

I can confirm that using v0.11.1-0.20201209121048-6df5d9af221d as suggested by @bcmills does apparently work.

@ondrejbudai
Copy link

The workaround works for me too. I would love to see 0.12.1 released though.

ondrejbudai added a commit to ondrejbudai/osbuild-composer that referenced this issue Jan 6, 2021
Fedora 33 and rawhide got an updated version of the azblob library. Sadly, it
introduced a non-compatible API change. This commit does the same thing as
a67baf5 did for kolo/xmlrpc:

We now have two wrappers around the affected part of the API. Fedora 32 uses
the wrapper around the old API, whereas Fedora 33 and 34 (and RHEL with its
vendored deps) use the wrapper around the new API. The switch is implemented
using go build flags and spec file magic.

See a67baf5 for more thoughts.

Also, there's v0.11.1-0.20201209121048-6df5d9af221d in go.mod, why?

The maintainers of azblob probably tagged a wrong commit with v0.12.0 which
breaks go. The long v0.11.1-.* version is basically the proper v0.12.0 commit.
See Azure/azure-storage-blob-go#236 for more
information.
ondrejbudai added a commit to ondrejbudai/osbuild-composer that referenced this issue Jan 6, 2021
Fedora 33 and rawhide got an updated version of the azblob library. Sadly, it
introduced a non-compatible API change. This commit does the same thing as
a67baf5 did for kolo/xmlrpc:

We now have two wrappers around the affected part of the API. Fedora 32 uses
the wrapper around the old API, whereas Fedora 33 and 34 (and RHEL with its
vendored deps) use the wrapper around the new API. The switch is implemented
using go build flags and spec file magic.

See a67baf5 for more thoughts.

Also, there's v0.11.1-0.20201209121048-6df5d9af221d in go.mod, why?

The maintainers of azblob probably tagged a wrong commit with v0.12.0 which
breaks go. The long v0.11.1-.* version is basically the proper v0.12.0 commit.
See Azure/azure-storage-blob-go#236 for more
information.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
ondrejbudai added a commit to ondrejbudai/osbuild-composer that referenced this issue Jan 6, 2021
Fedora 33 and rawhide got an updated version of the azblob library. Sadly, it
introduced a non-compatible API change. This commit does the same thing as
a67baf5 did for kolo/xmlrpc:

We now have two wrappers around the affected part of the API. Fedora 32 uses
the wrapper around the old API, whereas Fedora 33 and 34 (and RHEL with its
vendored deps) use the wrapper around the new API. The switch is implemented
using go build flags and spec file magic.

See a67baf5 for more thoughts.

Also, there's v0.11.1-0.20201209121048-6df5d9af221d in go.mod, why?

The maintainers of azblob probably tagged a wrong commit with v0.12.0 which
breaks go. The long v0.11.1-.* version is basically the proper v0.12.0 commit.
See Azure/azure-storage-blob-go#236 for more
information.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
ondrejbudai added a commit to ondrejbudai/osbuild-composer that referenced this issue Jan 6, 2021
Fedora 33 and rawhide got an updated version of the azblob library. Sadly, it
introduced a non-compatible API change. This commit does the same thing as
a67baf5 did for kolo/xmlrpc:

We now have two wrappers around the affected part of the API. Fedora 32 uses
the wrapper around the old API, whereas Fedora 33 and 34 (and RHEL with its
vendored deps) use the wrapper around the new API. The switch is implemented
using go build flags and spec file magic.

See a67baf5 for more thoughts.

Also, there's v0.11.1-0.20201209121048-6df5d9af221d in go.mod, why?

The maintainers of azblob probably tagged a wrong commit with v0.12.0 which
breaks go. The long v0.11.1-.* version is basically the proper v0.12.0 commit.
See Azure/azure-storage-blob-go#236 for more
information.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
ondrejbudai added a commit to ondrejbudai/osbuild-composer that referenced this issue Jan 6, 2021
Fedora 33 and rawhide got an updated version of the azblob library. Sadly, it
introduced a non-compatible API change. This commit does the same thing as
a67baf5 did for kolo/xmlrpc:

We now have two wrappers around the affected part of the API. Fedora 32 uses
the wrapper around the old API, whereas Fedora 33 and 34 (and RHEL with its
vendored deps) use the wrapper around the new API. The switch is implemented
using go build flags and spec file magic.

See a67baf5 for more thoughts.

Also, there's v0.11.1-0.20201209121048-6df5d9af221d in go.mod, why?

The maintainers of azblob probably tagged a wrong commit with v0.12.0 which
breaks go. The long v0.11.1-.* version is basically the proper v0.12.0 commit.
See Azure/azure-storage-blob-go#236 for more
information.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
ondrejbudai added a commit to osbuild/osbuild-composer that referenced this issue Jan 6, 2021
Fedora 33 and rawhide got an updated version of the azblob library. Sadly, it
introduced a non-compatible API change. This commit does the same thing as
a67baf5 did for kolo/xmlrpc:

We now have two wrappers around the affected part of the API. Fedora 32 uses
the wrapper around the old API, whereas Fedora 33 and 34 (and RHEL with its
vendored deps) use the wrapper around the new API. The switch is implemented
using go build flags and spec file magic.

See a67baf5 for more thoughts.

Also, there's v0.11.1-0.20201209121048-6df5d9af221d in go.mod, why?

The maintainers of azblob probably tagged a wrong commit with v0.12.0 which
breaks go. The long v0.11.1-.* version is basically the proper v0.12.0 commit.
See Azure/azure-storage-blob-go#236 for more
information.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
@zezha-msft
Copy link
Contributor

That's quite a strange issue, we apologize for the inconvenience.

A new release just went out: https://github.com/Azure/azure-storage-blob-go/releases/tag/v0.13.0

Please let us know if you are still encountering problems.

ondrejbudai added a commit to ondrejbudai/osbuild-composer that referenced this issue Jan 29, 2021
Due to Azure/azure-storage-blob-go#236 , we had to
use a weird version of this library (see 1b05192).

A new release came out yesterday that's tagged correctly so let's use it
so we can remove the hack from go.mod.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
@eccles
Copy link
Author

eccles commented Jan 29, 2021

HI - I can confirm that this is fixed by v0.13.0

ondrejbudai added a commit to ondrejbudai/osbuild-composer that referenced this issue Feb 2, 2021
Due to Azure/azure-storage-blob-go#236 , we had to
use a weird version of this library (see 1b05192).

A new release came out yesterday that's tagged correctly so let's use it
so we can remove the hack from go.mod.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
ondrejbudai added a commit to osbuild/osbuild-composer that referenced this issue Feb 3, 2021
Due to Azure/azure-storage-blob-go#236 , we had to
use a weird version of this library (see 1b05192).

A new release came out yesterday that's tagged correctly so let's use it
so we can remove the hack from go.mod.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants