Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: unclear error when SumDB tree note is corrupted #38348

Open
nicolascamilo opened this issue Apr 10, 2020 · 11 comments
Open

cmd/go: unclear error when SumDB tree note is corrupted #38348

nicolascamilo opened this issue Apr 10, 2020 · 11 comments

Comments

@nicolascamilo
Copy link

@nicolascamilo nicolascamilo commented Apr 10, 2020

What version of Go are you using (go version)?

$ go version
go version go1.14.2 windows/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
set GO111MODULE=on
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\Nicolas\AppData\Local\go-build
set GOENV=C:\Users\Nicolas\AppData\Roaming\go\env
set GOEXE=.exe
set GOFLAGS= -mod=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOINSECURE=
set GONOPROXY=
set GONOSUMDB=
set GOOS=windows
set GOPATH=D:\Mis Documentos\Programacion\Go
set GOPRIVATE=
set GOPROXY=direct
set GOROOT=C:\Go
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=C:\Go\pkg\tool\windows_amd64
set GCCGO=gccgo
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=NUL
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=C:\Users\Nicolas\AppData\Local\Temp\go-build045229398=/tmp/go-build -gno-record-gcc-switches

What did you do?

I'm trying to use go mod tidy command. Since my computer turned off because of a blackout it never worked again

What did you expect to see?

Succesful go.mod file

What did you see instead?

For every project and import I'm seeing this error:

github.com/sirupsen/logrus: github.com/sirupsen/logrus@v1.5.0: verifying module: github.com/sirupsen/logrus@v1.5.0: initializing sumdb.Client: reading tree note: malformed note
note:
 github.com/dgrijalva/jwt-go: github.com/dgrijalva/jwt-go@v1.0.2: verifying module: github.com/dgrijalva/jwt-go@v1.0.2: initializing sumdb.Client: reading tree note: malformed note
note:

I know I just copied 2 libraries, but It's happening with every one

@bcmills bcmills added this to the Go1.15 milestone Apr 10, 2020
@bcmills
Copy link
Member

@bcmills bcmills commented Apr 10, 2020

Probably you have a corrupted file somewhere, but that diagnostic isn't giving you enough information to find and fix it.

@FiloSottile, what method would you recommend to repair this, and can we either do that automatically or add detail to the error message?

CC @katiehockman @rsc @jayconrod @matloob

@bcmills bcmills changed the title gomod initializing sumdb.Client: reading tree note: malformed note error cmd/go: unclear error when SumDB tree note is corrupted Apr 10, 2020
@katiehockman
Copy link
Member

@katiehockman katiehockman commented Apr 10, 2020

@nicolascamilo for now in order to unblock you, can you try clearing your module cache to see if that resolves the issue? You can do that by running go clean -modcache

If that fixes your specific problem, then we should still leave this issue open for the investigation that @bcmills mentioned above.

@FiloSottile
Copy link
Member

@FiloSottile FiloSottile commented Apr 10, 2020

@nicolascamilo Can you share with us the contents of $(go env GOPATH)/pkg/sumdb/sum.golang.org/latest? Then you can delete that file and it should unblock you.

$ cat "$(go env GOPATH)/pkg/sumdb/sum.golang.org/latest"
go.sum database tree
1072016
5rxqXjwvPB0445OEz5nw+s/gdsfoR0EUt3FEqMzbtCc=

— sum.golang.org Az3grhjle9b69brGvNYM1ZPKtFcpzaeKwQHomx3L7S5WuvtNRoVIM/KgmpsbfQSMEwqNFmoVR/gizfLYnOhtS7sUJgk=

@bcmills If the corrupted file is GOPATH/pkg/sumdb/sum.golang.org/latest, it would be important to understand how it came to be corrupted, because if an attacker can cause that they can downgrade the security guarantees significantly. If it's any other file, we can blow it away.

@bcmills
Copy link
Member

@bcmills bcmills commented Apr 10, 2020

@FiloSottile

it would be important to understand how it came to be corrupted, because if an attacker can cause that they can downgrade the security guarantees significantly.

Per the original issue description:

Since my computer turned off because of a blackout it never worked again

So I think the scenario here is “local power cut with a non-journaled filesystem”, which I assume is beyond our threat model for the sumdb...

@FiloSottile
Copy link
Member

@FiloSottile FiloSottile commented Apr 10, 2020

Fair enough, totally fine to delete here. I am a bit reticent to have us delete it automatically because we wouldn't hear about other ways it might go corrupt. Let's punt on it until the next time?

@bcmills
Copy link
Member

@bcmills bcmills commented Apr 10, 2020

Ok. Putting on hold until / unless we get more reports.

@bcmills bcmills modified the milestones: Go1.15, Unplanned Apr 10, 2020
@nicolascamilo
Copy link
Author

@nicolascamilo nicolascamilo commented Apr 12, 2020

@nicolascamilo for now in order to unblock you, can you try clearing your module cache to see if that resolves the issue? You can do that by running go clean -modcache

If that fixes your specific problem, then we should still leave this issue open for the investigation that @bcmills mentioned above.

Tried that already, nothing happens.

@nicolascamilo Can you share with us the contents of $(go env GOPATH)/pkg/sumdb/sum.golang.org/latest? Then you can delete that file and it should unblock you.

$ cat "$(go env GOPATH)/pkg/sumdb/sum.golang.org/latest"
go.sum database tree
1072016
5rxqXjwvPB0445OEz5nw+s/gdsfoR0EUt3FEqMzbtCc=

— sum.golang.org Az3grhjle9b69brGvNYM1ZPKtFcpzaeKwQHomx3L7S5WuvtNRoVIM/KgmpsbfQSMEwqNFmoVR/gizfLYnOhtS7sUJgk=

@bcmills If the corrupted file is GOPATH/pkg/sumdb/sum.golang.org/latest, it would be important to understand how it came to be corrupted, because if an attacker can cause that they can downgrade the security guarantees significantly. If it's any other file, we can blow it away.

This is the result:

go.sum database tree
1077497
wBavjUxZoXIpqgvc3T3tEi+XaKQOOgdQ+YP5OR0Anyw=

— sum.golang.org Az3griPfGhP2UeDZ/87y3uE8qKV68nDWey+KoJ3y9se5HyipCsyRzt4vGnxhYrdZfNarDmh62xfaz12cbzztfGMCLQE=

Update: Apparently is working right now. I'd love to provide more information but I have no actually clue what I've done, but I guess it was my problem and not something you guys have to solve. I can't use go mod tidy now with the libs I'm working (I was able before this problem) but I guess that's just another issue.

Again, I'm deeply sorry for I can't provide any new hint about this problem.

@sajallimbu
Copy link

@sajallimbu sajallimbu commented Jul 11, 2020

@nicolascamilo Can you share with us the contents of $(go env GOPATH)/pkg/sumdb/sum.golang.org/latest? Then you can delete that file and it should unblock you.

$ cat "$(go env GOPATH)/pkg/sumdb/sum.golang.org/latest"
go.sum database tree
1072016
5rxqXjwvPB0445OEz5nw+s/gdsfoR0EUt3FEqMzbtCc=

— sum.golang.org Az3grhjle9b69brGvNYM1ZPKtFcpzaeKwQHomx3L7S5WuvtNRoVIM/KgmpsbfQSMEwqNFmoVR/gizfLYnOhtS7sUJgk=

@bcmills If the corrupted file is GOPATH/pkg/sumdb/sum.golang.org/latest, it would be important to understand how it came to be corrupted, because if an attacker can cause that they can downgrade the security guarantees significantly. If it's any other file, we can blow it away.

Deleting the file named latest worked for me!

@FiloSottile
Copy link
Member

@FiloSottile FiloSottile commented Jul 11, 2020

Deleting the file named latest worked for me!

Can you share what error you were seeing?

@mreddimasi
Copy link

@mreddimasi mreddimasi commented Jul 21, 2020

Hi,

Not sure, if it helps, but was facing the same issue, the following steps actually resolved the issue for me (Windows 10 - go version go1.14.4 windows/amd64)

The file %GOPATH%\pkg\sumdb\sum.golang.org\latest did not exist when I was got this error

Steps:

  1. touch %GOPATH%\pkg\sumdb\sum.golang.org\latest
  2. Ran the build got the following output

go: finding module for package github.com/gomodule/redigo/redis                                                                                                                                                          
go: finding module for package github.com/go-sql-driver/mysql                                                                                                                                                            
go: finding module for package github.com/jinzhu/gorm                                                                                                                                                                    
go: finding module for package github.com/google/uuid                                                                                                                                                                    
go: finding module for package github.com/gin-gonic/gin                                                                                                                                                                  
go: finding module for package github.com/dgrijalva/jwt-go                                                                                                                                                               
go: finding module for package github.com/nats-io/nats.go                                                                                                                                                                
go: finding module for package github.com/gin-contrib/cors                                                                                                                                                               
go: downloading github.com/jinzhu/gorm v1.9.15                                                                                                                                                                           
go: downloading github.com/go-sql-driver/mysql v1.5.0                                                                                                                                                                    
go: downloading github.com/google/uuid v1.1.1                                                                                                                                                                            
go: downloading github.com/gin-contrib/cors v1.3.1                                                                                                                                                                       
go: downloading github.com/gomodule/redigo/redis v0.0.0-do-not-use                                                                                                                                                       
go: downloading github.com/gin-gonic/gin v1.6.3                                                                                                                                                                          
go: downloading github.com/dgrijalva/jwt-go v1.0.2                                                                                                                                                                       
go: downloading github.com/gomodule/redigo v1.8.2                                                                                                                                                                        
go: downloading github.com/nats-io/nats.go v1.10.0                                                                                                                                                                       
internal\middlewares\auth.go:12:2: github.com/dgrijalva/jwt-go@v1.0.2: verifying module: github.com/dgrijalva/jwt-go@v1.0.2: initializing sumdb.Client: reading tree note: malformed note                                
note:                                                                                                                                                                                                                    
                                                                                                                                                                                                                         
internal\app\app.go:10:2: github.com/gin-contrib/cors@v1.3.1: verifying module: github.com/gin-contrib/cors@v1.3.1: initializing sumdb.Client: reading tree note: malformed note                                         
note:                                                                                                                                                                                                                    
                                                                                                                                                                                                                         
internal\app\app.go:11:2: github.com/gin-gonic/gin@v1.6.3: verifying module: github.com/gin-gonic/gin@v1.6.3: initializing sumdb.Client: reading tree note: malformed note                                               
note:                                                                                                                                                                                                                    
                                                                                                                                                                                                                         
internal\conf\configuration.go:12:2: github.com/go-sql-driver/mysql@v1.5.0: verifying module: github.com/go-sql-driver/mysql@v1.5.0: initializing sumdb.Client: reading tree note: malformed note                        
note:                                                                                                                                                                                                                    
                                                                                                                                                                                                                         
cmd\merchant-backend\main.go:9:2: github.com/gomodule/redigo/redis@v0.0.0-do-not-use: verifying module: github.com/gomodule/redigo/redis@v0.0.0-do-not-use: initializing sumdb.Client: reading tree note: malformed note 
note:                                                                                                                                                                                                                    
                                                                                                                                                                                                                         
internal\helpers\utils.go:19:2: github.com/google/uuid@v1.1.1: verifying module: github.com/google/uuid@v1.1.1: initializing sumdb.Client: reading tree note: malformed note                                             
note:                                                                                                                                                                                                                    
                                                                                                                                                                                                                         
cmd\merchant-backend\main.go:10:2: github.com/jinzhu/gorm@v1.9.15: verifying module: github.com/jinzhu/gorm@v1.9.15: initializing sumdb.Client: reading tree note: malformed note                                        
note:                                                                                                                                                                                                                    
                                                                                                                                                                                                                         
internal\conf\configuration.go:10:2: github.com/nats-io/nats.go@v1.10.0: verifying module: github.com/nats-io/nats.go@v1.10.0: initializing sumdb.Client: reading tree note: malformed note                              
note:                                                                                                                                                                                                                    

  1. go clean -modcache
  2. Ran go build again, with the same error message as in Step 2.
  3. rm %GOPATH%\pkg\sumdb\sum.golang.org\latest
  4. Ran go build again this time successfully with the following output
go: finding module for package github.com/gomodule/redigo/redis                         
go: finding module for package github.com/jinzhu/gorm                                   
go: finding module for package github.com/go-sql-driver/mysql                           
go: finding module for package github.com/gin-gonic/gin                                 
go: finding module for package github.com/nats-io/nats.go                               
go: finding module for package github.com/dgrijalva/jwt-go                              
go: finding module for package github.com/gin-contrib/cors                              
go: finding module for package github.com/google/uuid                                   
go: downloading github.com/nats-io/nats.go v1.10.0                                      
go: downloading github.com/go-sql-driver/mysql v1.5.0                                   
go: downloading github.com/dgrijalva/jwt-go v1.0.2                                      
go: downloading github.com/gin-contrib/cors v1.3.1                                      
go: downloading github.com/gomodule/redigo v1.8.2                                       
go: downloading github.com/google/uuid v1.1.1                                           
go: downloading github.com/gin-gonic/gin v1.6.3                                         
go: downloading github.com/gomodule/redigo/redis v0.0.0-do-not-use                      
go: downloading github.com/jinzhu/gorm v1.9.15                                          
go: downloading github.com/dgrijalva/jwt-go v3.2.0+incompatible                         
go: found github.com/gomodule/redigo/redis in github.com/gomodule/redigo v1.8.2         
go: found github.com/jinzhu/gorm in github.com/jinzhu/gorm v1.9.15                      
go: found github.com/gin-contrib/cors in github.com/gin-contrib/cors v1.3.1             
go: found github.com/gin-gonic/gin in github.com/gin-gonic/gin v1.6.3                   
go: found github.com/go-sql-driver/mysql in github.com/go-sql-driver/mysql v1.5.0       
go: found github.com/nats-io/nats.go in github.com/nats-io/nats.go v1.10.0              
go: found github.com/dgrijalva/jwt-go in github.com/dgrijalva/jwt-go v3.2.0+incompatible
go: found github.com/google/uuid in github.com/google/uuid v1.1.1                       
go: downloading github.com/mattn/go-isatty v0.0.12                                      
go: downloading github.com/gin-contrib/sse v0.1.0                                       
go: downloading github.com/jinzhu/inflection v1.0.0                                     
go: downloading github.com/ugorji/go v1.1.7                                             
go: downloading github.com/nats-io/nkeys v0.1.4                                         
go: downloading github.com/go-playground/validator/v10 v10.2.0                          
go: downloading gopkg.in/yaml.v2 v2.2.8                                                 
go: downloading github.com/golang/protobuf v1.3.3                                       
go: downloading github.com/ugorji/go/codec v1.1.7                                       
go: downloading github.com/nats-io/jwt v0.3.2                                           
go: downloading github.com/nats-io/nuid v1.0.1                                          
go: downloading golang.org/x/crypto v0.0.0-20200323165209-0ec3e9974c59                  
go: downloading github.com/go-playground/universal-translator v0.17.0                   
go: downloading github.com/leodido/go-urn v1.2.0                                        
go: downloading github.com/go-playground/locales v0.13.0         
```                       ` 

I can see the latest file had been created after this step. 
@YongHaoWu
Copy link

@YongHaoWu YongHaoWu commented Sep 16, 2020

Thanks @mreddimasi . I met this issue, and this does solve the issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
7 participants
You can’t perform that action at this time.