Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to build Caddy plugin #98

Closed
polarathene opened this issue Aug 13, 2021 · 12 comments · Fixed by #109
Closed

Unable to build Caddy plugin #98

polarathene opened this issue Aug 13, 2021 · 12 comments · Fixed by #109

Comments

@polarathene
Copy link

Fails to build with this Dockerfile:

FROM caddy:2-builder AS builder

RUN xcaddy build \
    --with github.com/darkweak/souin/plugins/caddy

FROM caddy:2

COPY --from=builder /usr/bin/caddy /usr/bin/caddy

Error output suggests a checksum mismatch:

Step 1/4 : FROM caddy:2-builder AS builder
 ---> f64009d1a35a
Step 2/4 : RUN xcaddy build     --with github.com/darkweak/souin/plugins/caddy
 ---> Running in 2182385757d5
2021/08/13 11:29:17 [INFO] Temporary folder: /tmp/buildenv_2021-08-13-1129.228235882
2021/08/13 11:29:17 [INFO] Writing main module: /tmp/buildenv_2021-08-13-1129.228235882/main.go                                                                                                                                                        
2021/08/13 11:29:17 [INFO] Initializing Go module                                                                                                                                                                                                      
2021/08/13 11:29:17 [INFO] exec (timeout=10s): /usr/local/go/bin/go mod init caddy                                                                                                                                                                     
go: creating new go.mod: module caddy                                                                                                                                                                                                                  
go: to add module requirements and sums:                                                                                                                                                                                                               

2021/08/13 11:29:17 [INFO] Pinning versions                                                                                                                                                                                                            
2021/08/13 11:29:17 [INFO] exec (timeout=0s): /usr/local/go/bin/go get -d -v github.com/caddyserver/caddy/v2@v2.4.3                                                                                                                                    
go: downloading github.com/caddyserver/caddy/v2 v2.4.3                                                                                                                                                                                                 
go: downloading github.com/caddyserver/certmagic v0.14.0                                                                                                                                                                                               
go: downloading github.com/google/uuid v1.2.0                                                                                                                                                                                                          
go: downloading github.com/prometheus/client_golang v1.10.1-0.20210603120351-253906201bda                                                                                                                                                              
go: downloading go.uber.org/zap v1.17.0                                                                                                                                                                                                                
go: downloading golang.org/x/term v0.0.0-20210503060354-a79de5458b56                                                                                                                                                                                   
go: downloading github.com/klauspost/cpuid v1.2.0                                                                                                                                                                                                      
go: downloading github.com/libdns/libdns v0.2.1                                                                                                                                                                                                        
go: downloading github.com/mholt/acmez v0.1.3                                                                                                                                                                                                          
go: downloading github.com/miekg/dns v1.1.42                                                                                                                                                                                                           
go: downloading golang.org/x/crypto v0.0.0-20210513164829-c07d793c2f9a                                                                                                                                                                                 
go: downloading golang.org/x/net v0.0.0-20210525063256-abc453219eb5                                                                                                                                                                                    
go: downloading github.com/klauspost/cpuid/v2 v2.0.6                                                                                                                                                                                                   
go: downloading github.com/beorn7/perks v1.0.1                                                                                                                                                                                                         
go: downloading github.com/cespare/xxhash v1.1.0                                                                                                                                                                                                       
go: downloading github.com/golang/protobuf v1.5.2                                                                                                                                                                                                      
go: downloading github.com/cespare/xxhash/v2 v2.1.1                                                                                                                                                                                                    
go: downloading github.com/prometheus/client_model v0.2.0                                                                                                                                                                                              
go: downloading github.com/prometheus/common v0.26.0                                                                                                                                                                                                   
go: downloading github.com/prometheus/procfs v0.6.0                                                                                                                                                                                                    
go: downloading golang.org/x/sys v0.0.0-20210603081109-ebe580a85c40                                                                                                                                                                                    
go: downloading go.uber.org/atomic v1.7.0                                                                                                                                                                                                              
go: downloading go.uber.org/multierr v1.6.0                                                                                                                                                                                                            
go: downloading google.golang.org/protobuf v1.26.0                                                                                                                                                                                                     
go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.1                                                                                                                                                                                
go: downloading golang.org/x/text v0.3.6                                                                                                                                                                                                               
go get: added github.com/caddyserver/caddy/v2 v2.4.3                                                                                                                                                                                                   
2021/08/13 11:29:52 [INFO] exec (timeout=0s): /usr/local/go/bin/go get -d -v github.com/darkweak/souin/plugins/caddy                                                                                                                                   
go: downloading github.com/darkweak/souin/plugins/caddy v0.0.0-20210811115113-01e06b75a576                                                                                                                                                             
go: downloading github.com/darkweak/souin v1.5.2                                                                                                                                                                                                       
github.com/darkweak/souin/plugins/caddy: github.com/darkweak/souin@v1.5.2: verifying module: checksum mismatch                                                                                                                                         



SECURITY ERROR                                                                                                                                                                                                                                         
This download does NOT match the one reported by the checksum server.                                                                                                                                                                                  
The bits may have been replaced on the origin server, or an attacker may                                                                                                                                                                               
have intercepted the download attempt.                                                                                                                                                                                                                 

For more information, see 'go help module-auth'.                                                                                                                                                                                                       

2021/08/13 11:30:04 [FATAL] exit status 1                                                                                                                                                                                                              

Not sure if it's related but the plugin seems to also reference a beta release of Caddy 2.4, whereas we're up to 2.4.3 now:

https://github.com/Darkweak/Souin/blob/01e06b75a5768c0c04d9b6e27aa7ffa94514bd4f/plugins/caddy/go.mod#L6

@darkweak
Copy link
Owner

I'm on it 🚀✌️

@polarathene
Copy link
Author

polarathene commented Aug 17, 2021

UPDATE: In the meantime, I have resolved this by additional --with args to xcaddy.

  1. Commit hash pinning for darkweak/souin dependency.
  2. Overriding the RC version of Badger v2 to use the latest v2 release. Presumably this version of Badger (or v1) are not used by Souin anymore and require housekeeping to remove old unused dependencies like these.

@darkweak did you test a successful build? It might help to add a github workflow to catch this xcaddy build failure in future?

ARG CADDY_VERSION=2.4.3
FROM caddy:${CADDY_VERSION}-builder AS builder
ARG SOUIN_VERSION=@ed8b9e9fd2d4

RUN xcaddy build \
    --with github.com/darkweak/souin/plugins/caddy${SOUIN_VERSION}

FROM caddy:${CADDY_VERSION}-alpine

COPY --from=builder /usr/bin/caddy /usr/bin/caddy

I have tried without appending a version to the --with statement, with @v1.5.2 and the latest commit hash (@ed8b9e9fd2d4), none works.

I continue to get a verifying module: checksum mismatch on github.com/darkweak/souin@1.5.2. I'm not familiar with Go or xcaddy and Caddy modules, but assume tagging a new version and having the caddy plugin go.mod reference that might help?

@polarathene
Copy link
Author

Workaround by adding a --with arg to xcaddy for darkweak/souin dependency to also use the same git commit instead. Now experiencing an error related to badger:

2021/08/18 00:24:04 [INFO] exec (timeout=0s): /usr/local/go/bin/go build -o /usr/bin/caddy -ldflags -w -s -trimpath                                                                                                         
# github.com/dgraph-io/badger/v2/table                                                                                                                                                                                      
/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.0.1-rc1.0.20201003150343-5d1bab4fc658/table/builder.go:101:23: not enough arguments in call to z.Calloc                                                                       


/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.0.1-rc1.0.20201003150343-5d1bab4fc658/table/builder.go:235:20: not enough arguments in call to z.Calloc                                                                       


/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.0.1-rc1.0.20201003150343-5d1bab4fc658/table/builder.go:479:17: not enough arguments in call to z.Calloc                                                                       


/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.0.1-rc1.0.20201003150343-5d1bab4fc658/table/builder.go:512:18: not enough arguments in call to z.Calloc                                                                       


/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.0.1-rc1.0.20201003150343-5d1bab4fc658/table/builder.go:516:18: not enough arguments in call to z.Calloc                                                                       


/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.0.1-rc1.0.20201003150343-5d1bab4fc658/table/table.go:849:18: not enough arguments in call to z.Calloc                                                                         


/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.0.1-rc1.0.20201003150343-5d1bab4fc658/table/table.go:851:18: not enough arguments in call to z.Calloc                                                                         


/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.0.1-rc1.0.20201003150343-5d1bab4fc658/table/table.go:860:17: not enough arguments in call to z.Calloc                                                                         


2021/08/18 00:24:47 [INFO] Skipping cleanup as requested; leaving folder intact: /tmp/buildenv_2021-08-18-0022.037374093                                                                                                    
2021/08/18 00:24:47 [FATAL] exit status 2

This version of badger is an RC release...why?

This version of badger had an official release back in Jan 2020, there's been multiple releases since, and since Jan 2021; v3 (not compatible with v2).


Looking through the repo a bit and the rest of the xcaddy build output, it seems that while Badger v3 is present, you still have references to v1 and v2? From what I can make of my build output, when building the latest commit for the caddy plugin, all 3 versions of Badger with downloaded, this is supposedly because of being referenced here?:

https://github.com/Darkweak/Souin/blob/ed8b9e9fd2d46fd1db317b72ccc1fdaa811abf03/plugins/caddy/go.sum#L161-L166

Can that be fixed by removing those lines if they're not needed? If so perhaps you have a way to cleanup/remove old content like that which shouldn't be part of the codebase/dependencies anymore?

I'm not familiar with package management in Go or committing to a repo, in NodeJS and Rust we don't commit dependencies like you seem to with vendor. It made for very big noisy diffs in your PRs :(

In the meantime, as a user I am able to finally get a successful build by also telling xcaddy to just use the latest v2, adding v3 in the same manner doesn't seem to do anything, presumably because Souin presently references all 3 major versions of Badger?

ARG CADDY_VERSION=2.4.3
FROM caddy:${CADDY_VERSION}-builder AS builder
ARG SOUIN_VERSION=@ed8b9e9fd2d4

RUN xcaddy build \
    --with github.com/darkweak/souin${SOUIN_VERSION} \
    --with github.com/dgraph-io/badger/v2 \
    --with github.com/darkweak/souin/plugins/caddy${SOUIN_VERSION}

FROM caddy:${CADDY_VERSION}-alpine

COPY --from=builder /usr/bin/caddy /usr/bin/caddy

@darkweak darkweak reopened this Aug 18, 2021
@darkweak
Copy link
Owner

I'll add a Github action to ensure it can be built

@darkweak
Copy link
Owner

Done https://github.com/Darkweak/Souin/actions/runs/1144408675.
I add the workflows for other plugins too before closing this issue.

@darkweak
Copy link
Owner

Every build are tested now, I'm waiting for the merge to be sure everything is ok and we will tag a new version with these changes.

@polarathene
Copy link
Author

polarathene commented Aug 18, 2021

Awesome work @darkweak ! 😀

I believe you can revert the v3 downgrade of Badger and builds should pass still. Would be good to resolve that before tagging the new release.

Seems the RC v2 badger dependency was on a Caddy dependency for smallstep/nosql, so long as that is updated to a recent update as detailed in my review for #109 there should be no build failure anymore 👍

@darkweak
Copy link
Owner

We have to wait for the patch on the badger side 😁

@polarathene
Copy link
Author

@darkweak which patch? Or did you mean the Caddy --> smallstep/nosql --> Badger v2 dependency update? (Caddy 2.4.4 when released)

@darkweak
Copy link
Owner

Yes that's what I mean, and the PR from @francislavoie must be merge in Badger too

@darkweak
Copy link
Owner

Reopened to keep in mind we have to bump deps when everything will be ready on badger/caddy sides

@darkweak darkweak reopened this Aug 21, 2021
@darkweak
Copy link
Owner

Should be okay now, reopen if the problem persists (or open a new issue) :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants