Trouble building with xcaddy - both 2.5.3 and 2.6.4

[kdevan]

I'm running into issues trying to build with this plugin. I'm hoping to use caddy latest which as of writing this issue is 2.6.4.

First I tried xcaddy build --with github.com/tailscale/caddy-tailscale and got the error panic: internal error: can't find reason for requirement on github.com/google/pprof.

That led me to this issue which clarifies that this is an upstream Golang issue: caddyserver/caddy#5301

I tried the tip from this comment caddyserver/caddy#5301 (comment) from mholt to try and build using v2.6.4 as well as hopefully solve the above problem.

xcaddy build \
  --with github.com/tailscale/caddy-tailscale \
  --with github.com/caddyserver/caddy/v2=github.com/caddyserver/caddy/v2@v2.6.4

That leads to a new error message go: github.com/caddyserver/caddy/v2@2.6.4: invalid version: unknown revision 2.6.4 which is similar to #9. Although I think in my case the cause is described by this stack overflow comment since 2.6.4 probably didn't exist when this plugin was built.

At this point I'm going to fork and try to build this plugin using 2.6.4 following these instructions: caddyserver/caddy#5301 (comment)

If I can't get 2.6.4 to build I'll try using the version listed in the dependencies 2.5.3. I'll update here with my findings.

Any help is much appreciated.

[kdevan]

git clone https://github.com/tailscale/caddy-tailscale.git

cd /app/caddy-tailscale

go clean -modcache
go get github.com/caddyserver/caddy/v2@v2.6.4
go mod tidy

xcaddy build \
  --with github.com/tailscale/caddy-tailscale=/app/caddy-tailscale \
  --with github.com/caddyserver/caddy/v2=github.com/caddyserver/caddy/v2@v2.6.4

I tried doing this to get rid of the invalid version: unknown revision 2.6.4 and it worked but then runs into a new error: golang.org/x/crypto/internal/subtle: module golang.org/x/crypto@latest found (v0.6.0), but does not contain package golang.org/x/crypto/internal/subtle.

Seems like the solution to these kinds of issues is to do a custom Caddy build so that Go can synchronize dependencies between Caddy and its plugins. Going to try that next.

[kdevan]

Here's at least one way to run this plugin with caddy 2.6.4 using a manual build and Dockerfile.

Copy main.go as instructed in the comment in main.go. Also run go mod init caddy before building the dockerfile. You need to be able to copy these files to the dockerfile or use a volume if using docker compose.

main.go

// Copyright 2015 Matthew Holt and The Caddy Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

// Package main is the entry point of the Caddy application.
// Most of Caddy's functionality is provided through modules,
// which can be plugged in by adding their import below.
//
// There is no need to modify the Caddy source code to customize your
// builds. You can easily build a custom Caddy with these simple steps:
//
//  1. Copy this file (main.go) into a new folder
//  2. Edit the imports below to include the modules you want plugged in
//  3. Run `go mod init caddy`
//  4. Run `go install` or `go build` - you now have a custom binary!
//
// Or you can use xcaddy which does it all for you as a command:
// https://github.com/caddyserver/xcaddy
package main

import (
	caddycmd "github.com/caddyserver/caddy/v2/cmd"

	// plug in Caddy modules here
	_ "github.com/caddyserver/caddy/v2/modules/standard"
	_ "github.com/tailscale/caddy-tailscale"
)

func main() {
	caddycmd.Main()
}

This is all you need in go.mod. The rest of the require statements will be set by the go get commands in the Dockerfile. go mod init caddy should create this file but you can double check that it's set to go 1.19.

go.mod

module caddy

go 1.19

Dockerfile

FROM golang:1.19-bullseye as builder

# https://github.com/caddyserver/caddy/releases
ENV CADDY_VERSION v2.6.4

WORKDIR /app/caddy

COPY ./where/you/copied/main.go/on/your/local /app/caddy

RUN go get github.com/caddyserver/caddy/v2/cmd
RUN go get github.com/caddyserver/caddy/v2/modules/standard
RUN go get github.com/tailscale/caddy-tailscale

RUN go build

FROM bitnami/minideb:bullseye

RUN mkdir --parents \
  /config/caddy \
  /data/caddy/caddy \
  /etc/caddy \
  /usr/share/caddy \
  ;

COPY --from=builder /app/caddy/caddy /usr/bin/caddy

COPY ./path/to/local/caddy.json /config/caddy/caddy.json

# https://github.com/caddyserver/caddy/releases
ENV CADDY_VERSION v2.6.4

# See https://caddyserver.com/docs/conventions#file-locations for details
ENV XDG_CONFIG_HOME /config
ENV XDG_DATA_HOME /data/caddy

EXPOSE 8080 8443 2019

WORKDIR /test

CMD ["caddy", "run", "--config", "/config/caddy/caddy.json", "--watch"]

And then if you used go mod init caddy you'll see the caddy binary in there.

#14 [8/8] RUN ls -al
#14 0.472 total 74348
#14 0.472 drwxr-xr-x 1 root root     4096 Mar  4 07:03 .
#14 0.472 drwxr-xr-x 1 root root     4096 Mar  4 06:34 ..
#14 0.472 -rwxr-xr-x 1 root root 75976074 Mar  4 07:03 caddy
#14 0.472 -rw-r--r-- 1 root root     9619 Mar  4 07:02 go.mod
#14 0.472 -rw-r--r-- 1 root root   126558 Mar  4 06:13 go.sum
#14 0.472 -rw-rw-r-- 1 root root     1507 Mar  4 06:26 main.go

I used go version 1.19 because 1.20 ran into issues with the gvisor dependency from caddy-tailscale that didn't support it.

Pretty awesome that the build system is so flexible. Really cool system Caddy has for doing custom builds, can't wait to mess with that more now hah.

[gbraad]

We're you successful to build with 2.6.4? If so, consider a PR

[kdevan]

Yes works with 2.6.4 and latest Tailscale 1.36.2. Updating the module itself would be a bit of a different path but I agree that would be great. If I have time I can see if a quick test down that path works right away or has issues. By using the main.go file and entering the modules there, the Go build system automatically syncs dependencies.