Build Caddyserver from source

[mohammedzee1000]

This PR modified the Caddy server to build from source, instead of downloading binaries from caddyserver.com

[mohammedzee1000]

Sample manual build from work so far:

$ cat /etc/centos-release
CentOS Linux release 7.4.1708 (Core)

$ sh cccp-prebuild.sh 
+ BUILD_DIR=/opt/cccp/caddy-build
+ CADDY_BIN=caddy
+ CADDY_MAIN_SRC_REPO=github.com/mholt/caddy
+ CADDY_MAIN_BUILDS_REPO=github.com/caddyserver/builds
+ CADDY_SRC_BUILD_REMOTE_NAME=caddy_build
+ CADDY_SRC_BUILD_REMOTE_URL=https://github.com/mohammedzee1000/caddy
+ CADDY_SRC_BUILD_REMOTE_BRANCH=centos_release
+ CADDY_PLUGIN_AUTH=github.com/casbin/caddy-authz
+ CADDY_PLUGIN_AWSES=github.com/miquella/caddy-awses
+ CADDY_PLUGIN_PROMETHEUS=github.com/miekg/caddy-prometheus
+ '[' -d /opt/cccp/caddy-build ']'
+ rm -rf /opt/cccp/caddy-build
+ mkdir /opt/cccp/caddy-build
++ pwd
+ CURR=/root/CentOS-Dockerfiles/caddy_server/centos7
+ pushd /opt/cccp/caddy-build
/opt/cccp/caddy-build ~/CentOS-Dockerfiles/caddy_server/centos7
+ yum -y install wget git
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: centos.mirror.net.in
 * epel: kartolo.sby.datautama.net.id
 * extras: centos.excellmedia.net
 * updates: centos.excellmedia.net
Package git-1.8.3.1-12.el7_4.x86_64 already installed and latest version
Resolving Dependencies
--> Running transaction check
---> Package wget.x86_64 0:1.14-15.el7_4.1 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

===========================================================
 Package  Arch       Version             Repository   Size
===========================================================
Installing:
 wget     x86_64     1.14-15.el7_4.1     updates     547 k

Transaction Summary
===========================================================
Install  1 Package

Total download size: 547 k
Installed size: 2.0 M
Downloading packages:
wget-1.14-15.el7_4.1.x86_64.rpm       | 547 kB   00:00     
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : wget-1.14-15.el7_4.1.x86_64             1/1 
  Verifying  : wget-1.14-15.el7_4.1.x86_64             1/1 

Installed:
  wget.x86_64 0:1.14-15.el7_4.1                            

Complete!
+ wget http://cbs.centos.org/kojifiles/packages/golang/1.9.2/4.el7/noarch/golang-src-1.9.2-4.el7.noarch.rpm
--2018-02-23 11:06:39--  http://cbs.centos.org/kojifiles/packages/golang/1.9.2/4.el7/noarch/golang-src-1.9.2-4.el7.noarch.rpm
Resolving cbs.centos.org (cbs.centos.org)... 8.43.84.196
Connecting to cbs.centos.org (cbs.centos.org)|8.43.84.196|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 5388116 (5.1M) [application/x-rpm]
Saving to: ‘golang-src-1.9.2-4.el7.noarch.rpm’

100%[=================>] 5,388,116   1.01MB/s   in 8.8s   

2018-02-23 11:06:49 (601 KB/s) - ‘golang-src-1.9.2-4.el7.noarch.rpm’ saved [5388116/5388116]

+ wget http://cbs.centos.org/kojifiles/packages/golang/1.9.2/4.el7/x86_64/golang-bin-1.9.2-4.el7.x86_64.rpm
--2018-02-23 11:06:49--  http://cbs.centos.org/kojifiles/packages/golang/1.9.2/4.el7/x86_64/golang-bin-1.9.2-4.el7.x86_64.rpm
Resolving cbs.centos.org (cbs.centos.org)... 8.43.84.196
Connecting to cbs.centos.org (cbs.centos.org)|8.43.84.196|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 50994536 (49M) [application/x-rpm]
Saving to: ‘golang-bin-1.9.2-4.el7.x86_64.rpm’

100%[=================>] 50,994,536  3.27MB/s   in 32s    

2018-02-23 11:07:21 (1.51 MB/s) - ‘golang-bin-1.9.2-4.el7.x86_64.rpm’ saved [50994536/50994536]

+ wget http://cbs.centos.org/kojifiles/packages/golang/1.9.2/4.el7/x86_64/golang-1.9.2-4.el7.x86_64.rpm
--2018-02-23 11:07:21--  http://cbs.centos.org/kojifiles/packages/golang/1.9.2/4.el7/x86_64/golang-1.9.2-4.el7.x86_64.rpm
Resolving cbs.centos.org (cbs.centos.org)... 8.43.84.196
Connecting to cbs.centos.org (cbs.centos.org)|8.43.84.196|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 623996 (609K) [application/x-rpm]
Saving to: ‘golang-1.9.2-4.el7.x86_64.rpm’

100%[=================>] 623,996      217KB/s   in 2.8s   

2018-02-23 11:07:25 (217 KB/s) - ‘golang-1.9.2-4.el7.x86_64.rpm’ saved [623996/623996]

+ yum -y install golang-1.9.2-4.el7.x86_64.rpm golang-bin-1.9.2-4.el7.x86_64.rpm golang-src-1.9.2-4.el7.noarch.rpm
Loaded plugins: fastestmirror
Examining golang-1.9.2-4.el7.x86_64.rpm: golang-1.9.2-4.el7.x86_64
Marking golang-1.9.2-4.el7.x86_64.rpm to be installed
Examining golang-bin-1.9.2-4.el7.x86_64.rpm: golang-bin-1.9.2-4.el7.x86_64
Marking golang-bin-1.9.2-4.el7.x86_64.rpm to be installed
Examining golang-src-1.9.2-4.el7.noarch.rpm: golang-src-1.9.2-4.el7.noarch
golang-src-1.9.2-4.el7.noarch.rpm: does not update installed package.
Resolving Dependencies
--> Running transaction check
---> Package golang.x86_64 0:1.9.2-4.el7 will be installed
---> Package golang-bin.x86_64 0:1.9.2-4.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

===========================================================
 Package
   Arch   Version     Repository                      Size
===========================================================
Installing:
 golang
   x86_64 1.9.2-4.el7 /golang-1.9.2-4.el7.x86_64     6.8 M
 golang-bin
   x86_64 1.9.2-4.el7 /golang-bin-1.9.2-4.el7.x86_64 187 M

Transaction Summary
===========================================================
Install  2 Packages

Total size: 193 M
Installed size: 193 M
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : golang-1.9.2-4.el7.x86_64               1/2 
  Installing : golang-bin-1.9.2-4.el7.x86_64           2/2 
  Verifying  : golang-bin-1.9.2-4.el7.x86_64           1/2 
  Verifying  : golang-1.9.2-4.el7.x86_64               2/2 

Installed:
  golang.x86_64 0:1.9.2-4.el7                              
  golang-bin.x86_64 0:1.9.2-4.el7                          

Complete!
+ yum -y remove wget
Loaded plugins: fastestmirror
Resolving Dependencies
--> Running transaction check
---> Package wget.x86_64 0:1.14-15.el7_4.1 will be erased
--> Finished Dependency Resolution

Dependencies Resolved

===========================================================
 Package Arch      Version               Repository   Size
===========================================================
Removing:
 wget    x86_64    1.14-15.el7_4.1       @updates    2.0 M

Transaction Summary
===========================================================
Remove  1 Package

Installed size: 2.0 M
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Erasing    : wget-1.14-15.el7_4.1.x86_64             1/1 
  Verifying  : wget-1.14-15.el7_4.1.x86_64             1/1 

Removed:
  wget.x86_64 0:1.14-15.el7_4.1                            

Complete!
+ yum clean all
Loaded plugins: fastestmirror
Cleaning repos: base epel extras updates
Cleaning up everything
Maybe you want: rm -rf /var/cache/yum, to also free up space taken by orphaned data from disabled or removed repos
Cleaning up list of fastest mirrors
+ export GOPATH=/opt/cccp/caddy-build
+ GOPATH=/opt/cccp/caddy-build
+ go get github.com/mholt/caddy
+ go get github.com/caddyserver/builds
+ go get github.com/casbin/caddy-authz
+ go get github.com/miquella/caddy-awses
+ go get github.com/miekg/caddy-prometheus
+ pushd src/github.com/mholt/caddy
/opt/cccp/caddy-build/src/github.com/mholt/caddy /opt/cccp/caddy-build ~/CentOS-Dockerfiles/caddy_server/centos7
+ git remote add caddy_build https://github.com/mohammedzee1000/caddy
+ git fetch caddy_build
remote: Counting objects: 16, done.
remote: Compressing objects: 100% (11/11), done.
remote: Total 16 (delta 6), reused 15 (delta 5), pack-reused 0
Unpacking objects: 100% (16/16), done.
From https://github.com/mohammedzee1000/caddy
 * [new branch]      caddyfile-conditions -> caddy_build/caddyfile-conditions
 * [new branch]      centos_release -> caddy_build/centos_release
 * [new branch]      diagnostics -> caddy_build/diagnostics
 * [new branch]      dynamic-proxy -> caddy_build/dynamic-proxy
 * [new branch]      empty-name-cert -> caddy_build/empty-name-cert
 * [new branch]      logexcept  -> caddy_build/logexcept
 * [new branch]      master     -> caddy_build/master
 * [new branch]      proxyencode-h -> caddy_build/proxyencode-h
 * [new branch]      quic_issue_1807 -> caddy_build/quic_issue_1807
+ git checkout centos_release
Branch centos_release set up to track remote branch centos_release from caddy_build.
Switched to a new branch 'centos_release'
+ pushd ./caddy
/opt/cccp/caddy-build/src/github.com/mholt/caddy/caddy /opt/cccp/caddy-build/src/github.com/mholt/caddy /opt/cccp/caddy-build ~/CentOS-Dockerfiles/caddy_server/centos7
+ go run build.go
+ cp -avrf ./caddy /root/CentOS-Dockerfiles/caddy_server/centos7/caddy
‘./caddy’ -> ‘/root/CentOS-Dockerfiles/caddy_server/centos7/caddy’
+ popd
/opt/cccp/caddy-build/src/github.com/mholt/caddy /opt/cccp/caddy-build ~/CentOS-Dockerfiles/caddy_server/centos7
+ popd
/opt/cccp/caddy-build ~/CentOS-Dockerfiles/caddy_server/centos7
+ popd
~/CentOS-Dockerfiles/caddy_server/centos7

$ docker build -t test -f Dockerfile .

docker build -t test -f Dockerfile .
Sending build context to Docker daemon 24.08 MB
Step 1 : FROM registry.centos.org/centos/centos
 ---> a292672ef08b
Step 2 : LABEL maintainer "Mohammed Zeeshan Ahmed <moahmed@redhat.com>"
 ---> Using cache
 ---> 2d8f133e9eb6
Step 3 : LABEL VERSION "0.10.11" ARCHITECTURE "amd64"
 ---> Using cache
 ---> 899900fbb3b8
Step 4 : RUN mkdir -p /etc/caddy && useradd -u 1001 -g 0 caddy
 ---> Using cache
 ---> 0a30aa4acdea
Step 5 : ADD ./caddy /usr/bin/caddy
 ---> Using cache
 ---> 5616f61a79e2
Step 6 : RUN chmod 0755 /usr/bin/caddy     && /usr/bin/caddy -version
 ---> Using cache
 ---> d61a02568ee1
Step 7 : RUN mkdir -p /var/www/html && chown -R 1001:0 /var/www && chmod -R 777 /var/www && chmod -R g+s /var/www
 ---> Using cache
 ---> f28c2adcf8e7
Step 8 : ADD Caddyfile /etc/caddy/Caddyfile
 ---> Using cache
 ---> 2c13e03671f0
Step 9 : RUN chown -R 1001:0 /etc/caddy && chmod -R 775 /etc/caddy
 ---> Using cache
 ---> ffd0128efc5d
Step 10 : USER 1001
 ---> Using cache
 ---> 0607df985611
Step 11 : EXPOSE 80 443 2015 8080 8443
 ---> Using cache
 ---> 838acfa337c2
Step 12 : WORKDIR /var/www/html
 ---> Using cache
 ---> f91661ceb061
Step 13 : ENTRYPOINT /usr/bin/caddy
 ---> Using cache
 ---> 5433d23c02e9
Step 14 : CMD -quic --conf /etc/caddy/Caddyfile
 ---> Using cache
 ---> 9cf5e9af669a
Successfully built 9cf5e9af669a

[kbsingh]

@mohammedzee1000 nice!

Have we had a chance to test this container for any behaviour changes compared with the prev one ? we might want to check and document that up a bit.

Also, I see you are using specific files in the build, we need to track when this golang goes either into the distro or the PaaS SIG repos and switch to using it from a repo instead of this direct download at the moment.

otherwise LGTM, just confirm once re behaviour change and we can merge + move forward.

[mohammedzee1000]

The primary behaviour change that i have seen so far is $ caddy -version showing (Unofficial) as part of tag. Other that that, as far as i used it at least, i didn't see much difference.

I will see if i can do more extensive testing :)

Of course, I am not fully aware of all use cases, so if anyone else wants to join in and help test it, they are welcome.

I have temporarily pushed a test container to hub with name as mohammedzee1000/caddy-test:latest

Of course, i agree with getting the go 1.9 rpms into a repository instead of curling it off cbs and we should get that done asap. But thats is just a couple of lines of from my side change :)

@kbsingh ^

[kbsingh]

LGTM