debootstrap fails with distrobuilder 1.1

[christoph-conrads]

My system runs Ubuntu 20.04 with distrobuilder 1.0 and because of #365, I tried to use distrobuilder 1.1 but debootstrap fails with an error message:

$ sudo ~/go/bin/distrobuilder build-lxd ~/lxc-ci/images/debian.yaml 
I: usage: [OPTION]... <suite> <target> [<mirror> [<script>]]
I: Try `debootstrap --help' for more information.
E: You must specify a suite and a target.
Error: Error while downloading source: exit status 1

The issue persists with commit b194aac.

Below are all the steps I made to build distrobuilder 1.1. The image is from the lxc-ci repo.

$ go get -d -v github.com/lxc/distrobuilder/distrobuilder
github.com/lxc/distrobuilder (download)
created GOPATH=/home/christoph/go; see 'go help gopath'
github.com/flosch/pongo2 (download)
github.com/lxc/lxd (download)
github.com/gorilla/websocket (download)
github.com/pkg/errors (download)
get "golang.org/x/sys/unix": found meta tag get.metaImport{Prefix:"golang.org/x/sys", VCS:"git", RepoRoot:"https://go.googlesource.com/sys"} at //golang.org/x/sys/unix?go-get=1
get "golang.org/x/sys/unix": verifying non-authoritative meta tag
golang.org/x/sys (download)
get "gopkg.in/robfig/cron.v2": found meta tag get.metaImport{Prefix:"gopkg.in/robfig/cron.v2", VCS:"git", RepoRoot:"https://gopkg.in/robfig/cron.v2"} at //gopkg.in/robfig/cron.v2?go-get=1
gopkg.in/robfig/cron.v2 (download)
get "gopkg.in/flosch/pongo2.v3": found meta tag get.metaImport{Prefix:"gopkg.in/flosch/pongo2.v3", VCS:"git", RepoRoot:"https://gopkg.in/flosch/pongo2.v3"} at //gopkg.in/flosch/pongo2.v3?go-get=1
gopkg.in/flosch/pongo2.v3 (download)
get "gopkg.in/yaml.v2": found meta tag get.metaImport{Prefix:"gopkg.in/yaml.v2", VCS:"git", RepoRoot:"https://gopkg.in/yaml.v2"} at //gopkg.in/yaml.v2?go-get=1
gopkg.in/yaml.v2 (download)
github.com/gobuffalo/packr (download)
github.com/gobuffalo/packd (download)
github.com/gobuffalo/logger (download)
github.com/sirupsen/logrus (download)
get "golang.org/x/crypto/ssh/terminal": found meta tag get.metaImport{Prefix:"golang.org/x/crypto", VCS:"git", RepoRoot:"https://go.googlesource.com/crypto"} at //golang.org/x/crypto/ssh/terminal?go-get=1
get "golang.org/x/crypto/ssh/terminal": verifying non-authoritative meta tag
golang.org/x/crypto (download)
github.com/karrick/godirwalk (download)
github.com/markbates/errx (download)
github.com/markbates/oncer (download)
github.com/markbates/safe (download)
github.com/mudler/docker-companion (download)
get "gopkg.in/antchfx/htmlquery.v1": found meta tag get.metaImport{Prefix:"gopkg.in/antchfx/htmlquery.v1", VCS:"git", RepoRoot:"https://gopkg.in/antchfx/htmlquery.v1"} at //gopkg.in/antchfx/htmlquery.v1?go-get=1
gopkg.in/antchfx/htmlquery.v1 (download)
github.com/antchfx/xpath (download)
github.com/golang/groupcache (download)
get "golang.org/x/net/html": found meta tag get.metaImport{Prefix:"golang.org/x/net", VCS:"git", RepoRoot:"https://go.googlesource.com/net"} at //golang.org/x/net/html?go-get=1
get "golang.org/x/net/html": verifying non-authoritative meta tag
golang.org/x/net (download)
get "golang.org/x/net/html/charset": found meta tag get.metaImport{Prefix:"golang.org/x/net", VCS:"git", RepoRoot:"https://go.googlesource.com/net"} at //golang.org/x/net/html/charset?go-get=1
get "golang.org/x/net/html/charset": verifying non-authoritative meta tag
get "golang.org/x/text/encoding": found meta tag get.metaImport{Prefix:"golang.org/x/text", VCS:"git", RepoRoot:"https://go.googlesource.com/text"} at //golang.org/x/text/encoding?go-get=1
get "golang.org/x/text/encoding": verifying non-authoritative meta tag
golang.org/x/text (download)
get "golang.org/x/text/transform": found meta tag get.metaImport{Prefix:"golang.org/x/text", VCS:"git", RepoRoot:"https://go.googlesource.com/text"} at //golang.org/x/text/transform?go-get=1
get "golang.org/x/text/transform": verifying non-authoritative meta tag
get "golang.org/x/text/encoding/charmap": found meta tag get.metaImport{Prefix:"golang.org/x/text", VCS:"git", RepoRoot:"https://go.googlesource.com/text"} at //golang.org/x/text/encoding/charmap?go-get=1
get "golang.org/x/text/encoding/charmap": verifying non-authoritative meta tag
get "golang.org/x/text/encoding/htmlindex": found meta tag get.metaImport{Prefix:"golang.org/x/text", VCS:"git", RepoRoot:"https://go.googlesource.com/text"} at //golang.org/x/text/encoding/htmlindex?go-get=1
get "golang.org/x/text/encoding/htmlindex": verifying non-authoritative meta tag
github.com/spf13/cobra (download)
github.com/spf13/pflag (download)
christoph@dell-t1700-428:~$ cd ~/go/src/github.com/lxc/distrobuilder/
christoph@dell-t1700-428:~/go/src/github.com/lxc/distrobuilder$ git checkout distrobuilder-1.1
Note: switching to 'distrobuilder-1.1'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 6bea3a3 Release distrobuilder 1.1
christoph@dell-t1700-428:~/go/src/github.com/lxc/distrobuilder$ make
gofmt -s -w .
go get -t -v -d ./...
github.com/stretchr/testify (download)
github.com/davecgh/go-spew (download)
github.com/pmezard/go-difflib (download)
get "gopkg.in/yaml.v3": found meta tag get.metaImport{Prefix:"gopkg.in/yaml.v3", VCS:"git", RepoRoot:"https://gopkg.in/yaml.v3"} at //gopkg.in/yaml.v3?go-get=1
gopkg.in/yaml.v3 (download)
go install -v ./...
github.com/lxc/lxd/lxd/include
golang.org/x/sys/unix
golang.org/x/crypto/ssh/terminal
github.com/sirupsen/logrus
github.com/lxc/lxd/shared
github.com/lxc/distrobuilder/shared
github.com/lxc/distrobuilder/sources
github.com/lxc/distrobuilder/distrobuilder
distrobuilder built successfully
christoph@dell-t1700-428:~/go/src/github.com/lxc/distrobuilder$ cd ~/lxd-images/
christoph@dell-t1700-428:~/lxd-images$ git -C ~/lxc-ci rev-parse HEAD
6f12da0392fe950fa83bd1d9e2858e76bccd7616
christoph@dell-t1700-428:~/lxd-images$ which debootstrap
/usr/sbin/debootstrap
christoph@dell-t1700-428:~/lxd-images$ debootstrap --version
debootstrap 1.0.118ubuntu1.2
christoph@dell-t1700-428:~/lxd-images$ sudo strace -f -qq --string-limit=128 --trace-path /usr/sbin/debootstrap -b execve --trace=execve ~/go/bin/distrobuilder build-lxd ~/lxc-ci/images/debian.yaml
[sudo] password for christoph: 
[pid 51631] --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=12, si_uid=0, si_status=0, si_utime=0, si_stime=0} ---
[pid 51638] --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=10, si_uid=0, si_status=0, si_utime=0, si_stime=0} ---
[pid 51638] --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=15, si_uid=0, si_status=0, si_utime=0, si_stime=0} ---
[pid 51638] --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=18, si_uid=0, si_status=0, si_utime=0, si_stime=0} ---
[pid 51634] --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=20, si_uid=0, si_status=0, si_utime=0, si_stime=0} ---
[pid 51634] --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=22, si_uid=0, si_status=0, si_utime=0, si_stime=0} ---
[pid 51634] --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=23, si_uid=0, si_status=0, si_utime=0, si_stime=0} ---
[pid 51654] execve("/usr/sbin/debootstrap", ["debootstrap", "--variant", "minbase", "--arch", "amd64", "--keyring", "/tmp/distrobuilder.939447840/distrobuilder.gpg", "", "/var/cache/distrobuilder.323545733/rootfs", "http://deb.debian.org/debian"], 0xc000422480 /* 22 vars */ <detached ...>
I: usage: [OPTION]... <suite> <target> [<mirror> [<script>]]
I: Try `debootstrap --help' for more information.
E: You must specify a suite and a target.
[pid 51646] --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=24, si_uid=0, si_status=1, si_utime=0, si_stime=0} ---
Error: Error while downloading source: exit status 1
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=51631, si_uid=0, si_status=1, si_utime=2, si_stime=0} ---

[stgraber]

That's normal. The YAML files in lxc-ci do not include the architecture or release fields as those are passed by Jenkins.
Pass -o image.architecture=amd64 -o image.release=buster or something along those lines and it should behave much better.

[stgraber]

You can see what arguments we pass for a given image build in the build log available at https://jenkins.linuxcontainers.org or in the build.log file alongside every image in https://images.linuxcontainers.org/images/

[christoph-conrads]

You can see what arguments we pass for a given image build in the build log available at https://jenkins.linuxcontainers.org or in the build.log file alongside every image in https://images.linuxcontainers.org/images/

Thank you, it is working now.

Example from https://uk.images.linuxcontainers.org/images/debian/stretch/amd64/default/20201007_05:24/build.log:

+ distrobuilder --timeout 1800 build-dir image.yaml rootfs -o image.serial=20201007_05:24 -o image.architecture=amd64 -o image.release=stretch -o image.variant=default -o source.url=http://ftp.us.debian.org/debian