Skip to content

Commit

Permalink
[BUILD] Minimize docker container sizes
Browse files Browse the repository at this point in the history
We use two techniques:

1) the use of a minimal docker base (FROM scratch + busybox)
2) the use of a static binary

 ... to create a minimally sized image for 'peer' and 'orderer'

Before this patch, these containers are approximately 1.4GB.  After this
patch, they are about 20MB-24MB.

It isn't strictly necessary to include busybox.  The main benefit
is achieved simply by eliminating external dependencies in the
golang binary using -static and then getting rid of all the
bloat in the baseimage via "FROM scratch".  However, in this mode
the image is pathologically bare-boned.  For instance, the image has
to be launched using the exec-form '["peer", "node", "start"]' since
there is no shell interpreter available to do the more natural
"CMD peer node start".  Further, any "docker exec" style debugging
would be impossible.  It is often helpful to jump into a container
and poke around with tools like ifconfig, ping, netstat, etc.

Enter busybox: We can create a basic unix environment with only
a 5MB payload.  This is impressive and is easily worth its weight
in the image.

However, the challenge isn't really justifying the utility of having
busybox over saving 5MB as much as it is about how we will get it
into the image.  If the world were a monochrome x86_64, we could simply
s/FROM scratch/FROM busybox and be done.  However, we have to consider
other multi $arch.  To support this, we forgo the temptation to use
FROM busybox and build busybox from source.  On my 2011 Macbook Pro,
this adds about 5 minutes to the build, at least on the first build.
Subsequent builds utilize the cache in ./build and thus are no-ops.

This is _just_ fast enough that I am not embarrassed to propose it
for consideration.  However, if this is perceived as a problem
we do have alternatives.  For instance, we could start distributing
a multi-$arch busybox base (hyperledger/fabric-busybox:$arch), TBD.

Change-Id: I4ed20a429c2cc2e72fd602b45c5c8dd5548bc995
Signed-off-by: Greg Haskins <gregory.haskins@gmail.com>
  • Loading branch information
ghaskins committed Nov 30, 2016
1 parent 9bd4e85 commit b61723f
Show file tree
Hide file tree
Showing 5 changed files with 51 additions and 7 deletions.
21 changes: 16 additions & 5 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ endif
PKGNAME = github.com/$(PROJECT_NAME)
GO_LDFLAGS = -X $(PKGNAME)/metadata.Version=$(PROJECT_VERSION)
CGO_FLAGS = CGO_CFLAGS=" "
GO_DOCKER_FLAGS= -ldflags "$(GO_LDFLAGS) -linkmode external -extldflags '-static -lpthread'"
ARCH=$(shell uname -m)
OS=$(shell uname)
CHAINTOOL_RELEASE=v0.10.0
Expand Down Expand Up @@ -100,7 +101,7 @@ GOSHIM_DEPS = $(shell ./scripts/goListFiles.sh $(PKGNAME)/core/chaincode/shim |
JAVASHIM_DEPS = $(shell git ls-files core/chaincode/shim/java)
PROTOS = $(shell git ls-files *.proto | grep -v vendor)
PROJECT_FILES = $(shell git ls-files)
IMAGES = peer orderer ccenv javaenv testenv
IMAGES = peer orderer ccenv javaenv testenv runtime

pkgmap.peer := $(PKGNAME)/peer
pkgmap.orderer := $(PKGNAME)/orderer
Expand Down Expand Up @@ -160,12 +161,12 @@ linter: testenv
build/docker/bin/%: $(PROJECT_FILES)
$(eval TARGET = ${patsubst build/docker/bin/%,%,${@}})
@echo "Building $@"
@mkdir -p build/docker/bin build/docker/pkg
@mkdir -p build/docker/bin build/docker/$(TARGET)/pkg
@$(DRUN) \
-v $(abspath build/docker/bin):/opt/gopath/bin \
-v $(abspath build/docker/pkg):/opt/gopath/pkg \
-v $(abspath build/docker/$(TARGET)/pkg):/opt/gopath/pkg \
hyperledger/fabric-baseimage:$(BASE_DOCKER_TAG) \
go install -ldflags "$(GO_LDFLAGS)" $(pkgmap.$(@F))
go install $(GO_DOCKER_FLAGS) $(pkgmap.$(@F))
@touch $@

build/bin:
Expand All @@ -181,10 +182,19 @@ build/docker/gotools: gotools/Makefile
hyperledger/fabric-baseimage:$(BASE_DOCKER_TAG) \
make install BINDIR=/opt/gotools/bin OBJDIR=/opt/gotools/obj

# Both peer and peer-docker depend on ccenv and javaenv (all docker env images it supports)
build/docker/busybox:
@$(DRUN) \
hyperledger/fabric-baseimage:$(BASE_DOCKER_TAG) \
make -f busybox/Makefile install BINDIR=$(@D)

# Both peer and peer-docker depend on ccenv and javaenv (all docker env images it supports).
build/bin/peer: build/image/ccenv/.dummy build/image/javaenv/.dummy
build/image/peer/.dummy: build/image/ccenv/.dummy build/image/javaenv/.dummy

# Both peer-docker and orderer-docker depend on the runtime image
build/image/peer/.dummy: build/image/runtime/.dummy
build/image/orderer/.dummy: build/image/runtime/.dummy

build/bin/%: $(PROJECT_FILES)
@mkdir -p $(@D)
@echo "$@"
Expand All @@ -205,6 +215,7 @@ build/image/peer/payload: build/docker/bin/peer \
build/image/orderer/payload: build/docker/bin/orderer \
orderer/orderer.yaml
build/image/testenv/payload: build/gotools.tar.bz2
build/image/runtime/payload: build/docker/busybox

build/image/%/payload:
mkdir -p $@
Expand Down
27 changes: 27 additions & 0 deletions busybox/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
BUSYBOX_VER=1.25.1
BUSYBOX_URL=https://www.busybox.net/downloads/busybox-$(BUSYBOX_VER).tar.bz2

OBJDIR=build/busybox-$(BUSYBOX_VER)

all: $(OBJDIR)/busybox

install: $(BINDIR)/busybox

$(BINDIR)/busybox: $(OBJDIR)/busybox
mkdir -p $(@D)
cp $< $@

$(OBJDIR)/.source:
mkdir -p $(@D)
curl -L $(BUSYBOX_URL) | (cd $(@D); tar --strip-components=1 -jx)
touch $@

$(OBJDIR)/.config: $(OBJDIR)/.source
make -C $(@D) defconfig

$(OBJDIR)/busybox: Makefile $(OBJDIR)/.config
make -C $(@D) -l 2.5 -j all LDFLAGS=-static

clean:
-rm -rf $(OBJDIR)

2 changes: 1 addition & 1 deletion images/orderer/Dockerfile.in
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM hyperledger/fabric-baseimage:_BASE_TAG_
FROM hyperledger/fabric-runtime:_TAG_
ENV ORDERER_CFG_PATH /etc/hyperledger/fabric
RUN mkdir -p /var/hyperledger/db /etc/hyperledger/fabric
COPY payload/orderer /usr/local/bin
Expand Down
2 changes: 1 addition & 1 deletion images/peer/Dockerfile.in
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM hyperledger/fabric-baseimage:_BASE_TAG_
FROM hyperledger/fabric-runtime:_TAG_
ENV PEER_CFG_PATH /etc/hyperledger/fabric
RUN mkdir -p /var/hyperledger/db $PEER_CFG_PATH/msp
COPY payload/peer /usr/local/bin
Expand Down
6 changes: 6 additions & 0 deletions images/runtime/Dockerfile.in
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
FROM scratch
COPY payload/busybox /bin/busybox
RUN ["/bin/busybox", "mkdir", "-p", "/usr/bin", "/sbin", "/usr/sbin"]
RUN ["/bin/busybox", "--install"]
RUN mkdir -p /usr/local/bin
ENV PATH=$PATH:/usr/local/bin

0 comments on commit b61723f

Please sign in to comment.