Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support multi-stage builds for Management & Treatment Services Dockerfile #66

Merged
merged 20 commits into from
Mar 14, 2023
Merged

Support multi-stage builds for Management & Treatment Services Dockerfile #66

Show file tree
Hide file tree
Changes from 19 commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
40190ad
Update management dockerfile
terryyylim Mar 6, 2023
93501fe
Make binary executable
terryyylim Mar 7, 2023
d4bcd3c
Add debugging lines
terryyylim Mar 7, 2023
d3b1cf8
Combine chmod
terryyylim Mar 7, 2023
5ed7faa
Revert last change
terryyylim Mar 7, 2023
798bbaf
Update Treatment Service Dockerfile
terryyylim Mar 8, 2023
8ffcfbd
Remove additional updates
terryyylim Mar 8, 2023
b093657
Update Dockerfile deps
terryyylim Mar 8, 2023
b44bf53
Update ln
terryyylim Mar 8, 2023
ece9a9b
Use debian
terryyylim Mar 8, 2023
7ba76d5
Just use ubuntu
terryyylim Mar 8, 2023
fe4bf47
Add multi-stage build
terryyylim Mar 9, 2023
c9f045f
Update workflow versions
terryyylim Mar 9, 2023
6752273
Update CMD...
terryyylim Mar 9, 2023
0f327a4
Update Treatment Service Dockerfile and deps
terryyylim Mar 10, 2023
f49a433
Make Management Service Dockerfile multi-step build
terryyylim Mar 10, 2023
2b6c32a
Remove binary download step in release workflow
terryyylim Mar 10, 2023
da86605
Copy API specs
terryyylim Mar 10, 2023
2b80ed7
Standardize schema file copy
terryyylim Mar 11, 2023
59f7ab4
Address PR comments
terryyylim Mar 14, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ jobs:
api-version: ${{ steps.build-image.outputs.api-version }}
steps:
- name: Check out code
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
fetch-depth: 0

Expand Down Expand Up @@ -116,7 +116,7 @@ jobs:
api-version: ${{ steps.build-image.outputs.api-version }}
steps:
- name: Check out code
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
fetch-depth: 0

Expand Down
10 changes: 5 additions & 5 deletions .github/workflows/xp.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ jobs:
env:
PYTHON: 3.7
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3
- name: Setup Python
id: setup-python
uses: actions/setup-python@v2
Expand All @@ -56,21 +56,21 @@ jobs:
go-version: ${{ env.GO_VERSION }}
- uses: actions/checkout@v3
- name: Lint Common module
uses: golangci/golangci-lint-action@v2
uses: golangci/golangci-lint-action@v3
with:
version: ${{ env.GO_LINT_VERSION }}
working-directory: common
args: --timeout 3m --verbose --skip-files=testutils/mocks/ManagementClientInterface.go
- name: Lint Management Service module
uses: golangci/golangci-lint-action@v2
uses: golangci/golangci-lint-action@v3
with:
version: ${{ env.GO_LINT_VERSION }}
working-directory: management-service
skip-pkg-cache: true
skip-build-cache: true
args: --timeout 3m --verbose
- name: Lint Treatment Service module
uses: golangci/golangci-lint-action@v2
uses: golangci/golangci-lint-action@v3
with:
version: ${{ env.GO_LINT_VERSION }}
working-directory: treatment-service
Expand Down Expand Up @@ -221,7 +221,7 @@ jobs:
outputs:
release-type: ${{ steps.release-rules.outputs.release-type }}
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3
- id: release-rules
uses: ./.github/actions/release-rules

Expand Down
22 changes: 20 additions & 2 deletions Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,19 @@
FROM golang:1.18-alpine as api-builder
ARG API_BIN_NAME=xp-management

ENV GO111MODULE=on \
GOOS=linux \
GOARCH=amd64

WORKDIR /app
COPY . .

# Build Management Service binary
WORKDIR /app/management-service
RUN go build \
-mod=vendor \
-o ./bin/${API_BIN_NAME}

FROM alpine:latest

# Install bash
Expand All @@ -19,8 +35,10 @@ RUN addgroup -S ${XP_USER_GROUP} \
&& mkdir /app \
&& chown -R ${XP_USER}:${XP_USER_GROUP} /app

COPY --chown=${XP_USER}:${XP_USER_GROUP} management-service/bin/* /app/
COPY --chown=${XP_USER}:${XP_USER_GROUP} management-service/database /app/database/
COPY --chown=${XP_USER}:${XP_USER_GROUP} management-service/bin/experiments.yaml /app/
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Can we do COPY --chown=${XP_USER}:${XP_USER_GROUP} management-service/bin/*.yaml /app/ instead of 2 separate commands to copy the OpenAPI specs?

COPY --chown=${XP_USER}:${XP_USER_GROUP} management-service/bin/schema.yaml /app/
COPY --from=api-builder --chown=${XP_USER}:${XP_USER_GROUP} /app/management-service/bin/* /app/
COPY --from=api-builder --chown=${XP_USER}:${XP_USER_GROUP} /app/management-service/database /app/database/

USER ${XP_USER}
WORKDIR /app
Expand Down
4 changes: 2 additions & 2 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ lint: lint-python lint-go
.PHONY: vendor
vendor:
@echo "Fetching dependencies..."
go mod vendor
cd management-service && go mod vendor
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probabl also add commands to vendor the common and clients modules before vendoring the management service.


.PHONY: version
version:
Expand Down Expand Up @@ -171,7 +171,7 @@ build-treatment-service: version
build: build-management-service build-treatment-service

.PHONY: build-image
build-image: version
build-image: version vendor
@$(eval IMAGE_TAG = $(if $(DOCKER_REGISTRY),$(DOCKER_REGISTRY)/,)${BIN_NAME}:${VERSION})
@echo "Building docker image: ${IMAGE_TAG}"
docker build --tag ${IMAGE_TAG} .
Expand Down
13 changes: 13 additions & 0 deletions management-service/appcontext/appcontext.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
package appcontext

import (
"log"

"github.com/pkg/errors"
"gorm.io/gorm"

Expand Down Expand Up @@ -30,36 +32,47 @@ func NewAppContext(db *gorm.DB, authorizer *mw.Authorizer, cfg *config.Config) (
}

// Init Services
log.Println("Initializing message queue publisher...")
messageQueueService, err := messagequeue.NewMessageQueueService(*cfg.MessageQueueConfig)
if err != nil {
return nil, err
}

log.Println("Initializing segmenter service...")
segmenterSvc, err := services.NewSegmenterService(&allServices, cfg.SegmenterConfig, db)
if err != nil {
return nil, err
}

log.Println("Initializing validation service...")
validationService, err := services.NewValidationService(cfg.ValidationConfig)
if err != nil {
return nil, err
}

log.Println("Initializing experiment history service...")
experimentHistorySvc := services.NewExperimentHistoryService(db)
log.Println("Initializing experiment service...")
experimentSvc := services.NewExperimentService(&allServices, db)
log.Println("Initializing project settings service...")
projectSettingsSvc := services.NewProjectSettingsService(&allServices, db)

log.Println("Initializing segment history service...")
segmentHistorySvc := services.NewSegmentHistoryService(db)
log.Println("Initializing segment service...")
segmentSvc := services.NewSegmentService(&allServices, db)

log.Println("Initializing treatment history service...")
treatmentHistorySvc := services.NewTreatmentHistoryService(db)
log.Println("Initializing treatment service...")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These seem like Debug logs. It would be enough to just have error logs if the initialization fails. Alternatively, if we want to keep these logs, we should log them at the appropriate level.

treatmentSvc := services.NewTreatmentService(&allServices, db)

mlpSvc, err := services.NewMLPService(cfg.MLPConfig.URL)
if err != nil {
return nil, errors.Wrapf(err, "Failed initializing MLP Service")
}

log.Println("Initializing configuration service...")
configurationSvc := services.NewConfigurationService(cfg)

allServices = services.NewServices(
Expand Down
30 changes: 25 additions & 5 deletions treatment-service/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,31 @@
FROM alpine:latest
FROM golang:1.18-alpine as api-builder
ARG API_BIN_NAME=xp-treatment

RUN apk update && apk add musl-dev gcc

ENV GO111MODULE=on \
GOOS=linux \
GOARCH=amd64

WORKDIR /app
COPY . .

RUN apk update && apk add git make bash unzip libtool gcc musl-dev ca-certificates dumb-init tzdata
# Build Treatment Service binary
RUN go build \
-mod=vendor \
-tags musl \
-o ./bin/${API_BIN_NAME}

FROM alpine:latest

RUN apk update && apk add git make bash unzip libtool ca-certificates dumb-init tzdata
RUN mkdir -p /opt/xp_treatment

COPY . /opt/xp_treatment/
COPY ./bin/* /opt/xp_treatment/
COPY --from=api-builder /app/bin/* /opt/xp_treatment/

WORKDIR /opt/xp_treatment
ENTRYPOINT [ "./xp-treatment" ]

ARG API_BIN_NAME=xp-treatment
ENV XP_API_BIN "./${API_BIN_NAME}"

ENTRYPOINT ["sh", "-c", "${XP_API_BIN} \"$@\"", "--"]
2 changes: 1 addition & 1 deletion treatment-service/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ version:

build-treatment-service: version
@echo "Building binary..."
@cd ${TREATMENT_SVC_PATH} && go build -o ./bin/${TREATMENT_SVC_BIN_NAME}
@go build -o ./bin/${TREATMENT_SVC_BIN_NAME}
@cp api/treatment.yaml ${TREATMENT_SVC_PATH}/bin
@cp api/schema.yaml ${TREATMENT_SVC_PATH}/bin

Expand Down