Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade to go 1.22.1 (with fix for dockerfiles) #1764

Merged
merged 8 commits into from
Mar 12, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ jobs:
linux-tests:
strategy:
matrix:
go: [ 1.21.x ]
go: [ 1.22.x ]
runs-on: ubuntu-20.04
timeout-minutes: 15

Expand Down Expand Up @@ -40,7 +40,7 @@ jobs:
- name: Setup Go
uses: actions/setup-go@v3
with:
go-version: "1.21"
go-version: "1.22"
- name: checkout code
uses: actions/checkout@v3
- name: golangci-lint
Expand Down
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
# Mount the gcsfuse to /mnt/gcs:
# > docker run --privileged --device /fuse -v /mnt/gcs:/gcs:rw,rshared gcsfuse

FROM golang:1.21.7-alpine as builder
FROM golang:1.22.1-alpine as builder

RUN apk add git

Expand Down
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
module github.com/googlecloudplatform/gcsfuse

go 1.21
go 1.22.1

require (
cloud.google.com/go/compute/metadata v0.2.3
Expand Down
2 changes: 1 addition & 1 deletion perfmetrics/scripts/ml_tests/pytorch/run_model.sh
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ NUM_EPOCHS=80
TEST_BUCKET="gcsfuse-ml-data"

# Install golang
wget -O go_tar.tar.gz https://go.dev/dl/go1.21.7.linux-amd64.tar.gz -q
wget -O go_tar.tar.gz https://go.dev/dl/go1.22.1.linux-amd64.tar.gz -q
rm -rf /usr/local/go && tar -C /usr/local -xzf go_tar.tar.gz
export PATH=$PATH:/usr/local/go/bin

Expand Down
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
#!/bin/bash

# Installs go1.21 on the container, builds gcsfuse using log_rotation file
# Installs go1.22.1 on the container, builds gcsfuse using log_rotation file
# and installs tf-models-official v2.13.2, makes update to include clear_kernel_cache
# and epochs functionality, and runs the model

# Install go lang
wget -O go_tar.tar.gz https://go.dev/dl/go1.21.7.linux-amd64.tar.gz -q
wget -O go_tar.tar.gz https://go.dev/dl/go1.22.1.linux-amd64.tar.gz -q
sudo rm -rf /usr/local/go && tar -xzf go_tar.tar.gz && sudo mv go /usr/local
export PATH=$PATH:/usr/local/go/bin

Expand Down
4 changes: 2 additions & 2 deletions perfmetrics/scripts/presubmit_test/pr_perf_test/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -34,8 +34,8 @@ set -e
sudo apt-get update
echo Installing git
sudo apt-get install git
echo Installing go-lang 1.21.7
wget -O go_tar.tar.gz https://go.dev/dl/go1.21.7.linux-amd64.tar.gz -q
echo Installing go-lang 1.22.1
wget -O go_tar.tar.gz https://go.dev/dl/go1.22.1.linux-amd64.tar.gz -q
sudo rm -rf /usr/local/go && tar -xzf go_tar.tar.gz && sudo mv go /usr/local
export PATH=$PATH:/usr/local/go/bin
export CGO_ENABLED=0
Expand Down
2 changes: 1 addition & 1 deletion perfmetrics/scripts/read_cache/setup.sh
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ sed -i 's/define \+FIO_IO_U_PLAT_GROUP_NR \+\([0-9]\+\)/define FIO_IO_U_PLAT_GRO
cd -

# Install and validate go.
version=1.21.7
version=1.22.1
wget -O go_tar.tar.gz https://go.dev/dl/go${version}.linux-amd64.tar.gz -q
sudo rm -rf /usr/local/go
tar -xzf go_tar.tar.gz && sudo mv go /usr/local
Expand Down
4 changes: 2 additions & 2 deletions perfmetrics/scripts/run_e2e_tests.sh
Original file line number Diff line number Diff line change
Expand Up @@ -43,8 +43,8 @@ upgrade_gcloud_version

# e.g. architecture=arm64 or amd64
architecture=$(dpkg --print-architecture)
echo "Installing go-lang 1.21.7..."
wget -O go_tar.tar.gz https://go.dev/dl/go1.21.7.linux-${architecture}.tar.gz -q
echo "Installing go-lang 1.22.1..."
wget -O go_tar.tar.gz https://go.dev/dl/go1.22.1.linux-${architecture}.tar.gz -q
sudo rm -rf /usr/local/go && tar -xzf go_tar.tar.gz && sudo mv go /usr/local
export PATH=$PATH:/usr/local/go/bin
# install python3-setuptools tools.
Expand Down
2 changes: 1 addition & 1 deletion tools/cd_scripts/e2e_test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,7 @@ else
fi

# install go
wget -O go_tar.tar.gz https://go.dev/dl/go1.21.7.linux-${architecture}.tar.gz
wget -O go_tar.tar.gz https://go.dev/dl/go1.22.1.linux-${architecture}.tar.gz
sudo tar -C /usr/local -xzf go_tar.tar.gz
export PATH=${PATH}:/usr/local/go/bin
#Write gcsfuse and go version to log file
Expand Down
16 changes: 8 additions & 8 deletions tools/containerize_gcsfuse_docker/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ ARG OS_VERSION
ARG OS_NAME

# Image with gcsfuse installed and its package (.deb)
FROM golang:1.21.7 as gcsfuse-package
FROM golang:1.22.1 as gcsfuse-package

RUN apt-get update -qq && apt-get install -y ruby ruby-dev rubygems build-essential rpm fuse && gem install --no-document bundler

Expand All @@ -43,21 +43,21 @@ ENV GOOS=linux
ENV GO111MODULE=auto

ARG GCSFUSE_VERSION
ARG GCSFUSE_REPO="github.com/googlecloudplatform/gcsfuse/"
ENV GCSFUSE_PATH "$GOPATH/src/$GCSFUSE_REPO"
RUN go get -d ${GCSFUSE_REPO}
ARG GCSFUSE_REPO="https://github.com/googlecloudplatform/gcsfuse/"
ARG BRANCH_NAME="v${GCSFUSE_VERSION}"
RUN git clone --branch ${BRANCH_NAME} --depth 1 ${GCSFUSE_REPO}

ARG GCSFUSE_PATH=${GOPATH}/gcsfuse
WORKDIR ${GCSFUSE_PATH}
# Branch name for building through a particular branch.
ARG BRANCH_NAME="v${GCSFUSE_VERSION}"
RUN git checkout "${BRANCH_NAME}"

# Install fpm package using bundle
RUN bundle install --gemfile=${GCSFUSE_PATH}/tools/gem_dependency/Gemfile

ARG GCSFUSE_BIN="/gcsfuse"
WORKDIR ${GOPATH}
RUN go install ${GCSFUSE_REPO}/tools/build_gcsfuse

WORKDIR ${GCSFUSE_PATH}/tools/build_gcsfuse
RUN go install
RUN mkdir -p ${GCSFUSE_BIN}
RUN build_gcsfuse ${GCSFUSE_PATH} ${GCSFUSE_BIN} ${GCSFUSE_VERSION}
RUN mkdir -p ${GCSFUSE_BIN}/usr && mv ${GCSFUSE_BIN}/bin ${GCSFUSE_BIN}/usr/bin
Expand Down
21 changes: 9 additions & 12 deletions tools/package_gcsfuse_docker/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
# Copy the gcsfuse packages to the host:
# > docker run -it -v /tmp:/output gcsfuse-release cp -r /packages /output

FROM golang:1.21.7 as builder
FROM golang:1.22.1 as builder

RUN apt-get update -qq && apt-get install -y ruby ruby-dev rubygems build-essential rpm && gem install --no-document bundler

Expand All @@ -26,19 +26,16 @@ ENV GOOS=linux
ENV GO111MODULE=auto

ARG GCSFUSE_VERSION
ARG GCSFUSE_REPO="github.com/googlecloudplatform/gcsfuse/"
ENV GCSFUSE_PATH "$GOPATH/src/$GCSFUSE_REPO"
RUN go get -d ${GCSFUSE_REPO}
ARG GCSFUSE_REPO="https://github.com/googlecloudplatform/gcsfuse/"
ARG BRANCH_NAME="v${GCSFUSE_VERSION}"
RUN git clone --branch ${BRANCH_NAME} --depth 1 ${GCSFUSE_REPO}

ARG GCSFUSE_PATH=${GOPATH}/gcsfuse
WORKDIR ${GCSFUSE_PATH}

ARG DEBEMAIL="gcs-fuse-maintainers@google.com"
ARG DEBFULLNAME="GCSFuse Team"

# Build Arg for building through a particular branch/commit. By default, it uses
# the tag corresponding to passed GCSFUSE VERSION
ARG BRANCH_NAME="v${GCSFUSE_VERSION}"
RUN git checkout "${BRANCH_NAME}"

# Install fpm package using bundle
RUN bundle install --gemfile=${GCSFUSE_PATH}/tools/gem_dependency/Gemfile

Expand All @@ -49,14 +46,14 @@ RUN if [ "${ARCHITECTURE}" != "amd64" ] && [ "${ARCHITECTURE}" != "arm64" ]; the
fi
ARG GCSFUSE_BIN="/gcsfuse_${GCSFUSE_VERSION}_${ARCHITECTURE}"
ARG GCSFUSE_DOC="${GCSFUSE_BIN}/usr/share/doc/gcsfuse"
WORKDIR ${GOPATH}
RUN go install ${GCSFUSE_REPO}/tools/build_gcsfuse
WORKDIR ${GCSFUSE_PATH}/tools/build_gcsfuse
RUN go install
RUN mkdir -p ${GCSFUSE_BIN}
RUN build_gcsfuse ${GCSFUSE_PATH} ${GCSFUSE_BIN} ${GCSFUSE_VERSION}
RUN mkdir -p ${GCSFUSE_BIN}/usr && mv ${GCSFUSE_BIN}/bin ${GCSFUSE_BIN}/usr/bin

# Creating structure for debian package as we are using 'dpkg-deb --build' to create debian package
RUN mkdir -p ${GCSFUSE_BIN}/DEBIAN && cp $GOPATH/src/$GCSFUSE_REPO/DEBIAN/* ${GCSFUSE_BIN}/DEBIAN/
RUN mkdir -p ${GCSFUSE_BIN}/DEBIAN && cp ${GCSFUSE_PATH}/DEBIAN/* ${GCSFUSE_BIN}/DEBIAN/
RUN mkdir -p ${GCSFUSE_DOC}
RUN mv ${GCSFUSE_BIN}/DEBIAN/copyright ${GCSFUSE_DOC} && \
mv ${GCSFUSE_BIN}/DEBIAN/changelog ${GCSFUSE_DOC} && \
Expand Down