Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Earthly configuration and use it to build the spaceros base image. #15

Merged
merged 34 commits into from
Nov 9, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
a607a92
Add Earthfile for building Space ROS base image.
nuclearsandwich Jul 5, 2022
18f61fd
Add build-testing target to build and run colcon tests.
nuclearsandwich Jul 11, 2022
8ab72c9
Use ubuntu:focal as default base image.
nuclearsandwich Sep 29, 2022
57be1a7
Use a remote artifact for the repos file.
nuclearsandwich Sep 29, 2022
9e7e43f
Don't change artifact paths.
nuclearsandwich Sep 30, 2022
0b45b71
WIPWIP adding build result archives.
nuclearsandwich Sep 30, 2022
f3167af
Switch base image to Ubuntu Jammy.
nuclearsandwich Oct 6, 2022
88154b7
Save artifacts directly to current directory instead of subdirectory.
nuclearsandwich Oct 6, 2022
79c319f
Add trailing slash to copy to src directory.
nuclearsandwich Oct 6, 2022
89d6929
Add Earthly workflow for building the docker image.
nuclearsandwich Oct 6, 2022
5329f28
Ignore immediate src, install, and log subdirectories.
nuclearsandwich Oct 6, 2022
f4c453f
Add job for uploading build results archive.
nuclearsandwich Oct 6, 2022
09dddcd
Fix earthly download url.
nuclearsandwich Oct 6, 2022
71194cf
Add Earthfile artifact for bootstrapping.
nuclearsandwich Oct 6, 2022
b8d7dfd
Fix spelling of COPY directive.
nuclearsandwich Oct 6, 2022
735185e
Set up sources locally.
nuclearsandwich Oct 6, 2022
3972391
Fix entrypoint name.
nuclearsandwich Oct 6, 2022
cf4c16e
Allow sources to supply output for the next step.
nuclearsandwich Oct 6, 2022
2a28324
Install all dependencies using apt.
nuclearsandwich Oct 6, 2022
28568a3
Use the same directory structure as the existing Docker image.
nuclearsandwich Nov 1, 2022
0b796f0
Save the install directory from a build-testing run.
nuclearsandwich Nov 1, 2022
3afc9e3
Add labels from base docker image.
nuclearsandwich Nov 1, 2022
20c7cb7
Start to add ikos support.
nuclearsandwich Nov 2, 2022
7e04a40
Revert "Start to add ikos support."
nuclearsandwich Nov 2, 2022
1105ddc
Update image tag.
nuclearsandwich Nov 3, 2022
63aeafd
Support VCS_REF arg from Docker image build.
nuclearsandwich Nov 3, 2022
c036caa
Use the current HEAD as the default VCS ref.
nuclearsandwich Nov 3, 2022
ecac7f5
Use earthly to build the base Space ROS image.
nuclearsandwich Nov 4, 2022
9798c56
Omit GPU arguments and update image / repository name.
nuclearsandwich Nov 4, 2022
5af1296
Remove unused Dockerfile and corresponding GitHub workflow.
nuclearsandwich Nov 4, 2022
3f37ef9
Add push step when the build is on main.
nuclearsandwich Nov 7, 2022
9f0efb9
Start updating README.md for Earthly workflow.
nuclearsandwich Nov 7, 2022
8ba23f5
This file has been replaced by the README.md and build.sh scripts.
nuclearsandwich Nov 8, 2022
5d0b373
Clean and re-fetch sources in automatic build script.
nuclearsandwich Nov 8, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 0 additions & 24 deletions .github/workflows/docker-build.yml

This file was deleted.

50 changes: 50 additions & 0 deletions .github/workflows/earthly-build.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
name: Earthly build
on:
push:
branches: ['main']
pull_request:
schedule:
- cron: '0 11 * * *'
jobs:
build-testing-results:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Set up earthly
run: |
sudo wget https://github.com/earthly/earthly/releases/latest/download/earthly-linux-amd64 -O /usr/local/bin/earthly
sudo chmod 755 /usr/local/bin/earthly
- name: Build Space ROS
id: build
run: |
cd spaceros
earthly --ci --output +sources
earthly --ci +build-testing
printf "::set-output name=archivename::%s\n" $(basename log/build_results_archives/build_results_*.tar.bz2)
- name: Upload build results archive
uses: actions/upload-artifact@v3
with:
name: ${{steps.build.outputs.archivename}}
path: spaceros/log/build_results_archives/${{steps.build.outputs.archivename}}
space-ros-image:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Set up earthly
run: |
sudo wget https://github.com/earthly/earthly/releases/latest/download/earthly-linux-amd64 -O /usr/local/bin/earthly
sudo chmod 755 /usr/local/bin/earthly
- name: Build spaceros image
run: |
cd spaceros
earthly --ci --output +sources
earthly --ci +image
- name: Push spaceros image
env:
DOCKER_HUB_TOKEN: ${{ secrets.DOCKER_HUB_RW_TOKEN }}
if: ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' }}
run: |
echo $DOCKER_HUB_TOKEN | docker login --username osrfbot --password-stdin
earthly --ci --push +image
4 changes: 4 additions & 0 deletions spaceros/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
spaceros_ws/
src/
install/
log/
162 changes: 0 additions & 162 deletions spaceros/Dockerfile

This file was deleted.

150 changes: 150 additions & 0 deletions spaceros/Earthfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,150 @@
# Copyright 2021 Open Source Robotics Foundation, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
VERSION 0.6
FROM ubuntu:jammy

earthfile:
COPY Earthfile Earthfile
SAVE ARTIFACT Earthfile

setup:
# Disable prompting during package installation
ARG DEBIAN_FRONTEND=noninteractive

# The following commands are based on the source install for ROS 2 Rolling Ridley.
# See: https://docs.ros.org/en/ros2_documentation/rolling/Installation/Ubuntu-Development-Setup.html
# The main variation is getting Space ROS sources instead of the Rolling sources.

# Update the Ubuntu software repository
RUN apt-get update

# Set the locale
RUN apt-get install -y locales
RUN locale-gen en_US en_US.UTF-8
RUN update-locale LC_ALL=en_US.UTF-8 LANG=en_US.UTF-8
ENV LANG=en_US.UTF-8

# Add the ROS 2 apt repository
RUN apt-get install -y software-properties-common
RUN add-apt-repository universe
RUN apt-get update && apt-get install -y curl gnupg lsb-release
RUN curl -sSL https://raw.githubusercontent.com/ros/rosdistro/master/ros.key -o /usr/share/keyrings/ros-archive-keyring.gpg
RUN echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/ros-archive-keyring.gpg] http://packages.ros.org/ros2/ubuntu $(lsb_release -cs) main" | tee /etc/apt/sources.list.d/ros2.list > /dev/null

# Install required software development tools and ROS tools (and vim included for convenience)
RUN apt-get update && apt-get install -y \
bison \
build-essential \
cmake \
git \
python3-colcon-common-extensions \
python3-flake8 \
python3-flake8-blind-except \
python3-flake8-builtins \
python3-flake8-class-newline \
python3-flake8-comprehensions \
python3-flake8-deprecated \
python3-flake8-docstrings \
python3-flake8-import-order \
python3-flake8-quotes \
python3-pip \
python3-pytest \
python3-pytest-cov \
python3-pytest-repeat \
python3-pytest-rerunfailures \
python3-rosdep \
python3-setuptools \
python3-vcstool \
wget \
vim
Comment on lines +45 to +70
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can replace a lot of this with ros-dev-tools and drop much of the rest once ament/ament_lint#420 is released.


# Define the username and key variables
ENV USERNAME spaceros-user
ENV SRC_DIR=/home/${USERNAME}/src
ENV SPACEROS_DIR=${SRC_DIR}/spaceros
ARG IKOS_DIR=${SRC_DIR}/ikos
ARG REPOS_FILE_URL="https://raw.githubusercontent.com/space-ros/space-ros/main/ros2.repos"
ENV ROSDISTRO=humble

# Create a spaceros user
RUN useradd -m $USERNAME && \
echo "$USERNAME:$USERNAME" | chpasswd && \
usermod --shell /bin/bash $USERNAME && \
usermod -aG sudo $USERNAME && \
mkdir -p /etc/sudoers.d && \
echo "$USERNAME ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers.d/$USERNAME && \
chmod 0440 /etc/sudoers.d/$USERNAME
USER ${USERNAME}

WORKDIR $SPACEROS_DIR


sources:
FROM +setup
COPY github.com/space-ros/space-ros:earthly-wrapper+repos-file/ros2.repos ros2.repos
RUN mkdir src
RUN vcs import src < ros2.repos
SAVE ARTIFACT src AS LOCAL src

workspace:
FROM +setup
COPY src src/

vcs-exact:
FROM +workspace
RUN vcs export --exact src > exact.repos
SAVE ARTIFACT exact.repos AS LOCAL exact.repos

rosdep:
FROM +workspace
# Install system package dependencies using rosdep
RUN sudo rosdep init && rosdep update
RUN rosdep install --from-paths src --ignore-src --rosdistro rolling -y --skip-keys "console_bridge fastcdr fastrtps rti-connext-dds-5.3.1 urdfdom_headers rmw_connextdds ros_testing rmw_connextdds rmw_fastrtps_cpp rmw_fastrtps_dynamic_cpp composition demo_nodes_py lifecycle rosidl_typesupport_fastrtps_cpp rosidl_typesupport_fastrtps_c ikos"

build:
FROM +rosdep
RUN colcon build --cmake-args -DCMAKE_BUILD_TYPE=Release -DCMAKE_EXPORT_COMPILE_COMMANDS=ON --no-warn-unused-cli
COPY +vcs-exact/exact.repos install/exact.repos
SAVE ARTIFACT install AS LOCAL install

build-testing:
FROM +rosdep
RUN colcon build --cmake-args -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_EXPORT_COMPILE_COMMANDS=ON --no-warn-unused-cli
RUN colcon test --retest-until-pass 2 --ctest-args -LE "(ikos|xfail)" --pytest-args -m "not xfail"
RUN . install/setup.sh && ros2 run process_sarif make_build_archive
COPY +vcs-exact/exact.repos install/exact.repos
SAVE ARTIFACT log/build_results_archives/build_results*.tar.bz2 AS LOCAL spaceros_ws/log/build_results_archives/
SAVE ARTIFACT install AS LOCAL install

image:
FROM +rosdep

ARG VCS_REF

# Specify the docker image metadata
LABEL org.label-schema.schema-version="1.0"
LABEL org.label-schema.name="Space ROS"
LABEL org.label-schema.description="Preview version of the Space ROS platform"
LABEL org.label-schema.vendor="Open Robotics"
LABEL org.label-schema.url="https://github.com/space-ros"
LABEL org.label-schema.vcs-url="https://github.com/space-ros/docker-images"
LABEL org.label-schema.vcs-ref=${VCS_REF}

COPY +build/install ${SPACEROS_DIR}/install
COPY +vcs-exact/exact.repos ${SPACEROS_DIR}/exact.repos
RUN rm -r src
COPY entrypoint.sh /ros_entrypoint.sh
ENTRYPOINT ["/ros_entrypoint.sh"]
CMD ["bash"]
SAVE IMAGE osrf/space-ros:latest
Loading