Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

extensions: support for content sharing for ROS #4221

Merged
merged 87 commits into from
Sep 14, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
87 commits
Select commit Hold shift + click to select a range
2c45643
ros: rationalize ws sourcing
artivis Mar 20, 2023
4434e97
ros: source build-snap ws
artivis Mar 20, 2023
4ff7f8f
ros: unmark build snap packages
artivis Mar 21, 2023
b621c61
ros: prepare build-snap ws
artivis Mar 21, 2023
b32527a
ros: command-chain source content-shared ws ros1
artivis Mar 21, 2023
37871b0
ros: add noetic meta extensions
artivis Mar 27, 2023
f7cd12b
ros: fix unit tests
artivis Mar 27, 2023
a84df0a
ros: add noetic meta spread test
artivis Mar 30, 2023
7b1a63d
ros: add foxy meta extensions
artivis Apr 5, 2023
7e2415d
ros: add foxy meta spread test
artivis Apr 5, 2023
fea4d03
ros: command-chain source content-sharing ws ros2
artivis Apr 12, 2023
556161a
ros: colcon export CMAKE_PREFIX_PATH pointing to build snap
artivis Apr 12, 2023
f6f3a85
feat(ros-content-sharing): recursively unmark the dependencies
Guillaumebeuzeboc Apr 20, 2023
bb9dd38
test(spread): catkin noetic content sharing
Guillaumebeuzeboc Apr 20, 2023
0ba59f4
test(spread): fix linter ros1-hello
Guillaumebeuzeboc Apr 21, 2023
7a4b865
ros: check if file exists before opening
artivis Apr 20, 2023
cb22a72
ros: typo
artivis Apr 20, 2023
b85f171
ros: fix colcon unit test
artivis Apr 20, 2023
4260d62
ros: test noetic meta fix name
artivis Apr 20, 2023
5eba53f
ros: test ros2 foxy meta extentions
artivis Apr 20, 2023
16c2d3d
ros: humble support content-sharing
artivis Apr 20, 2023
a46cae7
ros: test humble support content-sharing
artivis Apr 20, 2023
bb6b45b
ros: add humble meta extensions
artivis Apr 20, 2023
874702c
ros: rename meta snap and plugs
artivis Apr 20, 2023
f18694b
ros: rename plugs
artivis Apr 20, 2023
9222180
ros: use dev build snaps
artivis Apr 23, 2023
e8415e3
ros: use humble dev build snaps
artivis Apr 23, 2023
5db3e28
ros: fix spread test
artivis Apr 21, 2023
bbb1ccd
ros: fix legacy static tests
artivis Apr 23, 2023
4bce041
ros: fix parts static tests
artivis Apr 23, 2023
2007bae
ros: handle possibly undefined env var
artivis Apr 21, 2023
762f545
ros: fix LD_LIB export
artivis Apr 22, 2023
ed4eee4
ros: enable more spread test
artivis Apr 25, 2023
632b7c3
ros: get_installed_dependencies in colcon core22
artivis Apr 25, 2023
ce3f790
ros: handle empty env var
artivis Apr 25, 2023
5933716
ros: static tests fixes
artivis Apr 26, 2023
3b3a5f2
ros: handle empty env var
artivis Apr 26, 2023
7dbc17e
ros: static tests fixes
artivis Apr 26, 2023
725be46
ros: fix unbound variable
artivis Apr 26, 2023
95fa691
ros: handle empty env var
artivis Apr 26, 2023
c31e4bb
ros: move cmake fixes to build-snap
artivis May 22, 2023
c3097c5
ros: stage roslib
artivis May 22, 2023
e173c60
ros: content-sharing fix python paths
artivis May 22, 2023
68ff125
ros: stage ros-environment
artivis May 24, 2023
7105484
ros: move/fix paths to extensions
artivis May 24, 2023
be57a20
ros: match extension/part names
artivis May 24, 2023
fa19811
ros: ros-environment
artivis May 24, 2023
0d89565
ros: revamp content-sharing spread tests
artivis May 24, 2023
fc87285
ros: mark meta extensions experimental
artivis May 26, 2023
4a7454a
ros: revamp spread tests
artivis May 26, 2023
0e3a3fd
ros: core22 add build-pkg libpython3.10
artivis May 26, 2023
febd885
ros: fix paths
artivis May 26, 2023
ce95bb0
ros: mv from var to property
artivis May 26, 2023
920648d
ros: please linting
artivis May 26, 2023
f829641
ros: please static-test
artivis May 26, 2023
07800a4
ros: rename spread test
artivis May 30, 2023
75bcb5d
ros: add comment
artivis May 30, 2023
3f7ee45
ros: move stage package to meta extension
artivis Jun 1, 2023
5547a8b
ros: add spread tests
artivis Jun 1, 2023
f5dad45
ros: meta rename build-snaps -> ros-build-snaps
artivis Jun 2, 2023
23ed7bf
ros: noetic plugins common cmake args var
artivis Jun 8, 2023
6a47ebd
ros: catkin_tools support for content sharing
artivis Jun 8, 2023
f1babb2
ros: please linters
artivis Jun 8, 2023
1dd7350
ros: add missing LD_LIBRARY_PATH path
artivis Jun 19, 2023
432107b
legacy _deb: better handle default packages_filters
artivis Jun 20, 2023
91ee0c0
ros: small return fix
artivis Jun 20, 2023
c84e4b7
ros: fix spread tests
artivis Jun 20, 2023
568dbc5
ros: fix unit tests LD_LIBRARY_PATH
artivis Jun 21, 2023
3e57168
ros: humble rename ros-build-snaps -> colcon-ros-build-snaps
artivis Jun 21, 2023
68d5638
ros: mv PATH extension to command-chain
artivis Jun 22, 2023
9cca729
ros: pedantic default
artivis Jul 10, 2023
6195945
ros: IOError -> OSError
artivis Jul 10, 2023
6232717
ros: fix import
artivis Jul 10, 2023
789f7cd
ros: test simplifications
artivis Jul 10, 2023
e1d765d
ros: cleanup comments & quote
artivis Jul 10, 2023
98f950f
ros: use raw string for regex expr
artivis Jul 10, 2023
f7bd3da
ros: fix unit test
artivis Jul 11, 2023
4bf037c
ros: fix rebase
artivis Jul 13, 2023
2d5b98a
ros: plugin-aware meta extensions
artivis Jul 13, 2023
434d32c
ros: source colcon ws only if it exists
artivis Jul 26, 2023
1dba7e7
ros: fix ros1 noetic ext part-snippet
artivis Jul 27, 2023
80e30d2
ros: comply with python 3.8
artivis Jul 27, 2023
8af39ad
ros: fix unit test
artivis Jul 27, 2023
a784fd9
ros: pedantic
artivis Jul 28, 2023
f011ff0
Merge branch 'main' into feature/ros_content_sharing
sergiusens Aug 11, 2023
43e3aa2
Merge branch 'main' into feature/ros_content_sharing
mr-cal Sep 14, 2023
b08702e
Merge branch 'main' into feature/ros_content_sharing
mr-cal Sep 14, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 24 additions & 3 deletions extensions/ros1/launch
Original file line number Diff line number Diff line change
Expand Up @@ -2,19 +2,40 @@

function activate_workspace() {
rosdir="$1"
setup_sh="$rosdir/setup.bash"
setup_sh="$rosdir/setup.sh"

if [ ! -f "$setup_sh" ]; then
echo "error: $setup_sh not found"
exit 1
fi
# shellcheck disable=SC1090
_CATKIN_SETUP_DIR="$rosdir" source "$setup_sh"
_CATKIN_SETUP_DIR="$rosdir" source "$setup_sh" "${@:2}"
}

# Save off parameters, the sourced setup scripts may manipulate them.
original_args=("$@")

activate_workspace "$SNAP/opt/ros/$ROS_DISTRO"
# activate_workspace in content shared snaps
if [ -d "$SNAP/opt/ros/underlay_ws/opt/ros/$ROS_DISTRO" ]; then
activate_workspace "$SNAP/opt/ros/underlay_ws/opt/ros/$ROS_DISTRO" --local

case "$SNAP_ARCH" in
"amd64") ARCH_TRIPLET='x86_64-linux-gnu'
;;
"arm64") ARCH_TRIPLET="aarch64-linux-gnu"
;;
"armhf") ARCH_TRIPLET="arm-linux-gnueabihf"
;;
*)
echo "Unsupported architecture '$SNAP_ARCH' for this app build."
exit 1
;;
esac

export PATH="$SNAP/opt/ros/underlay_ws/usr/bin:$PATH"
export LD_LIBRARY_PATH="$SNAP/opt/ros/underlay_ws/usr/lib:$SNAP/opt/ros/underlay_ws/usr/lib/$ARCH_TRIPLET:$LD_LIBRARY_PATH"
fi

activate_workspace "$SNAP/opt/ros/$ROS_DISTRO" --local --extend

exec "${original_args[@]}"
37 changes: 33 additions & 4 deletions extensions/ros2/launch
Original file line number Diff line number Diff line change
Expand Up @@ -2,18 +2,47 @@

function source_with_prefix() {
export ROS_WORKSPACE_CURRENT_PREFIX="$1"
if [ ! -f "$ROS_WORKSPACE_CURRENT_PREFIX/local_setup.bash" ]; then
echo "error: $ROS_WORKSPACE_CURRENT_PREFIX/local_setup.bash not found"
SETUP_SCRIPT="$ROS_WORKSPACE_CURRENT_PREFIX/local_setup.bash"
if [ ! -f "$SETUP_SCRIPT" ]; then
echo "error: $SETUP_SCRIPT not found"
exit 1
fi
# shellcheck disable=SC1090,SC1091
source "$ROS_WORKSPACE_CURRENT_PREFIX/local_setup.bash"
source "$SETUP_SCRIPT"
}

# Save off parameters, the sourced setup scripts may manipulate them.
original_args=("$@")

if [ -d "$SNAP/opt/ros/underlay_ws/opt/ros/$ROS_DISTRO" ]; then
source_with_prefix "$SNAP/opt/ros/underlay_ws/opt/ros/$ROS_DISTRO"
fi
if [ -d "$SNAP/opt/ros/underlay_ws/opt/ros/snap" ]; then
source_with_prefix "$SNAP/opt/ros/underlay_ws/opt/ros/snap"
fi
if [ -d "$SNAP/opt/ros/underlay_ws/opt/ros/$ROS_DISTRO" ] || [ -d "$SNAP/opt/ros/underlay_ws/opt/ros/snap" ]; then

case "$SNAP_ARCH" in
"amd64") ARCH_TRIPLET='x86_64-linux-gnu'
;;
"arm64") ARCH_TRIPLET="aarch64-linux-gnu"
;;
"armhf") ARCH_TRIPLET="arm-linux-gnueabihf"
;;
*)
echo "Unsupported architecture '$SNAP_ARCH' for this app build."
exit 1
;;
esac

export PATH="$SNAP/opt/ros/underlay_ws/usr/bin:$PATH"
export LD_LIBRARY_PATH="$SNAP/opt/ros/underlay_ws/usr/lib:$SNAP/opt/ros/underlay_ws/usr/lib/$ARCH_TRIPLET:$LD_LIBRARY_PATH"
fi

source_with_prefix "$SNAP/opt/ros/$ROS_DISTRO"
artivis marked this conversation as resolved.
Show resolved Hide resolved
source_with_prefix "$SNAP/opt/ros/snap"

if [ -f "$SNAP/opt/ros/snap/local_setup.bash" ]; then
artivis marked this conversation as resolved.
Show resolved Hide resolved
source_with_prefix "$SNAP/opt/ros/snap"
fi

exec "${original_args[@]}"
10 changes: 9 additions & 1 deletion schema/snapcraft.json
Original file line number Diff line number Diff line change
Expand Up @@ -915,7 +915,15 @@
"gnome-3-38",
"kde-neon",
"ros1-noetic",
"ros2-foxy"
"ros1-noetic-desktop",
"ros1-noetic-perception",
"ros1-noetic-robot",
"ros1-noetic-ros-base",
"ros1-noetic-ros-core",
"ros2-foxy",
"ros2-foxy-ros-base",
"ros2-foxy-ros-core",
"ros2-foxy-desktop"
]
}
}
Expand Down
119 changes: 119 additions & 0 deletions snapcraft/extensions/_ros2_humble_meta.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*-
#
# Copyright 2023 Canonical Ltd.
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License version 3 as
# published by the Free Software Foundation.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.

# Import types and tell flake8 to ignore the "unused" List.

"""Base for ROS 2 Humble extensions to the Colcon plugin using content-sharing."""

import dataclasses
from abc import abstractmethod
from typing import Any, Dict, Optional

from overrides import overrides

from .ros2_humble import ROS2HumbleExtension


@dataclasses.dataclass
class ROS2HumbleSnaps:
"""A structure of ROS 2 Humble related snaps."""

sdk: str
content: str
variant: str


class ROS2HumbleMetaBase(ROS2HumbleExtension):
"""Drives ROS 2 build and runtime environment for snap using content-sharing."""

@property
@abstractmethod
def ros2_humble_snaps(self) -> ROS2HumbleSnaps:
"""Return the ROS 2 Humble related snaps to use to construct the environment."""
raise NotImplementedError

@staticmethod
@overrides
def is_experimental(base: Optional[str]) -> bool:
return True

@overrides
def get_root_snippet(self) -> Dict[str, Any]:
root_snippet = super().get_root_snippet()
root_snippet["plugs"] = {
"ros-humble": {
"interface": "content",
"content": "ros-humble",
"target": "$SNAP/opt/ros/underlay_ws",
"default-provider": self.ros2_humble_snaps.content,
}
}
return root_snippet

@overrides
def get_app_snippet(self) -> Dict[str, Any]:
app_snippet = super().get_app_snippet()
python_paths = app_snippet["environment"]["PYTHONPATH"]
new_python_paths = [
f"$SNAP/opt/ros/underlay_ws/opt/ros/{self.ROS_DISTRO}/lib/python3.10/site-packages",
"$SNAP/opt/ros/underlay_ws/usr/lib/python3/dist-packages",
]

app_snippet["environment"][
"PYTHONPATH"
] = f'{python_paths}:{":".join(new_python_paths)}'

return app_snippet

@overrides
def get_part_snippet(self, *, plugin_name: str) -> Dict[str, Any]:
part_snippet = super().get_part_snippet(plugin_name=plugin_name)

# These are colcon-plugin specific entries
if plugin_name == "colcon":
part_snippet["colcon-ros-build-snaps"] = [self.ros2_humble_snaps.sdk]
part_snippet["colcon-cmake-args"] = [
f'-DCMAKE_SYSTEM_PREFIX_PATH="/snap/{self.ros2_humble_snaps.sdk}/current/usr"'
]

return part_snippet

@overrides
def get_parts_snippet(self) -> Dict[str, Any]:
parts_snippet = super().get_parts_snippet()
# Very unlikely but it may happen that the snapped application doesn't
# even pull those deps. In that case, there is no valid ROS 2 ws to source.
# We make sure here that they are staged no matter what.
parts_snippet[f"ros2-{self.ROS_DISTRO}/ros2-launch"]["stage-packages"] = [
f"ros-{self.ROS_DISTRO}-ros-environment",
f"ros-{self.ROS_DISTRO}-ros-workspace",
f"ros-{self.ROS_DISTRO}-ament-index-cpp",
f"ros-{self.ROS_DISTRO}-ament-index-python",
]

# Something in the ROS 2 build chain requires to find this lib during cmake call,
# however its cmake files ship with the '-dev' package.
parts_snippet[f"ros2-{self.ROS_DISTRO}/ros2-launch"]["build-packages"].append(
"libpython3.10-dev"
)

# The part name must follow the format <extension-name>/<part-name>
parts_snippet[
f"ros2-{self.ROS_DISTRO}-{self.ros2_humble_snaps.variant}/ros2-launch"
] = parts_snippet[f"ros2-{self.ROS_DISTRO}/ros2-launch"]
parts_snippet.pop(f"ros2-{self.ROS_DISTRO}/ros2-launch")

return parts_snippet
6 changes: 6 additions & 0 deletions snapcraft/extensions/registry.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,9 @@
from .gnome import GNOME
from .kde_neon import KDENeon
from .ros2_humble import ROS2HumbleExtension
from .ros2_humble_desktop import ROS2HumbleDesktopExtension
from .ros2_humble_ros_base import ROS2HumbleRosBaseExtension
from .ros2_humble_ros_core import ROS2HumbleRosCoreExtension

if TYPE_CHECKING:
from .extension import Extension
Expand All @@ -32,6 +35,9 @@
_EXTENSIONS: Dict[str, "ExtensionType"] = {
"gnome": GNOME,
"ros2-humble": ROS2HumbleExtension,
"ros2-humble-ros-core": ROS2HumbleRosCoreExtension,
"ros2-humble-ros-base": ROS2HumbleRosBaseExtension,
"ros2-humble-desktop": ROS2HumbleDesktopExtension,
"kde-neon": KDENeon,
}

Expand Down
38 changes: 38 additions & 0 deletions snapcraft/extensions/ros2_humble_desktop.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*-
#
# Copyright 2023 Canonical Ltd.
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License version 3 as
# published by the Free Software Foundation.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.

# Import types and tell flake8 to ignore the "unused" List.

"""Extension to the Colcon plugin for ROS 2 Humble using content sharing."""

import functools

from overrides import overrides

from ._ros2_humble_meta import ROS2HumbleMetaBase, ROS2HumbleSnaps


class ROS2HumbleDesktopExtension(ROS2HumbleMetaBase):
"""Drives ROS 2 build and runtime environment for snap using content-sharing."""

@functools.cached_property
@overrides
def ros2_humble_snaps(self) -> ROS2HumbleSnaps:
return ROS2HumbleSnaps(
sdk="ros-humble-desktop-dev",
content="ros-humble-desktop",
variant="desktop",
)
38 changes: 38 additions & 0 deletions snapcraft/extensions/ros2_humble_ros_base.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*-
#
# Copyright 2023 Canonical Ltd.
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License version 3 as
# published by the Free Software Foundation.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.

# Import types and tell flake8 to ignore the "unused" List.

"""Extension to the Colcon plugin for ROS 2 Humble using content sharing."""

import functools

from overrides import overrides

from ._ros2_humble_meta import ROS2HumbleMetaBase, ROS2HumbleSnaps


class ROS2HumbleRosBaseExtension(ROS2HumbleMetaBase):
"""Drives ROS 2 build and runtime environment for snap using content-sharing."""

@functools.cached_property
@overrides
def ros2_humble_snaps(self) -> ROS2HumbleSnaps:
return ROS2HumbleSnaps(
sdk="ros-humble-ros-base-dev",
content="ros-humble-ros-base",
variant="ros-base",
)
38 changes: 38 additions & 0 deletions snapcraft/extensions/ros2_humble_ros_core.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*-
#
# Copyright 2023 Canonical Ltd.
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License version 3 as
# published by the Free Software Foundation.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.

# Import types and tell flake8 to ignore the "unused" List.

"""Extension to the Colcon plugin for ROS 2 Humble using content sharing."""

import functools

from overrides import overrides

from ._ros2_humble_meta import ROS2HumbleMetaBase, ROS2HumbleSnaps


class ROS2HumbleRosCoreExtension(ROS2HumbleMetaBase):
"""Drives ROS 2 build and runtime environment for snap using content-sharing."""

@functools.cached_property
@overrides
def ros2_humble_snaps(self) -> ROS2HumbleSnaps:
return ROS2HumbleSnaps(
sdk="ros-humble-ros-core-dev",
content="ros-humble-ros-core",
variant="ros-core",
)
Loading