Skip to content

Compare: Superflore OE Recipe Generation Scheme

Showing with 175 additions and 96 deletions.
  1. +130 −76 OpenEmbedded-Build-Instructions.md
  2. +45 −20 Superflore-OE-Recipe-Generation-Scheme.md
65 changes: 45 additions & 20 deletions Superflore-OE-Recipe-Generation-Scheme.md
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# **Superflore** OE Recipe Generation Scheme

Revision: 44 Published: 2019-12-10
Revision: 45 Published: 2019-12-19

## General

Expand Down Expand Up @@ -620,7 +620,8 @@ ROS_SUPERFLORE_GENERATION_COMMIT = "<ROS-ROSDISTRO-COMMIT>"
| `files/<ROS_DISTRO>/` | `meta-ros[12]-<ROS_DISTRO>/files/<ROS_DISTRO>/generated/` |
| `generated-recipes-<ROS_DISTRO>/` | `meta-ros[12]-<ROS_DISTRO>/generated-recipes/` |
| `includes-ros[12]/*/*.inc` | `meta-ros[12]-<ROS_DISTRO>/recipes-bbappends/*/*.bbappend` |
| `recipes-*/` | `{meta-ros-common,meta-ros[12],meta-ros-backports-warrior,meta-ros-webos}/recipes-*/`
| `recipes-*/` | `{meta-ros-common,meta-ros[12],meta-ros-backports-warrior,meta-ros-webos}/recipes-*/` |
| `recipes-core/packagegroups/packagegroup-ros-world-<ROS_DISTRO>.inc` | `meta-ros[12]-<ROS_DISTRO>/recipes-core/packagegroups//packagegroup-ros-world-<ROS_DISTRO>.bb` |
| `scripts/mcf` | separate `[build-draft]` branch in `meta-ros` |
Expand Down Expand Up @@ -668,7 +669,7 @@ ROS_SUPERFLORE_GENERATION_COMMIT = "<ROS-ROSDISTRO-COMMIT>"

## Milestones

### Milestone 1: ROS 2 with webOS OSE on Beanbird Bot (ROS 2 **bouncy** and OE-Core **morty**) [COMPLETED]
### Milestone 1: ROS 2 with webOS OSE on Beanbird Bot (ROS 2 **bouncy** and OE-Core **morty**)
- Use the latest `[devel]` branch of `build-ros2-lgsvl.git` to select the commits to build from for the layers other than
`meta-ros2`.
- Use `bitbake-layers show-overlayed` to find bbclass-s and recipes for platform packages in `meta-ros2` that are no longer needed
Expand Down Expand Up @@ -702,8 +703,9 @@ ROS_SUPERFLORE_GENERATION_COMMIT = "<ROS-ROSDISTRO-COMMIT>"
- Enhance `mcf` to recognize an optional `BblayersConfExtraLines` variable in `weboslayers.py` that will contain a tuple of
strings to be appended to the `conf/bblayers.conf` that it generates.
- Done when the image for Beanbird Bot works.
- **COMPLETED**

### Milestone 2: ROS Core on `qemux86` and `raspberrypi3` (ROS 2 **crystal** and OE-Core **morty**) [COMPLETED]
### Milestone 2: ROS Core on `qemux86` and `raspberrypi3` (ROS 2 **crystal** and OE-Core **morty**)
- Use `bitbake-layers show-overlayed` to find bbclass-s and recipes for platform packages in `meta-ros` that are no longer needed
because they are now provided by OE-Core **morty**.
- Upgrade the recipes included in `meta-ros` for the platform packages needed for **crystal** to the versions found in Ubuntu
Expand Down Expand Up @@ -775,8 +777,9 @@ ROS_SUPERFLORE_GENERATION_COMMIT = "<ROS-ROSDISTRO-COMMIT>"
data: Hello world
```
- **COMPLETED**

### Milestone 3: ROS World with webOS OSE on `qemux86` and `raspberrypi3` (ROS 2 **crystal** and OE-Core **morty**) [COMPLETED]
### Milestone 3: ROS World with webOS OSE on `qemux86` and `raspberrypi3` (ROS 2 **crystal** and OE-Core **morty**)
- Use the absence of `DISTRO_FEATURES` values to conditionally remove the "optional" packages associated with Gazebo, Navigation2,
Mobile Robot Programming Toolkit 2, RealSense, Qt, rqt, and RViz from `packagegroup-ros-world`. Adding the value for
BehaviorTree.CPP v3 to `DISTRO_FEATURES` removes the Navigation2 packages because they require v2 of BehaviorTree.CPP . Do this
Expand All @@ -792,8 +795,9 @@ ROS_SUPERFLORE_GENERATION_COMMIT = "<ROS-ROSDISTRO-COMMIT>"
running `webos-image-ros-world`, the webOS OSE menu must appear after pressing the **Windows** key and allow the settings for
**Network** to be launched. Note that webOS OSE automatically starts networking; issuing `/etc/init.d/networking start` is
unnecessary.
- **COMPLETED**

### Milestone 4: ROS World with webOS OSE on `qemux86` and `raspberrypi3` (ROS 2 **crystal** and OE-Core **thud**) [COMPLETED]
### Milestone 4: ROS World with webOS OSE on `qemux86` and `raspberrypi3` (ROS 2 **crystal** and OE-Core **thud**)
- Set `LAYERSERIES_COMPAT_ros-layer` to `thud` as it will be the only OE-Core release series with which the **crystal** `meta-ros`
will continue to be tested.
- Add `dashing` to `ROS_DISTRO_COMPAT`.
Expand All @@ -805,8 +809,9 @@ ROS_SUPERFLORE_GENERATION_COMMIT = "<ROS-ROSDISTRO-COMMIT>"
running `webos-image-ros-world`, the webOS OSE menu must appear after pressing the **Windows** key and allow the settings for
**Network** to be launched. Note that webOS OSE automatically starts networking; issuing `/etc/init.d/networking start` is
unnecessary.
- **COMPLETED**

### Milestone 5: ROS World with webOS OSE on `qemux86` and `raspberrypi3` (ROS 2 **dashing** and OE-Core **thud**) [COMPLETED]
### Milestone 5: ROS World with webOS OSE on `qemux86` and `raspberrypi3` (ROS 2 **dashing** and OE-Core **thud**)
- Add entries to `base.yaml` or `python.yaml` for platform packages whose mappings in `files/dashing/rosdep-resolve.yaml` have
names that do not contain `@<OE-LAYER>`.
- Update any `<PV>`-specific files for the new version of ROS packages in **dashing**.
Expand All @@ -825,8 +830,9 @@ ROS_SUPERFLORE_GENERATION_COMMIT = "<ROS-ROSDISTRO-COMMIT>"
running `webos-image-ros-world`, the webOS OSE menu must appear after pressing the **Windows** key and allow the settings for
**Network** to be launched. Note that webOS OSE automatically starts networking; issuing `/etc/init.d/networking start` is
unnecessary.
- **COMPLETED**

### Milestone 6: ROS Core with webOS OSE on `qemux86` and `raspberrypi3` (ROS 1 **melodic** and OE-Core **thud**) [COMPLETED]
### Milestone 6: ROS Core with webOS OSE on `qemux86` and `raspberrypi3` (ROS 1 **melodic** and OE-Core **thud**)
- Decide whether the ROS packages should installed under `/opt/ros/<ROS_DISTRO>` or folded into the rest of the OS under `/usr`
-- see [Issue #3](#Issues). Until this is decided. ROS 1 distros will install them under `/opt/ros/<ROS_DISTRO>` and ROS 2
distros will continue to install them under `/usr`.
Expand Down Expand Up @@ -894,8 +900,9 @@ ROS_SUPERFLORE_GENERATION_COMMIT = "<ROS-ROSDISTRO-COMMIT>"
(from http://wiki.ros.org/ROS/Tutorials/UnderstandingNodes). Additionally, when running `webos-image-ros-roscore`, the webOS OSE
menu must appear after pressing the **Windows** key and allow the settings for **Network** to be launched.
- **COMPLETED**
### Milestone 7: TurtleBot 3 Waffle Pi (using model B not B+) with webOS OSE (ROS 1 **melodic** and OE-Core **thud**) [COMPLETED]
### Milestone 7: TurtleBot 3 Waffle Pi (using model B not B+) with webOS OSE (ROS 1 **melodic** and OE-Core **thud**)
- Split off the generation of `files/<ROS_DISTRO>/cache.yaml` into own script **ros-generate-cache**.
- Change the name of the branch created during recipe generation from `superflore/<DATETIME> ` to
`superflore/<ROS_DISTRO>/<RELEASE-YYYYMMDD>`.
Expand Down Expand Up @@ -928,14 +935,15 @@ ROS_SUPERFLORE_GENERATION_COMMIT = "<ROS-ROSDISTRO-COMMIT>"
on them. (If `[WARN] [<timestamp>]: Failed to get param: timeout expired` is output, restart and try again.) Additionally,
when running `webos-image-ros-turtlebot3-all`, the webOS OSE menu must appear after pressing the **Windows** key and allow the
settings for **Network** to be launched.
- **COMPLETED**
### Milestone 8: Implement layer version 3
- Implement the items under **Phase 1** in the **Layer Version 3** section.
- Update this document with the file layout and content changes for layer version 3.
- Update <https://github.com/ros/meta-ros/wiki/OpenEmbedded-Build-Instructions> with the new image names.
- Update **superflore** to implement ROS_SUPERFLORE_GENERATION_SCHEME 2 and submit PR-s for the changes.
- Create a new branch `[legacy-v2]`, whose `files/*.mcf` fetch from `[legacy-v2]`, from the existing `[thud-draft]` branch. Its
files will not be changed nor rebased.
files will not be changed nor its commits rebased.
- Move `scripts/mcf` and `files/*.mcf` into a new independently-rooted branch `[build-draft]`. Give the `.mcf` file basenames a
`-thud` suffix.
- Add a rebaseable `[master-draft]` that is coincident with `[thud-draft]`.
Expand All @@ -945,9 +953,10 @@ ROS_SUPERFLORE_GENERATION_COMMIT = "<ROS-ROSDISTRO-COMMIT>"
- `ros-image-turtlebot3-all` (`DISTRO = "ros1"`) and `webos-image-ros-turtlebot3-all` (`DISTRO = "webos"`) build successfully
for `raspberrypi3` for **melodic**;
- The images pass the sanity tests.
- **COMPLETED**
### Milestone 9: Support ROS 2 **eloquent**
- Start with initial release from 2019-11-22 (see
- Use initial release from 2019-11-22 (see
[ROS 2 Eloquent Elusor](https://discourse.ros.org/t/ros-2-eloquent-elusor-released/11624)
- Upgrade webOS OSE a bit from build 84 to 91 so that it doesn't use the old **libmpc** from `meta-gplv2`. This allows many
components to be unblacklisted which weren't available because of that (most importantly, `python3-cryptography`, used by
Expand All @@ -960,17 +969,21 @@ ROS_SUPERFLORE_GENERATION_COMMIT = "<ROS-ROSDISTRO-COMMIT>"
**dashing**, and **eloquent**;
- `webos-image-ros-turtlebot3-all` (`DISTRO = "webos"`) builds successfully for `raspberrypi3` for **melodic**;
- The images pass the sanity tests.
- **COMPLETED**
### Milestone 10: Support latest releases of **melodic**, **crystal**, and **dashing**
- Incremental generation of recipes for latest Melodic (2019-12-10) and get to build.
- Incremental generation of recipes for latest Dashing (2019-12-12) and get to build.
- Incremental generation of recipes for latest Crystal (2019-12-12) and get to build.
### Milestone 10: Support latest releases of **melodic**, **crystal**, **dashing**, and **eloquent**
- Generate recipes for the latest **melodic** release (2019-12-10) and update its `.bbappend`-s so that it builds.
- Generate recipes for the latest **crystal** release (2019-12-12) and update its `.bbappend`-s so that it builds.
- Generate recipes for the latest **dashing** release (2019-12-12) and update its `.bbappend`-s so that it builds.
- Generate recipes for the latest **eloquent** release (2019-12-14) and update its `.bbappend`-s so that it builds.
- Move the recipes for components found in `files/<ROS_DISTRO>/newer-platform-components.list` to
`meta-ros-common/recipes-infrastructure`.
- Update the recipes for new versions of the infrastructure packages found in `newer-platform-components.list`.
- Have `ros-generate-cache.sh` write its version to the line 1 comments of `cache.* `and have `ros-generate-recipes` write its
- Have `ros-generate-cache.sh` write its version to the line 1 comments of `cache.* `and have `ros-generate-recipes.sh` write its
version to a variable `ROS_GENERATE_RECIPES_PROGRAM_VERSION` appended to `superflore-ros-distro.inc`.
- Done when:
- `ros-image-world` (`DISTRO = "ros2"`) and `webos-image-ros-world` (`DISTRO = "webos"`) build successfully for `qemux86` and
`raspberrypi3` for **crystal**, and **dashing**;
`raspberrypi3` for **crystal**, **dashing**, and **eloquent**;
- `ros-image-turtlebot3-all` (`DISTRO = "ros1"`) and `webos-image-ros-turtlebot3-all` (`DISTRO = "webos"`) build
successfully for `raspberrypi3` for **melodic**;
- The images pass the sanity tests.
Expand All @@ -988,6 +1001,9 @@ ROS_SUPERFLORE_GENERATION_COMMIT = "<ROS-ROSDISTRO-COMMIT>"
- Update the version differences tables for the platform packages provided by **warrior**.
- Update the tables in [REP-2000](https://www.ros.org/reps/rep-2000.html) for **warrior**.
- Add **swupdate** items to `meta-ros-webos/files/examples/webos-device-config/v1/rc.local`.
- Change the repo used to determine the OE version portion of `DISTRO_VERSION` for `DISTRO = "ros1"` and `DISTRO = "ros2"` from
`git.yoctoproject.org/poky.git` to `github.com/openembedded/openembedded-core.git` for OE-Core release series **warrior** and
later.
- Done when:
- `ros-image-world` (`DISTRO = "ros2"`) and `webos-image-ros-world` (`DISTRO = "webos"`) build successfully for `qemux86` and
`raspberrypi3` for **crystal**, and **dashing**, and **eloquent**;
Expand All @@ -998,8 +1014,8 @@ ROS_SUPERFLORE_GENERATION_COMMIT = "<ROS-ROSDISTRO-COMMIT>"
- Document what was done on a **How to Support a New OpenEmbedded Release** wiki page.
### Milestone 12: Implement layer version 3 (phase 2)
- Implement the items under **Phase 2** in the **Layer Version 3** section. If it turns out that **superflore** changes are
required, consider them to be ROS_SUPERFLORE_GENERATION_SCHEME 3.
- Implement the remainder items under **Phase 2** in the **Layer Version 3** section. If it turns out that **superflore** changes
are required, consider them to be ROS_SUPERFLORE_GENERATION_SCHEME 3.
- Support building from a locally modified source tree: add `inherit ${@ 'externalsrc' if d.getVar('EXTERNALSRC') else ''}` to
`ros_componment.bbclass`. Support regenerating recipes when `package.xml` is modified: implement
`ros-generate-recipes --from-local-source /path/to/package.xml`.
Expand All @@ -1020,7 +1036,7 @@ ROS_SUPERFLORE_GENERATION_COMMIT = "<ROS-ROSDISTRO-COMMIT>"
- The images pass the sanity tests.
### Milestone 13: Cleanup
- Finalize the commits (remove `DRAFT`, etc).
- Finalize the commits (remove `DRAFT`, fix up messages with `XXX`, etc).
- Finalize the `DISTRO_FEATURES` and `IMAGE_FEATURES` values; drop the `-trial` from them.
- Rename the branches to not be suffixed with `-draft`.
- Determine the root cause as to why the dependencies from `package.xml` are not always correct.
Expand Down Expand Up @@ -1182,6 +1198,15 @@ ROS_SUPERFLORE_GENERATION_COMMIT = "<ROS-ROSDISTRO-COMMIT>"
## Change Log
### Revision 45
- Add supporting the latest **eloquent** release to Milestone 10.
- Add moving the recipes for the components found in `files/<ROS_DISTRO>/newer-platform-components.list` to Milestone 10.
- Change how the OE version portion of `DISTRO_VERSION` for `DISTRO = "ros1"` and `DISTRO = "ros2"` is determined for OE-Core
release series **warrior** and later.
- Add the missing layer version 3 mapping for `recipes-core/packagegroups/packagegroup-ros-world-<ROS_DISTRO>.inc`.
- Indicate that Milestones 8 and 9 have been completed.
- Change how completed milestones are indicated so that their links aren't changed.
### Revision 44
- Update this document for the many changes introduced by layer version 3.
- Drop the **New Files and Changes to Existing Files** section because it describes the changes from version 1 to version 2.
Expand Down