Skip to content

Commit

Permalink
Merge pull request #546 from norlab-ulaval/dev-refactor-simplify-the-…
Browse files Browse the repository at this point in the history
…dockerfile-implementation-and-refactor-out-to-NBS-generic-install-logic-NMO-436

Build system improvements and fixes
  • Loading branch information
RedLeader962 committed Dec 18, 2023
2 parents 100e852 + 9048f30 commit 6ae246e
Show file tree
Hide file tree
Showing 33 changed files with 411 additions and 168 deletions.
35 changes: 35 additions & 0 deletions .env.libpointmatcher
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# =================================================================================================
#
# Set project related environment variables. Those are available for convenience
# and are also required by 'norlab-shell-script-tools' library.
#
# Usage:
#
# Important! Source this file from 'libpointmatcher' repository root
# $ cd <path/to/libpointmatcher/>
# $ set -o allexport && source .env.libpointmatcher && set +o allexport
#
# =================================================================================================

PROJECT_PROMPT_NAME='LPM'

# ....Programaticaly fetch source code information.................................................
PROJECT_GIT_REMOTE_URL="https://github.com/norlab-ulaval/libpointmatcher"
PROJECT_GIT_NAME=$( basename "${PROJECT_GIT_REMOTE_URL}" .git )
PROJECT_PATH=$( git rev-parse --show-toplevel )
PROJECT_SRC_NAME="$( basename ${PROJECT_PATH} )"

# ....Set LPM related environment variable with their own prefix...................................
# Note: Those with "PROJECT_" prefix will get eventualy overiden in the case where N2ST is used
# as a library. Using generic testing logic require that environment variables with
# "PROJECT_" prefix be available.
LPM_PROMPT_NAME="${PROJECT_PROMPT_NAME"
LPM_GIT_REMOTE_URL="${PROJECT_GIT_REMOTE_URL}"
LPM_GIT_NAME="${PROJECT_GIT_NAME}"
LPM_PATH="${PROJECT_PATH}"
LPM_SRC_NAME="${PROJECT_SRC_NAME}"

# ....Set dependencies path........................................................................
LPM_BUILD_SYSTEM_PATH="${PROJECT_PATH}/build_system"
N2ST_PATH="${PROJECT_PATH}/build_system/utilities/norlab-shell-script-tools"
NBS_PATH="${PROJECT_PATH}/build_system/utilities/norlab-build-system"
30 changes: 23 additions & 7 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
@@ -1,13 +1,29 @@
# =================================================================================================
# Files or directories with designated code owner.
# Branch with "Require review from Code Owners" will automaticaly trigger a request to a designated code owner
#
# Note:
# - Each line is a file pattern followed by one or more owners.
# - Branch with branch protection rule "Require review from Code Owners"
# will automaticaly trigger a request to a designated code owner
# =================================================================================================

# DevOps related
/build_system/ @RedLeader962
/.github/ @RedLeader962
/.dockerignore @RedLeader962
/.gitignore @RedLeader962
# ....Repository wide code owner...................................................................
# These owners will be the default owners for everything in the repo.
* @pomerlef

# Core
# ....Core directories and files...................................................................
/README.md @pomerlef
/LICENSE @pomerlef
/pointmatcher @boxanm @simonpierredeschenes @aguenette
/python @boxanm @simonpierredeschenes @aguenette
/utest @boxanm @simonpierredeschenes @aguenette
/.env.libpointmatcher @RedLeader962

# ....DevOps related...............................................................................
/.github/ @RedLeader962
/build_system/ @RedLeader962
/build_system/utilities @RedLeader962
/.gitmodules @RedLeader962
/.gitignore @RedLeader962
/.dockerignore @RedLeader962

47 changes: 47 additions & 0 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# Description
### Summary:






### Changes and type of changes:

-
-
-


---

# Checklist:

### Code related
- [ ] I have made corresponding changes to the documentation (i.e.: function, class, script header,
README.md)
- [ ] I have commented hard-to-understand code
- [ ] I have added tests that prove my fix is effective or that my feature works
- [ ] All tests pass locally with my changes (Check [README.md #Contributing](https://github.com/norlab-ulaval/libpointmatcher/tree/release#contributing) for local testing procedure using _libpointmatcher-build-system_)

### PR creation related
- [ ] My pull request `base ref` branch is set to the `develop` branch (the _build-system_ won't be triggered otherwise)
- [ ] My pull request branch is up-to-date with the `develop` branch (the _build-system_ will reject it otherwise)

### PR description related
- [ ] I have included a quick summary of the changes
- [ ] I have included a high-level list of changes and their corresponding type
- Types: `feat` `fix` `docs` `style` `refactor` `perf` `test` `build` `ci` `chore` `revert`
- Breaking changes: `<type>!`
- Reference:
- See [commit_msg_reference.md](https://github.com/norlab-ulaval/libpointmatcher/blob/release/commit_msg_reference.md) in the repository root for details
- https://www.conventionalcommits.org
- [ ] I have indicated the related issue's id with `# <issue-id>` if changes are of type `fix`

## Note for repository admins
### Release PR related
- Only repository admins have the privilege to `push/merge` on the default branch (ie: `master`) and the `release` branch.
- Keep PR in `draft` mode until all the release reviewers are ready to push the release.
- Once a PR from `release` -> `master` branch is created (not in draft mode),
- it triggers the _build-system_ test
- (in-progress) and it triggers the _semantic release automation_
2 changes: 1 addition & 1 deletion CHANGELOG.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
Changelog for package libpointmatcher
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

2.0.0 (2023-12-15)
1.4.0 (2023-12-15)
-----------------
* fix: N2ST path resolution in dependencies-doc docker image
* refactor: move libpointmatcher build-system logic to norlab-build-system submodule
Expand Down
2 changes: 1 addition & 1 deletion LICENSE
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
BSD 3-Clause License

Copyright (c) 2010-2021, the libpointmatcher authors
Copyright (c) 2010-2023, the libpointmatcher authors
All rights reserved.

Redistribution and use in source and binary forms, with or without
Expand Down
78 changes: 53 additions & 25 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,37 @@


---
`libpointmatcher` is currently tested on our build system with the following architecture and OS


`libpointmatcher` is tested on our build system under the following architecture and OS:
- x86 and arm64/v8
- Ubuntu bionic (18.04) and focal (20.04)
- Ubuntu bionic (18.04) and focal (20.04)

Note:
- support for Ubuntu jammy (22.04) comming soon
- `libpointmatcher` reportedly works on MacOs OsX (latest) and Windows (latest)

<div align="center">
<img src="https://img.shields.io/static/v1?label=powered by JetBrains TeamCity&message=CI/CD&color=green?style=plastic&logo=teamcity" />
</div>

<br>
<br>

<div style="padding: 20px;padding-top: 5px;padding-bottom: 5px; margin-top:20px;margin-bottom:20px; border: 2px solid LightGray; border-radius:10px;">

### ★ Version `1.4.0` release note (important)
This release of _libpointmatcher_ introduces the integration of [norlab-build-system (NBS)](https://github.com/norlab-ulaval/norlab-build-system) as a _git submodule_ for codebase development and testing.

Execute the following to clone the repository with its submodule:
```shell
git clone --recurse-submodules https://github.com/norlab-ulaval/libpointmatcher.git
```
If _libpointmatcher_ was previously cloned, execute the following to fetch its new submodule
```shell
git submodule update --remote --recursive --init
```
</div>

# Documentation and Tutorials

Expand All @@ -23,17 +51,10 @@ It is now maintained by the Northern Robotics Laboratory ([Norlab](https://norla

You can read the latest changes in the [release notes](doc/ReleaseNotes.md).

# Quick Start

Clone the repository with its submodule
```shell
git clone --recurse-submodules https://github.com/norlab-ulaval/libpointmatcher.git

# If libpointmatcher is already cloned, fetch its new submodule
git submodule update --remote --recursive --init
```


# Quick Start
Although we suggest to use the [tutorials](doc/index.md), here is a quick version of it:

The library has a light dependency list:
Expand Down Expand Up @@ -95,18 +116,22 @@ described [here](doc/ICPIntro.md) in more detail.

Please use our [github's issue tracker](http://github.com/ethz-asl/libpointmatcher/issues) to report bugs. If you are running the library on Ubuntu, copy-paste the output of the script [listVersionsUbuntu.sh](https://github.com/norlab-ulaval/libpointmatcher/blob/master/utest/listVersionsUbuntu.sh) to simplify the search of an answer.

## Codebase
## Codebase development

Libpointmatcher codebase now integrate [norlab-build-system (NBS)](https://github.com/norlab-ulaval/norlab-build-system) and [norlab-shell-script-tools (N2ST)](https://github.com/norlab-ulaval/norlab-shell-script-tools).
`NBS` is a build-infrastructure-agnostic build system custom-made to meet our needs in robotic software engineering at NorLab and `N2ST` is a library of shell script functions as well as a shell testing tools leveraging _**bats-core**_ and _**docker**_ .
`N2ST` purpose is to speed up shell script development and improve reliability.

Libpointmatcher codebase now integrate [norlab-build-system (NBS)](https://github.com/norlab-ulaval/norlab-build-system) and [norlab-shell-script-tools (N2ST)](https://github.com/norlab-ulaval/norlab-shell-script-tools). `NBS` is a build-infrastructure-agnostic build system custom-made for our need at NorLab and `N2ST` is library of shell script, function for shell script development and a shell testing tools leveraging `bats-core`.
`NBS` is deployed on our [TeamCity](https://www.jetbrains.com/teamcity/) continuous integration/deployment server and oversees protected branches of the [libpointmatcher](https://github.com/norlab-ulaval/libpointmatcher) GitHub repository:

NBS is deployed on our [TeamCity](https://www.jetbrains.com/teamcity/) continuous integration/deployment server and oversees both protected branch of the [libpointmatcher](https://github.com/norlab-ulaval/libpointmatcher) GitHub repository: the `master` branch and the `develop` branch.
- The `develop` branch can only be merged through a pull-request from any `<feature>` branches. Any contributor can submit a pull request to the `develop` branch;
- the `release` branch is a revision and preparation branch where we can freeze the codebase in a given state without stalling to `develop` branch progression;
- The `master` branch can only be merged through a pull-request from the `release` branch. Only repository admin can submit a PR to the `master` branch.

- The `develop` branch can only be merged through a pull-request from any `<feature>` branchs. Any contributor can submit a pull request to the `develop` branch;
- The `master` branch can only be merged from the `release` branch through a pull-request by a repository admin.
In both cases submiting a pull request will trigger a build configuration on our build system and the pull request will be granted if the build/test run succeede.
In any cases, submitting a pull request to `develop` or `master` will trigger a build/test configuration on our build system and the pull request will be granted if the build/test run succeed.

**Current build matrix:**
`[latest] x [x86, arm64] x [ubuntu] x [bionic, focal, jammy] x [Release, RelWithDebInfo, MinSizeRel]`
`[latest] x [x86, arm64] x [ubuntu] x [bionic, focal] x [Release, RelWithDebInfo, MinSizeRel]`

### Development workflow

Expand All @@ -133,16 +158,19 @@ bash lpm_create_multiarch_docker_builder.bash
```shell
cd <path/to/libpointmatcher>/build_system

# Run the build matrix as specified in ".env.build_matrix.libpointmatcher" on native architecture using "ci_PR" service
# Run the build matrix as specified in ".env.build_matrix.libpointmatcher"
# on native architecture using "ci_PR" service
bash lpm_crawl_libpointmatcher_build_matrix.bash --fail-fast -- build ci_PR

# Run a specific case using build flags with multi-architecture virtualization using "ci_PR_amd64" and "ci_PR_arm64v8" services
bash lpm_crawl_libpointmatcher_build_matrix.bash --repository-version-build-matrix-override latest \
--os-name-build-matrix-override ubuntu \
--cmake-build-type-build-matrix-override RelWithDebInfo \
--ubuntu-version-build-matrix-override focal \
--fail-fast \
-- build ci_PR_amd64 ci_PR_arm64v8
# Run a specific case using build flags with multi-architecture
# virtualization using "ci_PR_amd64" and "ci_PR_arm64v8" services
bash lpm_crawl_libpointmatcher_build_matrix.bash \
--repository-version-build-matrix-override latest \
--os-name-build-matrix-override ubuntu \
--cmake-build-type-build-matrix-override RelWithDebInfo \
--ubuntu-version-build-matrix-override focal \
--fail-fast \
-- build ci_PR_amd64 ci_PR_arm64v8

# Read the help for details
bash lpm_crawl_libpointmatcher_build_matrix.bash --help
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<component name="ProjectRunConfigurationManager">
<configuration default="false" name="(HELP) bash lpm_execute_compose.* › --help menu" type="ShConfigurationType" folderName="[LPM] B | dry-run">
<option name="SCRIPT_TEXT" value="bash lpm_crawl_libpointmatcher_build_matrix.bash --help &amp;&amp; bash nbs_execute_compose.bash --help" />
<configuration default="false" name="(HELP) bash lpm_crawl.* › --help menu" type="ShConfigurationType" folderName="[LPM] B | dry-run">
<option name="SCRIPT_TEXT" value="bash lpm_crawl_libpointmatcher_build_matrix.bash --help" />
<option name="INDEPENDENT_SCRIPT_PATH" value="true" />
<option name="SCRIPT_PATH" value="" />
<option name="SCRIPT_OPTIONS" value="" />
Expand Down

This file was deleted.

This file was deleted.

Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<component name="ProjectRunConfigurationManager">
<configuration default="false" name="bash lpm_crawl_libpointmatcher_build_matrix.bash › build &lt;service&gt; &lt;popup&gt;" type="ShConfigurationType" folderName="[LPM] A | Execute compose over build matrix" editBeforeRun="true">
<configuration default="false" name="bash lpm_crawl_libpointmatcher_build_matrix.bash › build &lt;service&gt; &lt;popup&gt;" type="ShConfigurationType" folderName="[LPM] C | Execute compose over build matrix" editBeforeRun="true">
<option name="SCRIPT_TEXT" value="bash lpm_crawl_libpointmatcher_build_matrix.bash --repository-version-build-matrix-override latest --os-name-build-matrix-override ubuntu --fail-fast -- build ci_PR_arm64v8" />
<option name="INDEPENDENT_SCRIPT_PATH" value="true" />
<option name="SCRIPT_PATH" value="" />
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<component name="ProjectRunConfigurationManager">
<configuration default="false" name="bash lpm_crawl_libpointmatcher_build_matrix.bash › build ci_PR_arm64v8 &lt;popup&gt;" type="ShConfigurationType" folderName="[LPM] A | Execute compose over build matrix" editBeforeRun="true">
<configuration default="false" name="bash lpm_crawl_libpointmatcher_build_matrix.bash › build ci_PR_arm64v8 &lt;popup&gt;" type="ShConfigurationType" folderName="[LPM] C | Execute compose over build matrix" editBeforeRun="true">
<option name="SCRIPT_TEXT" value="bash lpm_crawl_libpointmatcher_build_matrix.bash --repository-version-build-matrix-override latest --os-name-build-matrix-override ubuntu --cmake-build-type-build-matrix-override RelWithDebInfo --ubuntu-version-build-matrix-override focal --fail-fast -- build ci_PR_arm64v8" />
<option name="INDEPENDENT_SCRIPT_PATH" value="true" />
<option name="SCRIPT_PATH" value="" />
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<component name="ProjectRunConfigurationManager">
<configuration default="false" name="bash lpm_crawl_dependencies_build_matrix.bash › build dependencies-general dependencies (BUILD MATRIX SUBSET)" type="ShConfigurationType" folderName="[LPM] A | Execute compose over build matrix">
<configuration default="false" name="bash lpm_crawl_dependencies_build_matrix.bash › build dependencies-general dependencies (BUILD MATRIX SUBSET)" type="ShConfigurationType" folderName="[LPM] C | Execute compose over build matrix">
<option name="SCRIPT_TEXT" value="bash lpm_crawl_dependencies_build_matrix.bash --fail-fast --cmake-build-type-build-matrix-override None --os-name-build-matrix-override ubuntu --ubuntu-version-build-matrix-override focal -- build dependencies-general dependencies dependencies-doc" />
<option name="INDEPENDENT_SCRIPT_PATH" value="true" />
<option name="SCRIPT_PATH" value="" />
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<component name="ProjectRunConfigurationManager">
<configuration default="false" name="bash lpm_crawl_libpointmatcher_build_matrix.bash › build test_compilation_* (compilation test)" type="ShConfigurationType" folderName="[LPM] A | Execute compose over build matrix">
<configuration default="false" name="bash lpm_crawl_libpointmatcher_build_matrix.bash › build test_compilation_* (compilation test)" type="ShConfigurationType" folderName="[LPM] C | Execute compose over build matrix">
<option name="SCRIPT_TEXT" value="bash lpm_crawl_libpointmatcher_build_matrix.bash --fail-fast --cmake-build-type-build-matrix-override Release --os-name-build-matrix-override ubuntu -- build test_compilation_prefixed_path test_compilation_auto_path_resolution" />
<option name="INDEPENDENT_SCRIPT_PATH" value="true" />
<option name="SCRIPT_PATH" value="" />
Expand Down

0 comments on commit 6ae246e

Please sign in to comment.