[MISC] Use pre-commit, bump min py version -> 3.8, Handle correctly t…

…he case of flexible format + single non-required topic. Also includes: * [MISC] Avoid running twice the CI jobs when doing a PR * [MISC] Handle correctly the case of flexible format + single non-required topic.
slamcore · Nov 7, 2023 · aa1d36f · aa1d36f
1 parent a8d2ab2
commit aa1d36f
Show file tree

Hide file tree

Showing 29 changed files with 534 additions and 592 deletions.
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
@@ -1,8 +1,12 @@
 name: CI
 
 on:
+  workflow_dispatch:
   push:
-  pull_request:
+    branches:
+      - "*"
+    tags:
+      - "*"
 
 jobs:
   install_with_pip3:
@@ -49,22 +53,16 @@ jobs:
         with:
           python-version: ${{ matrix.python-version }}
 
-      # Remove this section when this gets fixed
-      # https://stackoverflow.com/questions/76175361/firebase-authentication-httpresponse-object-has-no-attribute-strict-status
-      # https://github.com/ionrock/cachecontrol/issues/292
-      - name: Fix urllib / cachecontrol incompatibility
-        run: python -m pip install urllib3==1.26.15
-
       - name: Setup Python Poetry
         uses: abatilo/actions-poetry@v2.1.3
         with:
-          poetry-version: 1.6.1
+          poetry-version: "1.6.1"
       - name: Install prerequisites
         run: "poetry version && poetry install --extras ros2"
       - name: Run tests
         run: "source /opt/ros/humble/setup.bash && poetry run pytest"
 
-  check_style:
+  style_and_linters:
     runs-on: ubuntu-22.04
     steps:
       - uses: actions/checkout@v2
@@ -78,56 +76,22 @@ jobs:
         with:
           python-version: "3.8"
 
-      # Remove this section when this gets fixed
-      # https://stackoverflow.com/questions/76175361/firebase-authentication-httpresponse-object-has-no-attribute-strict-status
-      # https://github.com/ionrock/cachecontrol/issues/292
-      - name: Fix urllib / cachecontrol incompatibility
-        run: python -m pip install urllib3==1.26.15
-
-      - name: Setup Python Poetry
-        uses: abatilo/actions-poetry@v2.1.3
-        with:
-          poetry-version: 1.6.1
-      - name: Install prerequisites
-        run: poetry install --extras ros2
-      - name: Check style
-        run: "source /opt/ros/humble/setup.bash && poetry run ./misc/check_style.py --check-only"
-  lint_code:
-    runs-on: ubuntu-22.04
-    steps:
-      - uses: actions/checkout@v2
-      - name: Setup ROS
-        uses: ros-tooling/setup-ros@v0.6
-        with:
-          use-ros2-testing: false
-          required-ros-distributions: humble
-      - name: Setup Python
-        uses: actions/setup-python@v2
-        with:
-          python-version: "3.8"
-
-      # Remove this section when this gets fixed
-      # https://stackoverflow.com/questions/76175361/firebase-authentication-httpresponse-object-has-no-attribute-strict-status
-      # https://github.com/ionrock/cachecontrol/issues/292
-      - name: Fix urllib / cachecontrol incompatibility
-        run: python -m pip install urllib3==1.26.15
-
       - name: Setup Python Poetry
         uses: abatilo/actions-poetry@v2.1.3
         with:
-          poetry-version: 1.6.1
+          poetry-version: "1.6.1"
       - name: Install prerequisites
         run: poetry install --extras ros2
-      - name: Lint code
-        run: "source /opt/ros/humble/setup.bash && poetry run ./misc/lint_code.py --check-only"
+      - name: Run style checkers and linters
+        run: poetry run pre-commit run --all-files
 
   publish_package:
     runs-on: ubuntu-22.04
     steps:
       - uses: actions/checkout@v2
       - name: Publish package to pypi
-        uses: JRubics/poetry-publish@v1.9
+        uses: JRubics/poetry-publish@v1.16
         if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags')
         with:
-          python_version: "3.8"
           pypi_token: ${{ secrets.PYPI_API_TOKEN }}
+          plugins: "poetry-dynamic-versioning[plugin]"
diff --git a/.github/workflows/greetings.yml b/.github/workflows/greetings.yml
diff --git a/.markdown-link-check.json b/.markdown-link-check.json
@@ -0,0 +1,7 @@
+{
+  "ignorePatterns": [
+    {
+      "pattern": "^https://docs.slamcore.com/release_23.04/ros2-wrapper.html$"
+    }
+  ]
+}
diff --git a/.markdownlint.json b/.markdownlint.json
@@ -0,0 +1,6 @@
+{
+  "verbose": false,
+  "no-duplicate-header": {
+    "allow_different_nesting": true
+  }
+}
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
@@ -0,0 +1,53 @@
+# See https://pre-commit.com for more information
+# See https://pre-commit.com/hooks.html for more hooks
+repos:
+  - repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: "v4.4.0"
+    hooks:
+      - id: check-added-large-files
+      - id: check-ast
+      - id: check-merge-conflict
+      - id: check-toml
+      - id: debug-statements
+      - id: detect-private-key
+      - id: end-of-file-fixer
+      - id: mixed-line-ending
+      - id: trailing-whitespace
+  - repo: https://github.com/tcort/markdown-link-check
+    rev: "v3.10.3"
+    hooks:
+      # https://github.com/tcort/markdown-link-check#config-file-format
+      - id: markdown-link-check
+        args: ["--progress", "--config", ".markdown-link-check.json"]
+  - repo: https://github.com/markdownlint/markdownlint
+    rev: "v0.12.0"
+    hooks:
+      - id: markdownlint
+        # exclude: ^.github/
+        entry: mdl -c .markdownlint.json -r ~MD024,~MD033,~MD013,~MD034,~MD029,~MD007
+  - repo: local
+    hooks:
+      - id: black
+        name: Black formatter
+        description: "Black: The uncompromising Python code formatter"
+        entry: black
+        language: system
+        require_serial: true
+        types_or: [python, pyi]
+      - id: isort
+        name: Isort check
+        entry: isort
+        language: system
+        pass_filenames: true
+        types: [python]
+      - id: check-poetry
+        name: Poetry check
+        description: Validates the structure of the pyproject.toml file
+        entry: poetry check
+        language: system
+        pass_filenames: false
+        files: pyproject.toml
+  - repo: meta
+    hooks:
+      - id: check-hooks-apply
+      - id: check-useless-excludes
diff --git a/LICENSE.md b/LICENSE.md
@@ -27,4 +27,3 @@ SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
 CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
 OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
 OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
diff --git a/README.md b/README.md
@@ -99,7 +99,7 @@ slamcore-convert-rosbag2 \
 Notes:
 
 - A test `rosbag2` is included in the test directory of this repo. It's in the
-  [mcap](https://mcap.dev/getting-started/ros-2.html) format. For Ubuntu 20.04
+  [mcap](https://mcap.dev/guides/getting-started/ros-2) format. For Ubuntu 20.04
   and the galactic distribution, you will have to install the
   `ros-galactic-rosbag2-storage-mcap` package to process a `rosbag2` in this
   format.
@@ -170,8 +170,6 @@ Notes:
     - Converter plugins 0       : None
     - Overwrite output directory: True
 
-
-
   INFO | 1685528724.276752680 | rosbag2_storage | Opened database 'tests/test_data/trimmed_rosbag2/trimmed_rosbag2_0.db3' for READ_ONLY.
   INFO | 1685528724.360516990 | rosbag2_storage | Opened database 'tests/test_data/trimmed_rosbag2/trimmed_rosbag2_0.db3' for READ_ONLY.
   13:25:24 | WARNING  -  Finished converting tests/test_data/trimmed_rosbag2/trimmed_rosbag2_0.db3 -> output .
@@ -199,13 +197,10 @@ Notes:
     - Converter plugins 0       : None
     - Overwrite output directory: True
 
-
-
   INFO | 1685528761.113749019 | rosbag2_storage | Opened database 'tests/test_data/trimmed_rosbag2/trimmed_rosbag2_0.db3' for READ_ONLY.
   INFO | 1685528761.195022167 | rosbag2_storage | Opened database 'tests/test_data/trimmed_rosbag2/trimmed_rosbag2_0.db3' for READ_ONLY.
   13:26:01 | DEBUG    -  Rosbag metadata:
 
-
   Files:             tests/test_data/trimmed_rosbag2/trimmed_rosbag2_0.db3
   Bag size:          36.4 MiB
   Storage id:        sqlite3
@@ -252,7 +247,7 @@ given that you teach it how to convert them and what type of messages they are.
 You can specify this in a separate python module, and specify this module's
 path during the `slamcore-convert-rosbag2` execution. For example, here's one
 plugin to convert the `TrackingStatus` messages published by the [Slamcore ROS 2
-wrapper](https://docs.slamcore.com/release_23.01/ros2-wrapper.html)
+wrapper](https://docs.slamcore.com/release_23.04/ros2-wrapper.html)
 
 <details>
   <summary>Code in Rosbag2 Converter Plugin - plugin.py</summary>

diff --git a/misc/check_style.py b/misc/check_style.py
diff --git a/misc/lint_code.py b/misc/lint_code.py