2.0 (#29)

* initial work for 2.0 * implements #28 - context aware regex rules * updated docs and release notes * fix tests * Apply suggestions from code review Co-authored-by: Glenn Matthews <glenn.matthews@networktocode.com> * chore: Remove RQ (#37) - Removed all "rq" occurences - kept in `poetry.lock` Co-authored-by: Jan Snasel <jan.snasel@networktocode.com> * chore: Use NautobotFilterSet and bump nautobot 1.5 (#40) * chore: Use NautobotFilterSet and bump nautobot 1.5 - Potentially we could use nautobot v1.4 with NautobotFilterSet, however, to be sure, it seems to be better to use v1.5 * fix: Nautobot version in tasks.py --------- Co-authored-by: Jan Snasel <jan.snasel@networktocode.com> * Simple fixes to next branch (#38) * fix: remove .coverage * doc * fix: Missing logger name in nautobot config * doc: Update compatibility matrix - mark v2.0.0 to be compatible up to 1.99.99 * doc: Fix links to unique rules images * fix: Log error in case of jinja exception - `self.validation_error` raises an Exception and LOGGER would not be called this way * chore: Reformat lists - one item per line - to better recognize changes in reviews * cleanup: Unnecessary imports * fix: Text `required` used for `unique` validator objects * fix: Replace TestCase with NameSlugFilterTestCase * fix: Blacklisted fields validation typo --------- Co-authored-by: Jan Snasel <jan.snasel@networktocode.com> Co-authored-by: Steven <111259311+DistantVoyager@users.noreply.github.com> * Updated plugin files and structure using cookiecutter template (#44) * Used cookiecutter template to update tasks and dev * Updated root files to follow cookiecutter * Update docs to correspond to cookiecutter template * Modded pyproject.toml format to match cookiecutter * Updated github ci.yml using cookiecutter template * Minor fixes for black formatting * Removed unused imports for flake8 * Formatting fixes for pydocstyle D200, D202, D415 * Fixed pydocstyle docstring issues D101, D102, D106 * Removed tabs for black formatting * Updates to mkdocs and gitignore using cookiecutter * Updated .github dir using cookiecutter-ntc * Updated development dir using cookiecutter-ntc * Updated docs dir using cookiecutter-ntc * Updated inits using cookiecutter-ntc * Updated root source files using cookiecutter-ntc * Quick fixes for nautobot version * Use default ver in dockerfile; update nautobot min * Update poetry.lock * More changes to min nautobot 1.5.0 * Removed 1.4 from dockerfile nautobot ver * Bumped nautobot ver min to 1.5.2 for app rel v2.0 * Disabling some pylint checks to be addressed later * Handling for skipping a bugged view unit test * Minor formatting * Added newline for linting --------- Co-authored-by: Glenn Matthews <glenn.matthews@networktocode.com> Co-authored-by: Jan Snasel <snaselj@gmail.com> Co-authored-by: Jan Snasel <jan.snasel@networktocode.com> Co-authored-by: Steven <111259311+DistantVoyager@users.noreply.github.com>
nautobot · Apr 3, 2023 · b4c3bf7 · b4c3bf7
1 parent ef64c88
commit b4c3bf7
Show file tree

Hide file tree

Showing 83 changed files with 6,581 additions and 1,698 deletions.
diff --git a/.bandit.yml b/.bandit.yml
@@ -1,2 +1,6 @@
 ---
 skips: []
+# No need to check for security issues in the test scripts!
+exclude_dirs:
+  - "./tests/"
+  - "./.venv/"
diff --git a/.cookiecutter.json b/.cookiecutter.json
@@ -0,0 +1,25 @@
+{
+  "cookiecutter": {
+    "codeowner_github_usernames": "@smith-ntc",
+    "full_name": "Network to Code, LLC",
+    "email": "info@networktocode.com",
+    "github_org": "nautobot",
+    "plugin_name": "nautobot_data_validation_engine",
+    "verbose_name": "Data Validation Engine",
+    "plugin_slug": "nautobot-data-validation-engine",
+    "project_slug": "nautobot-plugin-data-validation-engine",
+    "repo_url": "https://github.com/nautobot/nautobot-plugin-data-validation-engine",
+    "base_url": "nautobot-data-validation-engine",
+    "min_nautobot_version": "1.5.2",
+    "max_nautobot_version": "1.9999",
+    "nautobot_version": "latest",
+    "camel_name": "NautobotDataValidationEngine",
+    "project_short_description": "Provides UI to build custom data validation rules for data in Nautobot",
+    "version": "2.0.0",
+    "model_class_name": "None",
+    "open_source_license": "Apache-2.0",
+    "docs_base_url": "https://docs.nautobot.com",
+    "docs_app_url": "https://docs.nautobot.com/projects/data-validation/en/latest",
+    "_template": "nautobot-plugin"
+  }
+}
diff --git a/.dockerignore b/.dockerignore
@@ -0,0 +1,27 @@
+# Docker related
+development/Dockerfile
+development/docker-compose*.yml
+development/*.env
+*.env
+environments/
+
+# Python
+**/*.pyc
+**/*.pyo
+**/__pycache__/
+**/.pytest_cache/
+**/.venv/
+
+
+# Other
+docs/_build
+FAQ.md
+.git/
+.gitignore
+.github
+tasks.py
+LICENSE
+**/*.log
+**/.vscode/
+invoke*.yml
+tasks.py
diff --git a/.flake8 b/.flake8
@@ -1,4 +1,10 @@
 [flake8]
 # E501: Line length is enforced by Black, so flake8 doesn't need to check it
 # W503: Black disagrees with this rule, as does PEP 8; Black wins
-ignore = E501, W503, F811, F401, F403, F405
+ignore = E501, W503
+exclude =
+  migrations,
+  __pycache__,
+  manage.py,
+  settings.py,
+  .venv
diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md
@@ -1,12 +1,19 @@
 ---
 name: 🐛 Bug Report
-about: Report a reproducible bug in the current release of nautobot-data-validation-engine 
+about: Report a reproducible bug in the current release of nautobot-data-validation-engine
 ---
 
 ### Environment
 * Python version:  <!-- Example: 3.7.7 -->
-* Nautobot version:  <!-- Example: 1.0.0 -->
-* nautobot-data-validation-engine version:  <!-- Example: 1.0.0 -->
+* Nautobot version:  <!-- Example: 1.5.0 -->
+* nautobot-data-validation-engine version:  <!-- Example: 2.0.0 -->
+
+<!-- What did you expect to happen? -->
+### Expected Behavior
+
+
+<!-- What happened instead? -->
+### Observed Behavior
 
 <!--
     Describe in detail the exact steps that someone else can take to reproduce
@@ -16,10 +23,3 @@ about: Report a reproducible bug in the current release of nautobot-data-validat
 1.
 2.
 3.
-
-<!-- What did you expect to happen? -->
-### Expected Behavior
-
-
-<!-- What happened instead? -->
-### Observed Behavior
diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md
@@ -5,8 +5,8 @@ about: Propose a new feature or enhancement
 ---
 
 ### Environment
-* Nautobot version:  <!-- Example: 1.0.0 -->
-* nautobot-data-validation-engine version:  <!-- Example: 1.0.0 -->
+* Nautobot version:  <!-- Example: 1.5.0 -->
+* nautobot-data-validation-engine version:  <!-- Example: 2.0.0 -->
 
 <!--
     Describe in detail the new functionality you are proposing.

diff --git a/.github/PULL_REQUEST_TEMPLATE/pull_request_template.md b/.github/PULL_REQUEST_TEMPLATE/pull_request_template.md
@@ -0,0 +1,10 @@
+## New Pull Request
+
+Have you:
+- [ ] Updated the README if necessary?
+- [ ] Updated any configuration settings?
+- [ ] Written a unit test?
+
+## Change Notes
+
+## Justification
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
@@ -1,8 +1,19 @@
 ---
 name: "CI"
-on: # yamllint disable
-  - "push"
-  - "pull_request"
+concurrency:  # Cancel any existing runs of this workflow for this same PR
+  group: "${{ github.workflow }}-${{ github.ref }}"
+  cancel-in-progress: true
+on: # yamllint disable-line rule:truthy rule:comments
+  push:
+    branches:
+      - "main"
+      - "develop"
+    tags:
+      - "v*"
+  pull_request: ~
+
+env:
+  PLUGIN_NAME: "nautobot-plugin-data-validation-engine"
 
 jobs:
   black:
@@ -13,7 +24,7 @@ jobs:
       - name: "Check out repository code"
         uses: "actions/checkout@v2"
       - name: "Setup environment"
-        uses: "networktocode/gh-action-setup-poetry-environment@v2"
+        uses: "networktocode/gh-action-setup-poetry-environment@v4"
       - name: "Linting: black"
         run: "poetry run invoke black"
   bandit:
@@ -24,11 +35,9 @@ jobs:
       - name: "Check out repository code"
         uses: "actions/checkout@v2"
       - name: "Setup environment"
-        uses: "networktocode/gh-action-setup-poetry-environment@v2"
+        uses: "networktocode/gh-action-setup-poetry-environment@v4"
       - name: "Linting: bandit"
         run: "poetry run invoke bandit"
-    needs:
-      - "black"
   pydocstyle:
     runs-on: "ubuntu-20.04"
     env:
@@ -37,11 +46,9 @@ jobs:
       - name: "Check out repository code"
         uses: "actions/checkout@v2"
       - name: "Setup environment"
-        uses: "networktocode/gh-action-setup-poetry-environment@v2"
-        # - name: "Linting: pydocstyle"
-        #   run: "poetry run invoke pydocstyle"
-    needs:
-      - "black"
+        uses: "networktocode/gh-action-setup-poetry-environment@v4"
+      - name: "Linting: pydocstyle"
+        run: "poetry run invoke pydocstyle"
   flake8:
     runs-on: "ubuntu-20.04"
     env:
@@ -50,11 +57,9 @@ jobs:
       - name: "Check out repository code"
         uses: "actions/checkout@v2"
       - name: "Setup environment"
-        uses: "networktocode/gh-action-setup-poetry-environment@v2"
+        uses: "networktocode/gh-action-setup-poetry-environment@v4"
       - name: "Linting: flake8"
         run: "poetry run invoke flake8"
-    needs:
-      - "black"
   yamllint:
     runs-on: "ubuntu-20.04"
     env:
@@ -63,44 +68,70 @@ jobs:
       - name: "Check out repository code"
         uses: "actions/checkout@v2"
       - name: "Setup environment"
-        uses: "networktocode/gh-action-setup-poetry-environment@v2"
+        uses: "networktocode/gh-action-setup-poetry-environment@v4"
       - name: "Linting: yamllint"
         run: "poetry run invoke yamllint"
-    needs:
-      - "black"
-  build:
-    runs-on: "ubuntu-20.04"
-    steps:
-      - name: "Check out repository code"
-        uses: "actions/checkout@v2"
-      - name: "Setup environment"
-        uses: "networktocode/gh-action-setup-poetry-environment@v2"
-      - name: "Build Container"
-        run: "poetry run invoke build"
+  pylint:
     needs:
       - "bandit"
       - "pydocstyle"
       - "flake8"
       - "yamllint"
-  pylint:
+      - "black"
     runs-on: "ubuntu-20.04"
+    strategy:
+      fail-fast: true
+      matrix:
+        python-version: ["3.8"]
+        nautobot-version: ["1.5.2"]
+    env:
+      INVOKE_NAUTOBOT_DATA_VALIDATION_ENGINE_PYTHON_VER: "${{ matrix.python-version }}"
+      INVOKE_NAUTOBOT_DATA_VALIDATION_ENGINE_NAUTOBOT_VER: "${{ matrix.nautobot-version }}"
     steps:
       - name: "Check out repository code"
         uses: "actions/checkout@v2"
       - name: "Setup environment"
-        uses: "networktocode/gh-action-setup-poetry-environment@v2"
-        # - name: "Build Container"
-        #   run: "poetry run invoke build"
-        # - name: "Linting: Pylint"
-        #   run: "poetry run invoke pylint"
-    needs:
-      - "build"
+        uses: "networktocode/gh-action-setup-poetry-environment@v4"
+      - name: "Set up Docker Buildx"
+        id: "buildx"
+        uses: "docker/setup-buildx-action@v1"
+      - name: "Build"
+        uses: "docker/build-push-action@v2"
+        with:
+          builder: "${{ steps.buildx.outputs.name }}"
+          context: "./"
+          push: false
+          load: true
+          tags: "${{ env.PLUGIN_NAME }}/nautobot:${{ matrix.nautobot-version }}-py${{ matrix.python-version }}"
+          file: "./development/Dockerfile"
+          cache-from: "type=gha,scope=${{ matrix.nautobot-version }}-py${{ matrix.python-version }}"
+          cache-to: "type=gha,scope=${{ matrix.nautobot-version }}-py${{ matrix.python-version }}"
+          build-args: |
+            NAUTOBOT_VER=${{ matrix.nautobot-version }}
+            PYTHON_VER=${{ matrix.python-version }}
+      - name: "Copy credentials"
+        run: "cp development/creds.example.env development/creds.env"
+      - name: "Linting: pylint"
+        run: "poetry run invoke pylint"
   unittest:
+    needs:
+      - "pylint"
     strategy:
       fail-fast: true
       matrix:
-        python-version: ["3.7", "3.8", "3.9"]
-        nautobot-version: ["1.0.1"]
+        python-version: ["3.7", "3.8", "3.9", "3.10"]
+        db-backend: ["postgresql"]
+        nautobot-version: ["stable"]
+        include:
+          - python-version: "3.10"
+            db-backend: "postgresql"
+            nautobot-version: "1.5.2"
+          - python-version: "3.7"
+            db-backend: "mysql"
+            nautobot-version: "1.5.2"
+          - python-version: "3.10"
+            db-backend: "mysql"
+            nautobot-version: "stable"
     runs-on: "ubuntu-20.04"
     env:
       INVOKE_NAUTOBOT_DATA_VALIDATION_ENGINE_PYTHON_VER: "${{ matrix.python-version }}"
@@ -109,14 +140,34 @@ jobs:
       - name: "Check out repository code"
         uses: "actions/checkout@v2"
       - name: "Setup environment"
-        uses: "networktocode/gh-action-setup-poetry-environment@v2"
-      - name: "Build Container"
-        run: "poetry run invoke build"
+        uses: "networktocode/gh-action-setup-poetry-environment@v4"
+      - name: "Set up Docker Buildx"
+        id: "buildx"
+        uses: "docker/setup-buildx-action@v1"
+      - name: "Build"
+        uses: "docker/build-push-action@v2"
+        with:
+          builder: "${{ steps.buildx.outputs.name }}"
+          context: "./"
+          push: false
+          load: true
+          tags: "${{ env.PLUGIN_NAME }}/nautobot:${{ matrix.nautobot-version }}-py${{ matrix.python-version }}"
+          file: "./development/Dockerfile"
+          cache-from: "type=gha,scope=${{ matrix.nautobot-version }}-py${{ matrix.python-version }}"
+          cache-to: "type=gha,scope=${{ matrix.nautobot-version }}-py${{ matrix.python-version }}"
+          build-args: |
+            NAUTOBOT_VER=${{ matrix.nautobot-version }}
+            PYTHON_VER=${{ matrix.python-version }}
+      - name: "Copy credentials"
+        run: "cp development/creds.example.env development/creds.env"
+      - name: "Use Mysql invoke settings when needed"
+        run: "cp invoke.mysql.yml invoke.yml"
+        if: "matrix.db-backend == 'mysql'"
       - name: "Run Tests"
         run: "poetry run invoke unittest"
-    needs:
-      - "pylint"
   publish_gh:
+    needs:
+      - "unittest"
     name: "Publish to GitHub"
     runs-on: "ubuntu-20.04"
     if: "startsWith(github.ref, 'refs/tags/v')"
@@ -143,9 +194,9 @@ jobs:
           tag: "${{ github.ref }}"
           overwrite: true
           file_glob: true
+  publish_pypi:
     needs:
       - "unittest"
-  publish_pypi:
     name: "Push Package to PyPI"
     runs-on: "ubuntu-20.04"
     if: "startsWith(github.ref, 'refs/tags/v')"
@@ -169,5 +220,38 @@ jobs:
         with:
           user: "__token__"
           password: "${{ secrets.PYPI_API_TOKEN }}"
+  slack-notify:
     needs:
-      - "unittest"
+      - "publish_gh"
+      - "publish_pypi"
+    runs-on: "ubuntu-20.04"
+    env:
+      SLACK_WEBHOOK_URL: "${{ secrets.SLACK_WEBHOOK_URL }}"
+      SLACK_MESSAGE: >-
+        *NOTIFICATION: NEW-RELEASE-PUBLISHED*\n
+        Repository: <${{ github.server_url }}/${{ github.repository }}|${{ github.repository }}>\n
+        Release: <${{ github.server_url }}/${{ github.repository }}/releases/tag/${{ github.ref_name }}|${{ github.ref_name }}>\n
+        Published by: <${{ github.server_url }}/${{ github.actor }}|${{ github.actor }}>
+    steps:
+      - name: "Send a notification to Slack"
+        # ENVs cannot be used directly in job.if. This is a workaround to check
+        # if SLACK_WEBHOOK_URL is present.
+        if: "env.SLACK_WEBHOOK_URL != ''"
+        uses: "slackapi/slack-github-action@v1.17.0"
+        with:
+          payload: |
+            {
+              "text": "${{ env.SLACK_MESSAGE }}",
+              "blocks": [
+                {
+                  "type": "section",
+                  "text": {
+                    "type": "mrkdwn",
+                    "text": "${{ env.SLACK_MESSAGE }}"
+                  }
+                }
+              ]
+            }
+        env:
+          SLACK_WEBHOOK_URL: "${{ secrets.SLACK_WEBHOOK_URL }}"
+          SLACK_WEBHOOK_TYPE: "INCOMING_WEBHOOK"