Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(anta): Add tag mapping between devices and tests #377

Merged
merged 29 commits into from
Oct 26, 2023

Conversation

titom73
Copy link
Collaborator

@titom73 titom73 commented Sep 6, 2023

Overview

The --tags option can be used to target specific devices in your inventory and run only tests configured with this specific tags from your catalog. Expected behaviour is provided below:

Command Description
none Run all tests on all devices according tag definition in your inventory and tests catalog. And tests with no tag are executed on all devices
--tags leaf Run all tests marked with leaf tag on all devices configured with leaf tag.
All other tags are ignored
--tags leaf,spine Run all tests marked with leaf tag on all devices configured with leaf tag.
Run all tests marked with spine tag on all devices configured with spine tag.
All other tags are ignored

warning
This PR is not implementing device filtering

Usage example

Inventory & Tests

Inventory

---
anta_inventory:
  hosts:
  - host: 192.168.0.10
    name: spine01
    tags: ['fabric', 'spine']
  - host: 192.168.0.11
    name: spine02
    tags: ['fabric', 'spine']
  - host: 192.168.0.12
    name: leaf01
    tags: ['fabric', 'leaf']
  - host: 192.168.0.13
    name: leaf02
    tags: ['fabric', 'leaf']
  - host: 192.168.0.14
    name: leaf03
    tags: ['fabric', 'leaf']
  - host: 192.168.0.15
    name: leaf04
    tags: ['fabric', 'leaf']

Tests

anta.tests.system:
  - VerifyUptime:
      minimum: 10
      filters:
        tags: ['fabric']
  - VerifyReloadCause:
      filters:
        tags: ['leaf', spine']
  - VerifyCoredump:
  - VerifyAgentLogs:
  - VerifyCPUUtilization:
      filters:
        tags: ['leaf']
  - VerifyMemoryUtilization:
  - VerifyFileSystemUtilization:
  - VerifyNTP:

anta.tests.mlag:
  - VerifyMlagStatus:

anta.tests.interfaces:
  - VerifyL3MTU:
      mtu: 1500
      filters:
        tags: ['demo']

CLI output

image

@titom73 titom73 added framework-enhancement New feature or request Anta CLI All things around CLI labels Sep 6, 2023
@titom73 titom73 added this to the v0.8.0 milestone Sep 6, 2023
@titom73 titom73 marked this pull request as ready for review September 8, 2023 14:48
Copy link
Collaborator

@gmuloc gmuloc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we probably need to add in the doc some example of "using tags" and saying what test will be executed or not

docs/usage-inventory-catalog.md Outdated Show resolved Hide resolved
anta/runner.py Outdated Show resolved Hide resolved
anta/runner.py Outdated Show resolved Hide resolved
anta/runner.py Outdated Show resolved Hide resolved
anta/runner.py Outdated Show resolved Hide resolved
anta/models.py Outdated Show resolved Hide resolved
anta/cli/nrfu/commands.py Outdated Show resolved Hide resolved
mtache
mtache previously requested changes Sep 19, 2023
Copy link
Collaborator

@mtache mtache left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think about having the logic below?
Init:

  • Assuming tag: leaf is applied to some devices and tests.
  • All tests are marked with tag all by default
  • All devices are not marked with any tag by default

Logic:

  • Tests with tag all are run on every devices
  • A test with tag all cannot have any other tag (all includes all the tags)
  • All the tests marked with tag: leaf are executed on all devices marked with tag: leaf
  • All tests that do not have any tags are skipped
  • All tests marked with a tag that do not match any device are skipped

anta/cli/nrfu/commands.py Outdated Show resolved Hide resolved
anta/models.py Outdated Show resolved Hide resolved
anta/models.py Outdated Show resolved Hide resolved
anta/runner.py Outdated Show resolved Hide resolved
@titom73
Copy link
Collaborator Author

titom73 commented Sep 20, 2023

I think we probably need to add in the doc some example of "using tags" and saying what test will be executed or not

I have added a dedicated page for tags and behaviours

@titom73 titom73 requested a review from gmuloc September 20, 2023 07:55
@github-actions
Copy link
Contributor

This pull request has conflicts, please resolve those before we can evaluate the pull request.

@github-actions
Copy link
Contributor

Conflicts have been resolved. A maintainer will review the pull request shortly.

@github-actions
Copy link
Contributor

This pull request has conflicts, please resolve those before we can evaluate the pull request.

@github-actions
Copy link
Contributor

Conflicts have been resolved. A maintainer will review the pull request shortly.

@github-actions
Copy link
Contributor

This pull request has conflicts, please resolve those before we can evaluate the pull request.

@github-actions
Copy link
Contributor

Conflicts have been resolved. A maintainer will review the pull request shortly.

Copy link
Collaborator

@mtache mtache left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ship it 🐜

docs/usage-inventory-catalog.md Outdated Show resolved Hide resolved
docs/cli/tag-management.md Outdated Show resolved Hide resolved
docs/cli/tag-management.md Outdated Show resolved Hide resolved
anta/runner.py Outdated Show resolved Hide resolved
mtache and others added 3 commits October 26, 2023 09:55
Co-authored-by: Guillaume Mulocher <gmulocher@arista.com>
Co-authored-by: Guillaume Mulocher <gmulocher@arista.com>
@mtache mtache merged commit abd22ac into aristanetworks:main Oct 26, 2023
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Anta CLI All things around CLI framework-enhancement New feature or request rn: feat(anta)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add map between inventory tags and tests to execute
3 participants