Skip to content

Latest commit

 

History

History
executable file
·
235 lines (179 loc) · 11.6 KB

README.md

File metadata and controls

executable file
·
235 lines (179 loc) · 11.6 KB
Raw

License CI github release

Arista Network Test Automation (ANTA) Framework

This repository is a Python package to automate tests on Arista devices.

  • The package name is ANTA, which stands for Arista Network Test Automation.
  • This package (or some functions of this package) can be imported in Python scripts:
    • To automate NRFU (Network Ready For Use) test on a preproduction network
    • To automate tests on a live network (periodically or on demand)

ANTA provides a set of tests to validate the state of your network. All these tests are documented in the repository and can be used in your own python environment by importing this python package in your scripts.

This repository comes with a set of scripts to run Arista Network Test Automation (ANTA) framework

  • check-devices.py is an easy to use script to test your network with ANTA.
  • collect-eos-commands.py to collect commands output from devices
  • collect-sheduled-show-tech.py to collect the scheduled show tech-support files from devices

In addition you have also some useful scripts to help around testing:

  • clear-counters.py to clear counters on devices
  • evpn-blacklist-recovery.py to clear the list of MAC addresses which are blacklisted in EVPN
  • create-devices-inventory-from-cvp.py: Build inventory for scripts from Arista Cloudvision (CVP)

Most of these scripts use eAPI (EOS API). You can find examples of EOS automation with eAPI in this repository.

Installation

The easiest way to intall ANTA package is to run Python (>=3.7) and its pip package to install:

pip install git+https://github.com/arista-netdevops-community/network-test-automation.git

For more details about how to install package, please see the requirements and intallation docs.

Getting Started

This section shows how to use ANTA scripts with basic configuration. For more information, please refer to this page. Also a demo page is available in the repository with full outputs.

Configure Arista EOS devices

First, you need to configure your management interface

interface Management1
   description oob_management
   vrf MGMT
   ip address 10.73.1.105/24

Then, configure access to eAPI:

management api http-commands
   protocol https port 443
   no shutdown
   vrf MGMT
      no shutdown
   !
!

Create your inventory

First, we need to list devices we want to test. You can create a file manually with this format:

anta_inventory:
  hosts:
  - host: 192.168.0.10
  - host: 192.168.0.11
    # Optional tag to assign to this device
    tags: ['tag01', 'tag02']
  - host: 192.168.0.12
  - host: 192.168.0.13
  - host: 192.168.0.14
  - host: 192.168.0.15
  networks:
  - network: '192.168.110.0/24'
    # Optional tag to assign to all devices in this subnet
    tags: ['tag01', 'tag02']
  ranges:
  - start: 10.0.0.9
    end: 10.0.0.11
    # Optional tag to assign to all devices in this range
    tags: ['tag01', 'tag02']
  - start: 10.0.0.100
    end: 10.0.0.101

Or you can use create-devices-inventory-from-cvp.py script to generate from Cloudvision

# Available options
create-devices-inventory-from-cvp.py -h
usage: create-devices-inventory-from-cvp.py [-h] -cvp CVP -u USERNAME [-c CONTAINER] -o OUTPUT_DIRECTORY

Create devices inventory based on CVP containers

optional arguments:
  -h, --help           show this help message and exit
  -cvp CVP             CVP address
  -u USERNAME          CVP username
  -c CONTAINER         CVP container
  -o OUTPUT_DIRECTORY  Output directory

# Example
$ create-devices-inventory-from-cvp.py -cvp 192.168.0.5 -u arista -o inventory -c Spine

Test Catalog

To test your network, it is important to define a test catalog to list all the tests to run against your inventory. Test catalog references python functions into a yaml file. This file can be loaded by anta.loader.parse_catalog

The structure to follow is like:

<anta_tests_submodule>:
  - <anta_tests_submodule function name>:
      <test function option>:
        <test function option value>

Here is an example for basic things:

# Load anta.tests.software
software:
  - verify_eos_version: # Verifies the device is running one of the allowed EOS version.
      versions: # List of allowed EOS versions.
        - 4.25.4M
        - 4.26.1F

# Load anta.tests.system
system:
  - verify_uptime: # Verifies the device uptime is higher than a value.
      minimum: 1

# Load anta.tests.configuration
configuration:
  - verify_zerotouch: # Verifies ZeroTouch is disabled.
  - verify_running_config_diffs:

Test your network

To test EOS devices, this package comes with a generic script to run tests in your network. It requires an inventory file as well as a test catalog.

This script has multiple options to manage test coverage and reporting.

python scripts/check-devices.py -h

optional arguments:
  -h, --help            show this help message and exit
  --inventory INVENTORY, -i INVENTORY
                        ANTA Inventory file
  --catalog CATALOG, -c CATALOG
                        ANTA Tests catalog
  --username USERNAME, -u USERNAME
                        EOS Username
  --password PASSWORD, -p PASSWORD
                        EOS Password
  --enable_password ENABLE_PASSWORD, -e ENABLE_PASSWORD
                        EOS Enable Password
  --timeout TIMEOUT, -t TIMEOUT
                        eAPI connection timeout
  --hostip HOSTIP       search result for host
  --test TEST           search result for test
  --tags TAGS           List of device tags to limit scope of testing
  --list                Display internal data
  --json                Display data in json format
  --table               Result represented in tables
  --save SAVE           Save output to file. Only valid for --list and --json
  --all-results         Display all test cases results. Default table view (Only valid with --table)
  --by-host             Provides summary of test results per device (Only valid with --table)
  --by-test             Provides summary of test results per test case (Only valid with --table)

Default output is a table format listing all test results, and it can be changed to a report per test case or per host

Default report

$ check-devices.py -i .personal/avd-lab.yml -c .personal/ceos-catalog.yml --table

                             All tests results
┏━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┓
┃ Device IP     ┃ Test Name              ┃ Test Status ┃ Message(s)       ┃
┡━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━┩
│ 10.73.252.11  │ verify_mlag_interfaces │ success     │                  │
│ 10.73.252.12  │ verify_mlag_interfaces │ success     │                  │
│ 10.73.252.102 │ verify_mlag_interfaces │ skipped     │ MLAG is disabled │
└───────────────┴────────────────────────┴─────────────┴──────────────────┘

Report per test case

$ check-devices.py -i .personal/avd-lab.yml -c .personal/ceos-catalog.yml --table --by-test --test verify_mlag_status

                                              Summary per test case
┏━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Test Case          ┃ # of success ┃ # of skipped ┃ # of failure ┃ # of errors ┃ List of failed or error nodes ┃
┡━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ verify_mlag_status │ 8            │ 13           │ 0            │ 0           │ []                            │
└────────────────────┴──────────────┴──────────────┴──────────────┴─────────────┴───────────────────────────────┘

Report per host

$ check-devices.py -i .personal/avd-lab.yml -c .personal/ceos-catalog.yml --table --by-host --test verify_mlag_status --hostip 10.73.252.21

                                            Summary per host
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Host IP      ┃ # of success ┃ # of skipped ┃ # of failure ┃ # of errors ┃ List of failed ortest case ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ 10.73.252.21 │ 0            │ 1            │ 0            │ 0           │ []                         │
└──────────────┴──────────────┴──────────────┴──────────────┴─────────────┴────────────────────────────┘

You can find more information about usage in the following docs. Also a demo page is available in the repository with full outputs.

Contribution guide

Contributions are welcome. Please refer to the contribution guide

Credits

Thank you to Angélique Phillipps, Colin MacGiollaEáin, Khelil Sator, Matthieu Tache, Onur Gashi, Paul Lavelle, Guillaume Mulocher and Thomas Grimonet for their contributions and guidances.