Sync dev to main (#56)

* Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com>
google · Jun 28, 2023 · 098de20 · 098de20
1 parent 94e937f
commit 098de20
Show file tree

Hide file tree

Showing 10 changed files with 98 additions and 158 deletions.
diff --git a/cmd/start b/cmd/start
@@ -22,25 +22,4 @@ source venv/bin/activate
 export PYTHONPATH="$PWD/framework/python/src"
 python -u framework/python/src/core/test_runner.py $@
 
-# TODO: Work in progress code for containerization of OVS module
-# asyncRun() {
-#     "$@" &
-#     pid="$!"
-#     echo "PID Running: " $pid
-#     trap "echo 'Stopping PID $pid'; kill -SIGTERM $pid" SIGINT SIGTERM
-
-#     sleep 10
-
-#     # A signal emitted while waiting will make the wait command return code > 128
-#     # Let's wrap it in a loop that doesn't end before the process is indeed stopped
-#     while kill -0 $pid > /dev/null 2>&1; do
-#     #while $(kill -0 $pid 2>/dev/null); do
-#         wait
-#     done
-# }
-
-# # -u flag allows python print statements
-# # to be logged by docker by running unbuffered
-# asyncRun python3 -u python/src/run.py $@
-
 deactivate
diff --git a/framework/python/src/common/logger.py b/framework/python/src/common/logger.py
@@ -21,7 +21,7 @@
 _LOG_FORMAT = '%(asctime)s %(name)-8s %(levelname)-7s %(message)s'
 _DATE_FORMAT = '%b %02d %H:%M:%S'
 _DEFAULT_LEVEL = logging.INFO
-_CONF_DIR = 'conf'
+_CONF_DIR = 'local'
 _CONF_FILE_NAME = 'system.json'
 
 # Set log level

diff --git a/framework/python/src/common/util.py b/framework/python/src/common/util.py
@@ -13,6 +13,8 @@
 # limitations under the License.
 
 """Provides basic utilities for the network orchestrator."""
+import getpass
+import os
 import subprocess
 import shlex
 from common import logger
@@ -37,7 +39,7 @@ def run_command(cmd, output=True):
 
   if process.returncode != 0 and output:
     err_msg = f'{stderr.strip()}. Code: {process.returncode}'
-    LOGGER.error('Command Failed: ' + cmd)
+    LOGGER.error('Command failed: ' + cmd)
     LOGGER.error('Error: ' + err_msg)
   else:
     success = True
@@ -50,6 +52,44 @@ def run_command(cmd, output=True):
 def interface_exists(interface):
   return interface in netifaces.interfaces()
 
-
 def prettify(mac_string):
   return ':'.join([f'{ord(b):02x}' for b in mac_string])
+
+def get_host_user():
+  user = get_os_user()
+
+  # If primary method failed, try secondary
+  if user is None:
+    user = get_user()
+
+  return user
+
+def get_os_user():
+  user = None
+  try:
+    user = os.getlogin()
+  except OSError:
+    # Handle the OSError exception
+    LOGGER.error('An OS error occured whilst calling os.getlogin()')
+  except Exception:
+    # Catch any other unexpected exceptions
+    LOGGER.error('An unknown exception occured whilst calling os.getlogin()')
+  return user
+
+def get_user():
+  user = None
+  try:
+    user = getpass.getuser()
+  except (KeyError, ImportError, ModuleNotFoundError, OSError) as e:
+    # Handle specific exceptions individually
+    if isinstance(e, KeyError):
+      LOGGER.error('USER environment variable not set or unavailable.')
+    elif isinstance(e, ImportError):
+      LOGGER.error('Unable to import the getpass module.')
+    elif isinstance(e, ModuleNotFoundError):
+      LOGGER.error('The getpass module was not found.')
+    elif isinstance(e, OSError):
+      LOGGER.error('An OS error occurred while retrieving the username.')
+    else:
+      LOGGER.error('An exception occurred:', e)
+  return user
diff --git a/framework/python/src/core/device.py b/framework/python/src/core/device.py
@@ -22,6 +22,6 @@
 class Device(NetworkDevice):
   """Represents a physical device and it's configuration."""
 
-  make: str = None
+  manufacturer: str = None
   model: str = None
   test_modules: str = None
diff --git a/framework/python/src/core/testrun.py b/framework/python/src/core/testrun.py
@@ -25,7 +25,7 @@
 import json
 import signal
 import time
-from common import logger
+from common import logger, util
 
 # Locate parent directory
 current_dir = os.path.dirname(os.path.realpath(__file__))
@@ -46,7 +46,7 @@
 LOCAL_DEVICES_DIR = 'local/devices'
 RESOURCE_DEVICES_DIR = 'resources/devices'
 DEVICE_CONFIG = 'device_config.json'
-DEVICE_MAKE = 'make'
+DEVICE_MANUFACTURER = 'manufacturer'
 DEVICE_MODEL = 'model'
 DEVICE_MAC_ADDR = 'mac_addr'
 DEVICE_TEST_MODULES = 'test_modules'
@@ -76,7 +76,6 @@ def __init__(self,
     self._net_orc = net_orc.NetworkOrchestrator(
       config_file=config_file_abs,
       validate=validate,
-      async_monitor=not self._net_only,
       single_intf = self._single_intf)
 
     self._test_orc = test_orc.TestOrchestrator(self._net_orc)
@@ -85,17 +84,30 @@ def start(self):
 
     self._load_all_devices()
 
+    self._start_network()
+
     if self._net_only:
       LOGGER.info('Network only option configured, no tests will be run')
-      self._start_network()
+
+      self._net_orc.listener.register_callback(
+        self._device_discovered,
+        [NetworkEvent.DEVICE_DISCOVERED]
+      )
+
+      self._net_orc.start_listener()
+      LOGGER.info('Waiting for devices on the network...')
+
+      while True:
+        time.sleep(RUNTIME)
+
     else:
-      self._start_network()
       self._test_orc.start()
 
       self._net_orc.listener.register_callback(
           self._device_stable,
           [NetworkEvent.DEVICE_STABLE]
       )
+
       self._net_orc.listener.register_callback(
         self._device_discovered,
         [NetworkEvent.DEVICE_DISCOVERED]
@@ -106,13 +118,13 @@ def start(self):
 
       time.sleep(RUNTIME)
 
-      if not self._test_orc.test_in_progress():
-        LOGGER.info('Timed out whilst waiting for device')
+      if not (self._test_orc.test_in_progress() or self._net_orc.monitor_in_progress()):
+        LOGGER.info('Timed out whilst waiting for device or stopping due to test completion')
       else:
-        while self._test_orc.test_in_progress():
+        while self._test_orc.test_in_progress() or self._net_orc.monitor_in_progress():
           time.sleep(5)
 
-    self.stop()
+      self.stop()
 
   def stop(self, kill=False):
     self._stop_tests()
@@ -157,18 +169,19 @@ def _load_devices(self, device_dir):
     LOGGER.debug('Loading devices from ' + device_dir)
 
     os.makedirs(device_dir, exist_ok=True)
+    util.run_command(f'chown -R {util.get_host_user()} {device_dir}')
 
     for device_folder in os.listdir(device_dir):
       with open(os.path.join(device_dir, device_folder, DEVICE_CONFIG),
                 encoding='utf-8') as device_config_file:
         device_config_json = json.load(device_config_file)
 
-        device_make = device_config_json.get(DEVICE_MAKE)
+        device_manufacturer = device_config_json.get(DEVICE_MANUFACTURER)
         device_model = device_config_json.get(DEVICE_MODEL)
         mac_addr = device_config_json.get(DEVICE_MAC_ADDR)
         test_modules = device_config_json.get(DEVICE_TEST_MODULES)
 
-        device = Device(make=device_make,
+        device = Device(manufacturer=device_manufacturer,
                         model=device_model,
                         mac_addr=mac_addr,
                         test_modules=json.dumps(test_modules))
@@ -184,7 +197,7 @@ def _device_discovered(self, mac_addr):
     device = self.get_device(mac_addr)
     if device is not None:
       LOGGER.info(
-        f'Discovered {device.make} {device.model} on the network')
+        f'Discovered {device.manufacturer} {device.model} on the network')
     else:
       device = Device(mac_addr=mac_addr)
       self._devices.append(device)

diff --git a/framework/python/src/net_orc/network_orchestrator.py b/framework/python/src/net_orc/network_orchestrator.py
@@ -21,8 +21,6 @@
 import shutil
 import subprocess
 import sys
-import time
-import threading
 import docker
 from docker.types import Mount
 from common import logger
@@ -41,7 +39,6 @@
 TEST_DIR = 'test'
 MONITOR_PCAP = 'monitor.pcap'
 NET_DIR = 'runtime/network'
-#NETWORK_MODULES_DIR = 'network/modules'
 NETWORK_MODULES_DIR = 'modules/network'
 NETWORK_MODULE_METADATA = 'conf/module_config.json'
 DEVICE_BRIDGE = 'tr-d'
@@ -56,21 +53,18 @@
 DEFAULT_RUNTIME = 1200
 DEFAULT_MONITOR_PERIOD = 300
 
-RUNTIME = 1500
-
-
 class NetworkOrchestrator:
   """Manage and controls a virtual testing network."""
 
   def __init__(self,
                config_file=CONFIG_FILE,
                validate=True,
-               async_monitor=False,
                single_intf=False):
 
     self._runtime = DEFAULT_RUNTIME
     self._startup_timeout = DEFAULT_STARTUP_TIMEOUT
     self._monitor_period = DEFAULT_MONITOR_PERIOD
+    self._monitor_in_progress = False
 
     self._int_intf = None
     self._dev_intf = None
@@ -80,7 +74,6 @@ def __init__(self,
     self._net_modules = []
     self._devices = []
     self.validate = validate
-    self.async_monitor = async_monitor
 
     self._path = os.path.dirname(
         os.path.dirname(
@@ -99,7 +92,7 @@ def start(self):
 
     LOGGER.debug('Starting network orchestrator')
 
-    self._host_user = self._get_host_user()
+    self._host_user = util.get_host_user()
 
     # Get all components ready
     self.load_network_modules()
@@ -109,14 +102,6 @@ def start(self):
 
     self.start_network()
 
-    if self.async_monitor:
-      # Run the monitor method asynchronously to keep this method non-blocking
-      self._monitor_thread = threading.Thread(target=self.monitor_network)
-      self._monitor_thread.daemon = True
-      self._monitor_thread.start()
-    else:
-      self.monitor_network()
-
   def start_network(self):
     """Start the virtual testing network."""
     LOGGER.info('Starting network')
@@ -130,7 +115,7 @@ def start_network(self):
       self.validator.start()
 
     # Get network ready (via Network orchestrator)
-    LOGGER.info('Network is ready.')
+    LOGGER.debug('Network is ready')
 
   def start_listener(self):
     self.listener.start_listener()
@@ -151,13 +136,6 @@ def stop_network(self, kill=False):
     self.stop_networking_services(kill=kill)
     self.restore_net()
 
-  def monitor_network(self):
-    # TODO: This time should be configurable (How long to hold before exiting,
-    # this could be infinite too)
-    time.sleep(RUNTIME)
-
-    self.stop()
-
   def load_config(self, config_file=None):
     if config_file is None:
       # If not defined, use relative pathing to local file
@@ -178,8 +156,11 @@ def load_config(self, config_file=None):
 
   def _device_discovered(self, mac_addr):
 
+    self._monitor_in_progress = True
+
     LOGGER.debug(
         f'Discovered device {mac_addr}. Waiting for device to obtain IP')
+
     device = self._get_device(mac_addr=mac_addr)
 
     device_runtime_dir = os.path.join(RUNTIME_DIR, TEST_DIR,
@@ -204,6 +185,9 @@ def _device_discovered(self, mac_addr):
 
     self._start_device_monitor(device)
 
+  def monitor_in_progress(self):
+    return self._monitor_in_progress
+
   def _device_has_ip(self, packet):
     device = self._get_device(mac_addr=packet.src)
     if device is None or device.ip_addr is None:
@@ -225,6 +209,8 @@ def _start_device_monitor(self, device):
     wrpcap(
         os.path.join(RUNTIME_DIR, TEST_DIR, device.mac_addr.replace(':', ''),
                      'monitor.pcap'), packet_capture)
+
+    self._monitor_in_progress = False
     self.listener.call_callback(NetworkEvent.DEVICE_STABLE, device.mac_addr)
 
   def _get_device(self, mac_addr):
@@ -490,46 +476,6 @@ def _start_network_service(self, net_module):
     if network != 'host':
       self._attach_service_to_network(net_module)
 
-  def _get_host_user(self):
-    user = self._get_os_user()
-
-    # If primary method failed, try secondary
-    if user is None:
-      user = self._get_user()
-
-    LOGGER.debug("Network orchestrator host user: " + user)
-    return user
-
-  def _get_os_user(self):
-    user = None
-    try:
-      user = os.getlogin()
-    except OSError as e:
-      # Handle the OSError exception
-      LOGGER.error("An OS error occurred while retrieving the login name.")
-    except Exception as e:
-      # Catch any other unexpected exceptions
-      LOGGER.error("An exception occurred:", e)
-    return user
-
-  def _get_user(self):
-    user = None
-    try:
-      user = getpass.getuser()
-    except (KeyError, ImportError, ModuleNotFoundError, OSError) as e:
-      # Handle specific exceptions individually
-      if isinstance(e, KeyError):
-        LOGGER.error("USER environment variable not set or unavailable.")
-      elif isinstance(e, ImportError):
-        LOGGER.error("Unable to import the getpass module.")
-      elif isinstance(e, ModuleNotFoundError):
-        LOGGER.error("The getpass module was not found.")
-      elif isinstance(e, OSError):
-        LOGGER.error("An OS error occurred while retrieving the username.")
-      else:
-        LOGGER.error("An exception occurred:", e)
-    return user
-
   def _stop_service_module(self, net_module, kill=False):
     LOGGER.debug('Stopping Service container ' + net_module.container_name)
     try: