Add connection single ip test #47
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jboddey
approved these changes
Jun 12, 2023
jboddey
added a commit
that referenced
this pull request
Jun 27, 2023
* Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com>
jboddey
added a commit
that referenced
this pull request
Jun 28, 2023
* Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com>
jboddey
added a commit
that referenced
this pull request
Jul 12, 2023
* Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) * Sync dev to main (#56) * Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix missing results on udp tests when tcp ports are also defined (#59) * Add licence header (#61) * Resolve merge conflict * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com>
jboddey
added a commit
that referenced
this pull request
Jul 14, 2023
* Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) * Sync dev to main (#56) * Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix missing results on udp tests when tcp ports are also defined (#59) * Add licence header (#61) * Resolve merge conflict * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS * Connection private address (#71) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * formatting * Change isc-dhcp service setup Fix dhcpd logging Add start and stop methods to grpc dhcp client Add dhcp2 client Inttial private_addr test * Add max lease time Add unit tests * fix last commit * finish initial work on test * pylinting * Breakup test and allow better failure reporting * restore network after test * Wait for device to get a lease from original dhcp range after network restore * pylinting * Fix ipv6 tests --------- Co-authored-by: Jacob Boddey <boddey@google.com> * fix windows line ending * Fix python import * move isc-dhcp service commands to their own class update logging pylinting * fix dhcp1 * Initial CI testing for tests (#72) * Fix radvd conf --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com>
jboddey
added a commit
that referenced
this pull request
Jul 14, 2023
* Merge dev into main (Add license header) (#62) Add license header * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS * Merge dev into main (Sprint 9) (#66) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) * Sync dev to main (#56) * Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix missing results on udp tests when tcp ports are also defined (#59) * Add licence header (#61) * Resolve merge conflict * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Connection private address (#71) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * formatting * Change isc-dhcp service setup Fix dhcpd logging Add start and stop methods to grpc dhcp client Add dhcp2 client Inttial private_addr test * Add max lease time Add unit tests * fix last commit * finish initial work on test * pylinting * Breakup test and allow better failure reporting * restore network after test * Wait for device to get a lease from original dhcp range after network restore * pylinting * Fix ipv6 tests --------- Co-authored-by: Jacob Boddey <boddey@google.com> * fix windows line ending * Fix python import * move isc-dhcp service commands to their own class update logging pylinting * fix dhcp1 * Initial CI testing for tests (#72) * Fix radvd conf --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com>
jboddey
added a commit
that referenced
this pull request
Jul 24, 2023
* Start API * Write interfaces * Get current configuration * Set versions * Add more API methods * Correct no-ui flag * Do not launch API on baseline test * Move loading devices back to Test Run core * Merge dev into api (#74) * Merge dev into main (Add license header) (#62) Add license header * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS * Merge dev into main (Sprint 9) (#66) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) * Sync dev to main (#56) * Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix missing results on udp tests when tcp ports are also defined (#59) * Add licence header (#61) * Resolve merge conflict * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Connection private address (#71) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * formatting * Change isc-dhcp service setup Fix dhcpd logging Add start and stop methods to grpc dhcp client Add dhcp2 client Inttial private_addr test * Add max lease time Add unit tests * fix last commit * finish initial work on test * pylinting * Breakup test and allow better failure reporting * restore network after test * Wait for device to get a lease from original dhcp range after network restore * pylinting * Fix ipv6 tests --------- Co-authored-by: Jacob Boddey <boddey@google.com> * fix windows line ending * Fix python import * move isc-dhcp service commands to their own class update logging pylinting * fix dhcp1 * Initial CI testing for tests (#72) * Fix radvd conf --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix testing command * Disable API on testing * Add API session * Remove old method * Remove local vars * Replace old var * Add device config * Add device configs * Fix paths * Change MAC address * Revert mac * Fix copy path * Debug loading devices * Remove reference * Changes * Re-add checks to prevent null values * Fix variable * Fix * Use dict instead of string * Try without json conversion * Container output to log * Undo changes to nmap module * Add post devices route --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com>
jboddey
pushed a commit
that referenced
this pull request
Aug 4, 2023
jboddey
added a commit
that referenced
this pull request
Aug 4, 2023
* Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: Jonathan Hughes <jonathanhughes@google.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: Jonathan Hughes <jonathanhughes@google.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: Jonathan Hughes <jonathanhughes@google.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: Jonathan Hughes <jonathanhughes@google.com>
jboddey
added a commit
that referenced
this pull request
Aug 4, 2023
* Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: Jonathan Hughes <jonathanhughes@google.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: Jonathan Hughes <jonathanhughes@google.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: Jonathan Hughes <jonathanhughes@google.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: Jonathan Hughes <jonathanhughes@google.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: Jonathan Hughes <jonathanhughes@google.com>
jboddey
added a commit
that referenced
this pull request
Aug 4, 2023
* Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: Jonathan Hughes <jonathanhughes@google.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: Jonathan Hughes <jonathanhughes@google.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: Jonathan Hughes <jonathanhughes@google.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) * Sync dev to main (#56) * Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: Jonathan Hughes <jonathanhughes@google.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: Jonathan Hughes <jonathanhughes@google.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: Jonathan Hughes <jonathanhughes@google.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: Jonathan Hughes <jonathanhughes@google.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: Jonathan Hughes <jonathanhughes@google.com> * Fix missing results on udp tests when tcp ports are also defined (#59) * Add licence header (#61) * Resolve merge conflict * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: Jonathan Hughes <jonathanhughes@google.com>
jboddey
added a commit
that referenced
this pull request
Aug 4, 2023
* Start API * Write interfaces * Get current configuration * Set versions * Add more API methods * Correct no-ui flag * Do not launch API on baseline test * Move loading devices back to Test Run core * Merge dev into api (#74) * Merge dev into main (Add license header) (#62) Add license header * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS * Merge dev into main (Sprint 9) (#66) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: Jonathan Hughes <jonathanhughes@google.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: Jonathan Hughes <jonathanhughes@google.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: Jonathan Hughes <jonathanhughes@google.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) * Sync dev to main (#56) * Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: Jonathan Hughes <jonathanhughes@google.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: Jonathan Hughes <jonathanhughes@google.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: Jonathan Hughes <jonathanhughes@google.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: Jonathan Hughes <jonathanhughes@google.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: Jonathan Hughes <jonathanhughes@google.com> * Fix missing results on udp tests when tcp ports are also defined (#59) * Add licence header (#61) * Resolve merge conflict * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: Jonathan Hughes <jonathanhughes@google.com> * Connection private address (#71) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * formatting * Change isc-dhcp service setup Fix dhcpd logging Add start and stop methods to grpc dhcp client Add dhcp2 client Inttial private_addr test * Add max lease time Add unit tests * fix last commit * finish initial work on test * pylinting * Breakup test and allow better failure reporting * restore network after test * Wait for device to get a lease from original dhcp range after network restore * pylinting * Fix ipv6 tests --------- Co-authored-by: Jacob Boddey <boddey@google.com> * fix windows line ending * Fix python import * move isc-dhcp service commands to their own class update logging pylinting * fix dhcp1 * Initial CI testing for tests (#72) * Fix radvd conf --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: Jonathan Hughes <jonathanhughes@google.com> * Fix testing command * Disable API on testing * Add API session * Remove old method * Remove local vars * Replace old var * Add device config * Add device configs * Fix paths * Change MAC address * Revert mac * Fix copy path * Debug loading devices * Remove reference * Changes * Re-add checks to prevent null values * Fix variable * Fix * Use dict instead of string * Try without json conversion * Container output to log * Undo changes to nmap module * Add post devices route --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: Jonathan Hughes <jonathanhughes@google.com>
jboddey
added a commit
that referenced
this pull request
Aug 21, 2023
* Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) * Sync dev to main (#56) * Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix missing results on udp tests when tcp ports are also defined (#59) * Add licence header (#61) * Resolve merge conflict * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS * Connection private address (#71) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * formatting * Change isc-dhcp service setup Fix dhcpd logging Add start and stop methods to grpc dhcp client Add dhcp2 client Inttial private_addr test * Add max lease time Add unit tests * fix last commit * finish initial work on test * pylinting * Breakup test and allow better failure reporting * restore network after test * Wait for device to get a lease from original dhcp range after network restore * pylinting * Fix ipv6 tests --------- Co-authored-by: Jacob Boddey <boddey@google.com> * fix windows line ending * Fix python import * move isc-dhcp service commands to their own class update logging pylinting * fix dhcp1 * Initial CI testing for tests (#72) * Fix radvd conf * Fix individual test disable * Add NTP Pass CI test (#76) * add shared address test (#75) * Fix single ip test (#58) * Fix single ip test from detecting faux-device during validation as a failure * remove dhcp server capture file from scan --------- Co-authored-by: J Boddey <boddey@google.com> * Merge API into dev (#70) * Start API * Write interfaces * Get current configuration * Set versions * Add more API methods * Correct no-ui flag * Do not launch API on baseline test * Move loading devices back to Test Run core * Merge dev into api (#74) * Merge dev into main (Add license header) (#62) Add license header * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS * Merge dev into main (Sprint 9) (#66) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) * Sync dev to main (#56) * Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix missing results on udp tests when tcp ports are also defined (#59) * Add licence header (#61) * Resolve merge conflict * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Connection private address (#71) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * formatting * Change isc-dhcp service setup Fix dhcpd logging Add start and stop methods to grpc dhcp client Add dhcp2 client Inttial private_addr test * Add max lease time Add unit tests * fix last commit * finish initial work on test * pylinting * Breakup test and allow better failure reporting * restore network after test * Wait for device to get a lease from original dhcp range after network restore * pylinting * Fix ipv6 tests --------- Co-authored-by: Jacob Boddey <boddey@google.com> * fix windows line ending * Fix python import * move isc-dhcp service commands to their own class update logging pylinting * fix dhcp1 * Initial CI testing for tests (#72) * Fix radvd conf --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix testing command * Disable API on testing * Add API session * Remove old method * Remove local vars * Replace old var * Add device config * Add device configs * Fix paths * Change MAC address * Revert mac * Fix copy path * Debug loading devices * Remove reference * Changes * Re-add checks to prevent null values * Fix variable * Fix * Use dict instead of string * Try without json conversion * Container output to log * Undo changes to nmap module * Add post devices route --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Dhcp tests (#81) * Separate dhcp control methods into their own module Implement ip change test Add place holder for dhcp failover test * Stabilize network before leaving ip_change test Add dhcp_failover test * fix regression issue with individual test enable/disable setting * fix gitignore * Merge tls tests into dev (#80) * initial add of security module and tls tests * Fix server test and implement 1.3 version * pylinting * More work on client tests * Add client tls tests Add unit tets Add common python code to base test module * re-enable dhcp unit tests disab…
jboddey
added a commit
that referenced
this pull request
Sep 12, 2023
* Merge dev into main (Sprint 10 and 11) (#86) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) * Sync dev to main (#56) * Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix missing results on udp tests when tcp ports are also defined (#59) * Add licence header (#61) * Resolve merge conflict * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS * Connection private address (#71) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * formatting * Change isc-dhcp service setup Fix dhcpd logging Add start and stop methods to grpc dhcp client Add dhcp2 client Inttial private_addr test * Add max lease time Add unit tests * fix last commit * finish initial work on test * pylinting * Breakup test and allow better failure reporting * restore network after test * Wait for device to get a lease from original dhcp range after network restore * pylinting * Fix ipv6 tests --------- Co-authored-by: Jacob Boddey <boddey@google.com> * fix windows line ending * Fix python import * move isc-dhcp service commands to their own class update logging pylinting * fix dhcp1 * Initial CI testing for tests (#72) * Fix radvd conf * Fix individual test disable * Add NTP Pass CI test (#76) * add shared address test (#75) * Fix single ip test (#58) * Fix single ip test from detecting faux-device during validation as a failure * remove dhcp server capture file from scan --------- Co-authored-by: J Boddey <boddey@google.com> * Merge API into dev (#70) * Start API * Write interfaces * Get current configuration * Set versions * Add more API methods * Correct no-ui flag * Do not launch API on baseline test * Move loading devices back to Test Run core * Merge dev into api (#74) * Merge dev into main (Add license header) (#62) Add license header * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS * Merge dev into main (Sprint 9) (#66) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) * Sync dev to main (#56) * Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix missing results on udp tests when tcp ports are also defined (#59) * Add licence header (#61) * Resolve merge conflict * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Connection private address (#71) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * formatting * Change isc-dhcp service setup Fix dhcpd logging Add start and stop methods to grpc dhcp client Add dhcp2 client Inttial private_addr test * Add max lease time Add unit tests * fix last commit * finish initial work on test * pylinting * Breakup test and allow better failure reporting * restore network after test * Wait for device to get a lease from original dhcp range after network restore * pylinting * Fix ipv6 tests --------- Co-authored-by: Jacob Boddey <boddey@google.com> * fix windows line ending * Fix python import * move isc-dhcp service commands to their own class update logging pylinting * fix dhcp1 * Initial CI testing for tests (#72) * Fix radvd conf --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix testing command * Disable API on testing * Add API session * Remove old method * Remove local vars * Replace old var * Add device config * Add device configs * Fix paths * Change MAC address * Revert mac * Fix copy path * Debug loading devices * Remove reference * Changes * Re-add checks to prevent null values * Fix variable * Fix * Use dict instead of string * Try without json conversion * Container output to log * Undo changes to nmap module * Add post devices route --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Dhcp tests (#81) * Separate dhcp control methods into their own module Implement ip change test Add place holder for dhcp failover test * Stabilize network before leaving ip_change test Add dhcp_failover test * fix regression issue with individual test enable/disable setting * fix gitignore * Merge tls tests into dev (#80) * initial add of security module and tls tests * Fix server test and implement 1.3 version * pylinting * More work on client tests * Add client tls tests Add unit tets Add common python code to base test module …
jboddey
added a commit
that referenced
this pull request
Sep 28, 2023
* Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) * Sync dev to main (#56) * Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix missing results on udp tests when tcp ports are also defined (#59) * Add licence header (#61) * Resolve merge conflict * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS * Connection private address (#71) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * formatting * Change isc-dhcp service setup Fix dhcpd logging Add start and stop methods to grpc dhcp client Add dhcp2 client Inttial private_addr test * Add max lease time Add unit tests * fix last commit * finish initial work on test * pylinting * Breakup test and allow better failure reporting * restore network after test * Wait for device to get a lease from original dhcp range after network restore * pylinting * Fix ipv6 tests --------- Co-authored-by: Jacob Boddey <boddey@google.com> * fix windows line ending * Fix python import * move isc-dhcp service commands to their own class update logging pylinting * fix dhcp1 * Initial CI testing for tests (#72) * Fix radvd conf * Fix individual test disable * Add NTP Pass CI test (#76) * add shared address test (#75) * Fix single ip test (#58) * Fix single ip test from detecting faux-device during validation as a failure * remove dhcp server capture file from scan --------- Co-authored-by: J Boddey <boddey@google.com> * Merge API into dev (#70) * Start API * Write interfaces * Get current configuration * Set versions * Add more API methods * Correct no-ui flag * Do not launch API on baseline test * Move loading devices back to Test Run core * Merge dev into api (#74) * Merge dev into main (Add license header) (#62) Add license header * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS * Merge dev into main (Sprint 9) (#66) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) * Sync dev to main (#56) * Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix missing results on udp tests when tcp ports are also defined (#59) * Add licence header (#61) * Resolve merge conflict * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Connection private address (#71) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * formatting * Change isc-dhcp service setup Fix dhcpd logging Add start and stop methods to grpc dhcp client Add dhcp2 client Inttial private_addr test * Add max lease time Add unit tests * fix last commit * finish initial work on test * pylinting * Breakup test and allow better failure reporting * restore network after test * Wait for device to get a lease from original dhcp range after network restore * pylinting * Fix ipv6 tests --------- Co-authored-by: Jacob Boddey <boddey@google.com> * fix windows line ending * Fix python import * move isc-dhcp service commands to their own class update logging pylinting * fix dhcp1 * Initial CI testing for tests (#72) * Fix radvd conf --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix testing command * Disable API on testing * Add API session * Remove old method * Remove local vars * Replace old var * Add device config * Add device configs * Fix paths * Change MAC address * Revert mac * Fix copy path * Debug loading devices * Remove reference * Changes * Re-add checks to prevent null values * Fix variable * Fix * Use dict instead of string * Try without json conversion * Container output to log * Undo changes to nmap module * Add post devices route --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Dhcp tests (#81) * Separate dhcp control methods into their own module Implement ip change test Add place holder for dhcp failover test * Stabilize network before leaving ip_change test Add dhcp_failover test * fix regression issue with individual test enable/disable setting * fix gitignore * Merge tls tests into dev (#80) * initial add of security module and tls tests * Fix server test and implement 1.3 version * pylinting * More work on client tests * Add client tls tests Add unit tets Add common python code to base test module * re-enable dhcp unit tests disabled during dev …
jboddey
added a commit
that referenced
this pull request
Oct 16, 2023
* Merge dev into main (#117) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) * Sync dev to main (#56) * Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix missing results on udp tests when tcp ports are also defined (#59) * Add licence header (#61) * Resolve merge conflict * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS * Connection private address (#71) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * formatting * Change isc-dhcp service setup Fix dhcpd logging Add start and stop methods to grpc dhcp client Add dhcp2 client Inttial private_addr test * Add max lease time Add unit tests * fix last commit * finish initial work on test * pylinting * Breakup test and allow better failure reporting * restore network after test * Wait for device to get a lease from original dhcp range after network restore * pylinting * Fix ipv6 tests --------- Co-authored-by: Jacob Boddey <boddey@google.com> * fix windows line ending * Fix python import * move isc-dhcp service commands to their own class update logging pylinting * fix dhcp1 * Initial CI testing for tests (#72) * Fix radvd conf * Fix individual test disable * Add NTP Pass CI test (#76) * add shared address test (#75) * Fix single ip test (#58) * Fix single ip test from detecting faux-device during validation as a failure * remove dhcp server capture file from scan --------- Co-authored-by: J Boddey <boddey@google.com> * Merge API into dev (#70) * Start API * Write interfaces * Get current configuration * Set versions * Add more API methods * Correct no-ui flag * Do not launch API on baseline test * Move loading devices back to Test Run core * Merge dev into api (#74) * Merge dev into main (Add license header) (#62) Add license header * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS * Merge dev into main (Sprint 9) (#66) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) * Sync dev to main (#56) * Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix missing results on udp tests when tcp ports are also defined (#59) * Add licence header (#61) * Resolve merge conflict * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Connection private address (#71) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * formatting * Change isc-dhcp service setup Fix dhcpd logging Add start and stop methods to grpc dhcp client Add dhcp2 client Inttial private_addr test * Add max lease time Add unit tests * fix last commit * finish initial work on test * pylinting * Breakup test and allow better failure reporting * restore network after test * Wait for device to get a lease from original dhcp range after network restore * pylinting * Fix ipv6 tests --------- Co-authored-by: Jacob Boddey <boddey@google.com> * fix windows line ending * Fix python import * move isc-dhcp service commands to their own class update logging pylinting * fix dhcp1 * Initial CI testing for tests (#72) * Fix radvd conf --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix testing command * Disable API on testing * Add API session * Remove old method * Remove local vars * Replace old var * Add device config * Add device configs * Fix paths * Change MAC address * Revert mac * Fix copy path * Debug loading devices * Remove reference * Changes * Re-add checks to prevent null values * Fix variable * Fix * Use dict instead of string * Try without json conversion * Container output to log * Undo changes to nmap module * Add post devices route --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Dhcp tests (#81) * Separate dhcp control methods into their own module Implement ip change test Add place holder for dhcp failover test * Stabilize network before leaving ip_change test Add dhcp_failover test * fix regression issue with individual test enable/disable setting * fix gitignore * Merge tls tests into dev (#80) * initial add of security module and tls tests * Fix server test and implement 1.3 version * pylinting * More work on client tests * Add client tls tests Add unit tets Add common python code to base test module * re-enable dhcp uni…
jhughesbiot
added a commit
that referenced
this pull request
Oct 31, 2023
* Merge dev into main (#117) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) * Sync dev to main (#56) * Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix missing results on udp tests when tcp ports are also defined (#59) * Add licence header (#61) * Resolve merge conflict * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS * Connection private address (#71) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * formatting * Change isc-dhcp service setup Fix dhcpd logging Add start and stop methods to grpc dhcp client Add dhcp2 client Inttial private_addr test * Add max lease time Add unit tests * fix last commit * finish initial work on test * pylinting * Breakup test and allow better failure reporting * restore network after test * Wait for device to get a lease from original dhcp range after network restore * pylinting * Fix ipv6 tests --------- Co-authored-by: Jacob Boddey <boddey@google.com> * fix windows line ending * Fix python import * move isc-dhcp service commands to their own class update logging pylinting * fix dhcp1 * Initial CI testing for tests (#72) * Fix radvd conf * Fix individual test disable * Add NTP Pass CI test (#76) * add shared address test (#75) * Fix single ip test (#58) * Fix single ip test from detecting faux-device during validation as a failure * remove dhcp server capture file from scan --------- Co-authored-by: J Boddey <boddey@google.com> * Merge API into dev (#70) * Start API * Write interfaces * Get current configuration * Set versions * Add more API methods * Correct no-ui flag * Do not launch API on baseline test * Move loading devices back to Test Run core * Merge dev into api (#74) * Merge dev into main (Add license header) (#62) Add license header * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS * Merge dev into main (Sprint 9) (#66) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) * Sync dev to main (#56) * Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix missing results on udp tests when tcp ports are also defined (#59) * Add licence header (#61) * Resolve merge conflict * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Connection private address (#71) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * formatting * Change isc-dhcp service setup Fix dhcpd logging Add start and stop methods to grpc dhcp client Add dhcp2 client Inttial private_addr test * Add max lease time Add unit tests * fix last commit * finish initial work on test * pylinting * Breakup test and allow better failure reporting * restore network after test * Wait for device to get a lease from original dhcp range after network restore * pylinting * Fix ipv6 tests --------- Co-authored-by: Jacob Boddey <boddey@google.com> * fix windows line ending * Fix python import * move isc-dhcp service commands to their own class update logging pylinting * fix dhcp1 * Initial CI testing for tests (#72) * Fix radvd conf --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix testing command * Disable API on testing * Add API session * Remove old method * Remove local vars * Replace old var * Add device config * Add device configs * Fix paths * Change MAC address * Revert mac * Fix copy path * Debug loading devices * Remove reference * Changes * Re-add checks to prevent null values * Fix variable * Fix * Use dict instead of string * Try without json conversion * Container output to log * Undo changes to nmap module * Add post devices route --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Dhcp tests (#81) * Separate dhcp control methods into their own module Implement ip change test Add place holder for dhcp failover test * Stabilize network before leaving ip_change test Add dhcp_failover test * fix regression issue with individual test enable/disable setting * fix gitignore * Merge tls tests into dev (#80) * initial add of security module and tls tests * Fix server test and implement 1.3 version * pylinting * More work on client tests * Add client tls tests Add unit tets Add common python code to base test module * re-enable …
jboddey
added a commit
that referenced
this pull request
Jan 31, 2024
* Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) * Sync dev to main (#56) * Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix missing results on udp tests when tcp ports are also defined (#59) * Add licence header (#61) * Resolve merge conflict * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS * Connection private address (#71) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * formatting * Change isc-dhcp service setup Fix dhcpd logging Add start and stop methods to grpc dhcp client Add dhcp2 client Inttial private_addr test * Add max lease time Add unit tests * fix last commit * finish initial work on test * pylinting * Breakup test and allow better failure reporting * restore network after test * Wait for device to get a lease from original dhcp range after network restore * pylinting * Fix ipv6 tests --------- Co-authored-by: Jacob Boddey <boddey@google.com> * fix windows line ending * Fix python import * move isc-dhcp service commands to their own class update logging pylinting * fix dhcp1 * Initial CI testing for tests (#72) * Fix radvd conf * Fix individual test disable * Add NTP Pass CI test (#76) * add shared address test (#75) * Fix single ip test (#58) * Fix single ip test from detecting faux-device during validation as a failure * remove dhcp server capture file from scan --------- Co-authored-by: J Boddey <boddey@google.com> * Merge API into dev (#70) * Start API * Write interfaces * Get current configuration * Set versions * Add more API methods * Correct no-ui flag * Do not launch API on baseline test * Move loading devices back to Test Run core * Merge dev into api (#74) * Merge dev into main (Add license header) (#62) Add license header * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS * Merge dev into main (Sprint 9) (#66) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) * Sync dev to main (#56) * Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix missing results on udp tests when tcp ports are also defined (#59) * Add licence header (#61) * Resolve merge conflict * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Connection private address (#71) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * formatting * Change isc-dhcp service setup Fix dhcpd logging Add start and stop methods to grpc dhcp client Add dhcp2 client Inttial private_addr test * Add max lease time Add unit tests * fix last commit * finish initial work on test * pylinting * Breakup test and allow better failure reporting * restore network after test * Wait for device to get a lease from original dhcp range after network restore * pylinting * Fix ipv6 tests --------- Co-authored-by: Jacob Boddey <boddey@google.com> * fix windows line ending * Fix python import * move isc-dhcp service commands to their own class update logging pylinting * fix dhcp1 * Initial CI testing for tests (#72) * Fix radvd conf --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix testing command * Disable API on testing * Add API session * Remove old method * Remove local vars * Replace old var * Add device config * Add device configs * Fix paths * Change MAC address * Revert mac * Fix copy path * Debug loading devices * Remove reference * Changes * Re-add checks to prevent null values * Fix variable * Fix * Use dict instead of string * Try without json conversion * Container output to log * Undo changes to nmap module * Add post devices route --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Dhcp tests (#81) * Separate dhcp control methods into their own module Implement ip change test Add place holder for dhcp failover test * Stabilize network before leaving ip_change test Add dhcp_failover test * fix regression issue with individual test enable/disable setting * fix gitignore * Merge tls tests into dev (#80) * initial add of security module and tls tests * Fix server test and implement 1.3 version * pylinting * More work on client tests * Add client tls tests Add unit tets Add common python code to base test module * re-enable dhcp unit tests disabled dur…
jboddey
added a commit
that referenced
this pull request
Feb 22, 2024
* Merge dev into main (Release 1.1-alpha) (#182) * Update README.md (#232) Signed-off-by: J Boddey <boddey@google.com> * Merge release v1.1.1 into main (#229) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) * Sync dev to main (#56) * Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix missing results on udp tests when tcp ports are also defined (#59) * Add licence header (#61) * Resolve merge conflict * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS * Connection private address (#71) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * formatting * Change isc-dhcp service setup Fix dhcpd logging Add start and stop methods to grpc dhcp client Add dhcp2 client Inttial private_addr test * Add max lease time Add unit tests * fix last commit * finish initial work on test * pylinting * Breakup test and allow better failure reporting * restore network after test * Wait for device to get a lease from original dhcp range after network restore * pylinting * Fix ipv6 tests --------- Co-authored-by: Jacob Boddey <boddey@google.com> * fix windows line ending * Fix python import * move isc-dhcp service commands to their own class update logging pylinting * fix dhcp1 * Initial CI testing for tests (#72) * Fix radvd conf * Fix individual test disable * Add NTP Pass CI test (#76) * add shared address test (#75) * Fix single ip test (#58) * Fix single ip test from detecting faux-device during validation as a failure * remove dhcp server capture file from scan --------- Co-authored-by: J Boddey <boddey@google.com> * Merge API into dev (#70) * Start API * Write interfaces * Get current configuration * Set versions * Add more API methods * Correct no-ui flag * Do not launch API on baseline test * Move loading devices back to Test Run core * Merge dev into api (#74) * Merge dev into main (Add license header) (#62) Add license header * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS * Merge dev into main (Sprint 9) (#66) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) * Sync dev to main (#56) * Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix missing results on udp tests when tcp ports are also defined (#59) * Add licence header (#61) * Resolve merge conflict * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Connection private address (#71) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * formatting * Change isc-dhcp service setup Fix dhcpd logging Add start and stop methods to grpc dhcp client Add dhcp2 client Inttial private_addr test * Add max lease time Add unit tests * fix last commit * finish initial work on test * pylinting * Breakup test and allow better failure reporting * restore network after test * Wait for device to get a lease from original dhcp range after network restore * pylinting * Fix ipv6 tests --------- Co-authored-by: Jacob Boddey <boddey@google.com> * fix windows line ending * Fix python import * move isc-dhcp service commands to their own class update logging pylinting * fix dhcp1 * Initial CI testing for tests (#72) * Fix radvd conf --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix testing command * Disable API on testing * Add API session * Remove old method * Remove local vars * Replace old var * Add device config * Add device configs * Fix paths * Change MAC address * Revert mac * Fix copy path * Debug loading devices * Remove reference * Changes * Re-add checks to prevent null values * Fix variable * Fix * Use dict instead of string * Try without json conversion * Container output to log * Undo changes to nmap module * Add post devices route --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Dhcp tests (#81) * Separate dhcp control methods into their own module Implement ip change test Add place holder for dhcp failover test * Stabilize network before leaving ip_change test Add dhcp_failover test * fix regression issue with individual test enable/disable setting * fix gitignore * Merge tls tests into dev (#80) * initial add of security module and tls tests * Fix server test and implement 1.3 version *…
jboddey
added a commit
that referenced
this pull request
Apr 6, 2024
* Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) * Sync dev to main (#56) * Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix missing results on udp tests when tcp ports are also defined (#59) * Add licence header (#61) * Resolve merge conflict * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS * Connection private address (#71) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * formatting * Change isc-dhcp service setup Fix dhcpd logging Add start and stop methods to grpc dhcp client Add dhcp2 client Inttial private_addr test * Add max lease time Add unit tests * fix last commit * finish initial work on test * pylinting * Breakup test and allow better failure reporting * restore network after test * Wait for device to get a lease from original dhcp range after network restore * pylinting * Fix ipv6 tests --------- Co-authored-by: Jacob Boddey <boddey@google.com> * fix windows line ending * Fix python import * move isc-dhcp service commands to their own class update logging pylinting * fix dhcp1 * Initial CI testing for tests (#72) * Fix radvd conf * Fix individual test disable * Add NTP Pass CI test (#76) * add shared address test (#75) * Fix single ip test (#58) * Fix single ip test from detecting faux-device during validation as a failure * remove dhcp server capture file from scan --------- Co-authored-by: J Boddey <boddey@google.com> * Merge API into dev (#70) * Start API * Write interfaces * Get current configuration * Set versions * Add more API methods * Correct no-ui flag * Do not launch API on baseline test * Move loading devices back to Test Run core * Merge dev into api (#74) * Merge dev into main (Add license header) (#62) Add license header * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS * Merge dev into main (Sprint 9) (#66) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) * Sync dev to main (#56) * Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix missing results on udp tests when tcp ports are also defined (#59) * Add licence header (#61) * Resolve merge conflict * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Connection private address (#71) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * formatting * Change isc-dhcp service setup Fix dhcpd logging Add start and stop methods to grpc dhcp client Add dhcp2 client Inttial private_addr test * Add max lease time Add unit tests * fix last commit * finish initial work on test * pylinting * Breakup test and allow better failure reporting * restore network after test * Wait for device to get a lease from original dhcp range after network restore * pylinting * Fix ipv6 tests --------- Co-authored-by: Jacob Boddey <boddey@google.com> * fix windows line ending * Fix python import * move isc-dhcp service commands to their own class update logging pylinting * fix dhcp1 * Initial CI testing for tests (#72) * Fix radvd conf --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix testing command * Disable API on testing * Add API session * Remove old method * Remove local vars * Replace old var * Add device config * Add device configs * Fix paths * Change MAC address * Revert mac * Fix copy path * Debug loading devices * Remove reference * Changes * Re-add checks to prevent null values * Fix variable * Fix * Use dict instead of string * Try without json conversion * Container output to log * Undo changes to nmap module * Add post devices route --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Dhcp tests (#81) * Separate dhcp control methods into their own module Implement ip change test Add place holder for dhcp failover test * Stabilize network before leaving ip_change test Add dhcp_failover test * fix regression issue with individual test enable/disable setting * fix gitignore * Merge tls tests into dev (#80) * initial add of security module and tls tests * Fix server test and implement 1.3 version * pylinting * More work on client tests * Add client tls tests Add unit tets Add common python code to base test module * re-enable dhcp unit tests disabled…
jboddey
added a commit
that referenced
this pull request
May 10, 2024
* Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) * Sync dev to main (#56) * Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix missing results on udp tests when tcp ports are also defined (#59) * Add licence header (#61) * Resolve merge conflict * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS * Connection private address (#71) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * formatting * Change isc-dhcp service setup Fix dhcpd logging Add start and stop methods to grpc dhcp client Add dhcp2 client Inttial private_addr test * Add max lease time Add unit tests * fix last commit * finish initial work on test * pylinting * Breakup test and allow better failure reporting * restore network after test * Wait for device to get a lease from original dhcp range after network restore * pylinting * Fix ipv6 tests --------- Co-authored-by: Jacob Boddey <boddey@google.com> * fix windows line ending * Fix python import * move isc-dhcp service commands to their own class update logging pylinting * fix dhcp1 * Initial CI testing for tests (#72) * Fix radvd conf * Fix individual test disable * Add NTP Pass CI test (#76) * add shared address test (#75) * Fix single ip test (#58) * Fix single ip test from detecting faux-device during validation as a failure * remove dhcp server capture file from scan --------- Co-authored-by: J Boddey <boddey@google.com> * Merge API into dev (#70) * Start API * Write interfaces * Get current configuration * Set versions * Add more API methods * Correct no-ui flag * Do not launch API on baseline test * Move loading devices back to Test Run core * Merge dev into api (#74) * Merge dev into main (Add license header) (#62) Add license header * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS * Merge dev into main (Sprint 9) (#66) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) * Sync dev to main (#56) * Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix missing results on udp tests when tcp ports are also defined (#59) * Add licence header (#61) * Resolve merge conflict * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Connection private address (#71) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * formatting * Change isc-dhcp service setup Fix dhcpd logging Add start and stop methods to grpc dhcp client Add dhcp2 client Inttial private_addr test * Add max lease time Add unit tests * fix last commit * finish initial work on test * pylinting * Breakup test and allow better failure reporting * restore network after test * Wait for device to get a lease from original dhcp range after network restore * pylinting * Fix ipv6 tests --------- Co-authored-by: Jacob Boddey <boddey@google.com> * fix windows line ending * Fix python import * move isc-dhcp service commands to their own class update logging pylinting * fix dhcp1 * Initial CI testing for tests (#72) * Fix radvd conf --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix testing command * Disable API on testing * Add API session * Remove old method * Remove local vars * Replace old var * Add device config * Add device configs * Fix paths * Change MAC address * Revert mac * Fix copy path * Debug loading devices * Remove reference * Changes * Re-add checks to prevent null values * Fix variable * Fix * Use dict instead of string * Try without json conversion * Container output to log * Undo changes to nmap module * Add post devices route --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Dhcp tests (#81) * Separate dhcp control methods into their own module Implement ip change test Add place holder for dhcp failover test * Stabilize network before leaving ip_change test Add dhcp_failover test * fix regression issue with individual test enable/disable setting * fix gitignore * Merge tls tests into dev (#80) * initial add of security module and tls tests * Fix server test and implement 1.3 version * pylinting * More work on client tests * Add client tls tests Add unit tets Add common python code to base test module * re-enable dhcp unit tests disabled during dev * rename module to tls * fix renaming * Fix unit tests broken by module rename Add TLS 1.3 tests to config * Add TLS 1.3 tests to config fix unit tests * Add certificate signature checks * Add local cert mounting for signature validatoin Fix test results * Update tls 1.2 server to pass with tls 1.3 compliance Add unit tests around tls 1.2 server Misc updates and cleanup * pylinting * Update cipher checks and add test * Fix test results when None is returned with details * Fix duplicate results --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Test output restructure (#79) * Change runtime test structure to allow for multiple old tests * fix current test move * logging changes * Add device test count to device config * Change max report naming Add optional default value to system.json * Copy current test instead of moving to keep a consistent location of the most recent test * fix merge issue * pylint * Use local device folder and use session for config --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Keep test results in memory (#82) * Keep results in memory * More useful debug message * Fix file path * Result descriptions (#92) * Add short descriptions to conn module Add result_description to results with shorter wording for UI usage * Add result details to ipv6 tests * Update test descriptions in baseline module * Update dns test result details * Update skip results to include details when present * dns module formatting * add result details to nmap tests * add result details to ntp tests * Add short descriptions to tls module and formatting * misc test module formatting * fix typo * Misc cleanup (#93) * Fix network request from module config Misc formatting issues in test orchestrator * fix misc network orchestrator formatting issues * fix misc ovs control formatting issues * fix misc ip control formatting issues * Allow CORS (#91) * Allow CORS * Fix add device * Configurable API port * Add /history and device config endpoints (#88) * Add /history and device config endpoints * Add total tests * Add report to device * Only run tests if baseline passes * Re-enable actions, fix conn module (#89) * Re-enable actions, fix conn module * Fix net_orc init * Update report file name in testing * Add required result to module configs (#95) * Fix DNS test name (#110) * Clear runtime folder on start (#111) * Initial work on pdf report output and format via html (#103) * Add user interface (#98) * Add UI, small changes * Update UI * Update expected tests * Add some documentation * Save test modules (#115) * Expand testing to include API and more testing (#96) * Create Testrun package (#114) * Merge dev into main (Sprint 10 and 11) (#86) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test con…
jboddey
added a commit
that referenced
this pull request
May 13, 2024
* Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) * Sync dev to main (#56) * Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix missing results on udp tests when tcp ports are also defined (#59) * Add licence header (#61) * Resolve merge conflict * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS * Connection private address (#71) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * formatting * Change isc-dhcp service setup Fix dhcpd logging Add start and stop methods to grpc dhcp client Add dhcp2 client Inttial private_addr test * Add max lease time Add unit tests * fix last commit * finish initial work on test * pylinting * Breakup test and allow better failure reporting * restore network after test * Wait for device to get a lease from original dhcp range after network restore * pylinting * Fix ipv6 tests --------- Co-authored-by: Jacob Boddey <boddey@google.com> * fix windows line ending * Fix python import * move isc-dhcp service commands to their own class update logging pylinting * fix dhcp1 * Initial CI testing for tests (#72) * Fix radvd conf * Fix individual test disable * Add NTP Pass CI test (#76) * add shared address test (#75) * Fix single ip test (#58) * Fix single ip test from detecting faux-device during validation as a failure * remove dhcp server capture file from scan --------- Co-authored-by: J Boddey <boddey@google.com> * Merge API into dev (#70) * Start API * Write interfaces * Get current configuration * Set versions * Add more API methods * Correct no-ui flag * Do not launch API on baseline test * Move loading devices back to Test Run core * Merge dev into api (#74) * Merge dev into main (Add license header) (#62) Add license header * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS * Merge dev into main (Sprint 9) (#66) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) * Sync dev to main (#56) * Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix missing results on udp tests when tcp ports are also defined (#59) * Add licence header (#61) * Resolve merge conflict * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Connection private address (#71) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * formatting * Change isc-dhcp service setup Fix dhcpd logging Add start and stop methods to grpc dhcp client Add dhcp2 client Inttial private_addr test * Add max lease time Add unit tests * fix last commit * finish initial work on test * pylinting * Breakup test and allow better failure reporting * restore network after test * Wait for device to get a lease from original dhcp range after network restore * pylinting * Fix ipv6 tests --------- Co-authored-by: Jacob Boddey <boddey@google.com> * fix windows line ending * Fix python import * move isc-dhcp service commands to their own class update logging pylinting * fix dhcp1 * Initial CI testing for tests (#72) * Fix radvd conf --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix testing command * Disable API on testing * Add API session * Remove old method * Remove local vars * Replace old var * Add device config * Add device configs * Fix paths * Change MAC address * Revert mac * Fix copy path * Debug loading devices * Remove reference * Changes * Re-add checks to prevent null values * Fix variable * Fix * Use dict instead of string * Try without json conversion * Container output to log * Undo changes to nmap module * Add post devices route --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Dhcp tests (#81) * Separate dhcp control methods into their own module Implement ip change test Add place holder for dhcp failover test * Stabilize network before leaving ip_change test Add dhcp_failover test * fix regression issue with individual test enable/disable setting * fix gitignore * Merge tls tests into dev (#80) * initial add of security module and tls tests * Fix server test and implement 1.3 version * pylinting * More work on client tests * Add client tls tests Add unit tets Add common python code to base test module * re-enable dhcp unit tests disabled during dev * rename module to tls * fix renaming * Fix unit tests broken by module rename Add TLS 1.3 tests to config * Add TLS 1.3 tests to config fix unit tests * Add certificate signature checks * Add local cert mounting for signature validatoin Fix test results * Update tls 1.2 server to pass with tls 1.3 compliance Add unit tests around tls 1.2 server Misc updates and cleanup * pylinting * Update cipher checks and add test * Fix test results when None is returned with details * Fix duplicate results --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Test output restructure (#79) * Change runtime test structure to allow for multiple old tests * fix current test move * logging changes * Add device test count to device config * Change max report naming Add optional default value to system.json * Copy current test instead of moving to keep a consistent location of the most recent test * fix merge issue * pylint * Use local device folder and use session for config --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Keep test results in memory (#82) * Keep results in memory * More useful debug message * Fix file path * Result descriptions (#92) * Add short descriptions to conn module Add result_description to results with shorter wording for UI usage * Add result details to ipv6 tests * Update test descriptions in baseline module * Update dns test result details * Update skip results to include details when present * dns module formatting * add result details to nmap tests * add result details to ntp tests * Add short descriptions to tls module and formatting * misc test module formatting * fix typo * Misc cleanup (#93) * Fix network request from module config Misc formatting issues in test orchestrator * fix misc network orchestrator formatting issues * fix misc ovs control formatting issues * fix misc ip control formatting issues * Allow CORS (#91) * Allow CORS * Fix add device * Configurable API port * Add /history and device config endpoints (#88) * Add /history and device config endpoints * Add total tests * Add report to device * Only run tests if baseline passes * Re-enable actions, fix conn module (#89) * Re-enable actions, fix conn module * Fix net_orc init * Update report file name in testing * Add required result to module configs (#95) * Fix DNS test name (#110) * Clear runtime folder on start (#111) * Initial work on pdf report output and format via html (#103) * Add user interface (#98) * Add UI, small changes * Update UI * Update expected tests * Add some documentation * Save test modules (#115) * Expand testing to include API and more testing (#96) * Create Testrun package (#114) * Merge dev into main (Sprint 10 and 11) (#86) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) *…
hitnik
pushed a commit
that referenced
this pull request
Jul 22, 2024
hitnik
pushed a commit
that referenced
this pull request
Jul 22, 2024
* Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com>
hitnik
pushed a commit
that referenced
this pull request
Jul 22, 2024
* Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com>
hitnik
pushed a commit
that referenced
this pull request
Jul 22, 2024
* Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) * Sync dev to main (#56) * Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix missing results on udp tests when tcp ports are also defined (#59) * Add licence header (#61) * Resolve merge conflict * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com>
hitnik
pushed a commit
that referenced
this pull request
Jul 22, 2024
* Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) * Sync dev to main (#56) * Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix missing results on udp tests when tcp ports are also defined (#59) * Add licence header (#61) * Resolve merge conflict * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS * Connection private address (#71) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * formatting * Change isc-dhcp service setup Fix dhcpd logging Add start and stop methods to grpc dhcp client Add dhcp2 client Inttial private_addr test * Add max lease time Add unit tests * fix last commit * finish initial work on test * pylinting * Breakup test and allow better failure reporting * restore network after test * Wait for device to get a lease from original dhcp range after network restore * pylinting * Fix ipv6 tests --------- Co-authored-by: Jacob Boddey <boddey@google.com> * fix windows line ending * Fix python import * move isc-dhcp service commands to their own class update logging pylinting * fix dhcp1 * Initial CI testing for tests (#72) * Fix radvd conf --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com>
hitnik
pushed a commit
that referenced
this pull request
Jul 22, 2024
* Start API * Write interfaces * Get current configuration * Set versions * Add more API methods * Correct no-ui flag * Do not launch API on baseline test * Move loading devices back to Test Run core * Merge dev into api (#74) * Merge dev into main (Add license header) (#62) Add license header * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS * Merge dev into main (Sprint 9) (#66) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) * Sync dev to main (#56) * Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix missing results on udp tests when tcp ports are also defined (#59) * Add licence header (#61) * Resolve merge conflict * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Connection private address (#71) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * formatting * Change isc-dhcp service setup Fix dhcpd logging Add start and stop methods to grpc dhcp client Add dhcp2 client Inttial private_addr test * Add max lease time Add unit tests * fix last commit * finish initial work on test * pylinting * Breakup test and allow better failure reporting * restore network after test * Wait for device to get a lease from original dhcp range after network restore * pylinting * Fix ipv6 tests --------- Co-authored-by: Jacob Boddey <boddey@google.com> * fix windows line ending * Fix python import * move isc-dhcp service commands to their own class update logging pylinting * fix dhcp1 * Initial CI testing for tests (#72) * Fix radvd conf --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix testing command * Disable API on testing * Add API session * Remove old method * Remove local vars * Replace old var * Add device config * Add device configs * Fix paths * Change MAC address * Revert mac * Fix copy path * Debug loading devices * Remove reference * Changes * Re-add checks to prevent null values * Fix variable * Fix * Use dict instead of string * Try without json conversion * Container output to log * Undo changes to nmap module * Add post devices route --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com>
hitnik
pushed a commit
that referenced
this pull request
Jul 22, 2024
* Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) * Sync dev to main (#56) * Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix missing results on udp tests when tcp ports are also defined (#59) * Add licence header (#61) * Resolve merge conflict * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS * Connection private address (#71) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * formatting * Change isc-dhcp service setup Fix dhcpd logging Add start and stop methods to grpc dhcp client Add dhcp2 client Inttial private_addr test * Add max lease time Add unit tests * fix last commit * finish initial work on test * pylinting * Breakup test and allow better failure reporting * restore network after test * Wait for device to get a lease from original dhcp range after network restore * pylinting * Fix ipv6 tests --------- Co-authored-by: Jacob Boddey <boddey@google.com> * fix windows line ending * Fix python import * move isc-dhcp service commands to their own class update logging pylinting * fix dhcp1 * Initial CI testing for tests (#72) * Fix radvd conf * Fix individual test disable * Add NTP Pass CI test (#76) * add shared address test (#75) * Fix single ip test (#58) * Fix single ip test from detecting faux-device during validation as a failure * remove dhcp server capture file from scan --------- Co-authored-by: J Boddey <boddey@google.com> * Merge API into dev (#70) * Start API * Write interfaces * Get current configuration * Set versions * Add more API methods * Correct no-ui flag * Do not launch API on baseline test * Move loading devices back to Test Run core * Merge dev into api (#74) * Merge dev into main (Add license header) (#62) Add license header * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS * Merge dev into main (Sprint 9) (#66) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) * Sync dev to main (#56) * Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix missing results on udp tests when tcp ports are also defined (#59) * Add licence header (#61) * Resolve merge conflict * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Connection private address (#71) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * formatting * Change isc-dhcp service setup Fix dhcpd logging Add start and stop methods to grpc dhcp client Add dhcp2 client Inttial private_addr test * Add max lease time Add unit tests * fix last commit * finish initial work on test * pylinting * Breakup test and allow better failure reporting * restore network after test * Wait for device to get a lease from original dhcp range after network restore * pylinting * Fix ipv6 tests --------- Co-authored-by: Jacob Boddey <boddey@google.com> * fix windows line ending * Fix python import * move isc-dhcp service commands to their own class update logging pylinting * fix dhcp1 * Initial CI testing for tests (#72) * Fix radvd conf --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix testing command * Disable API on testing * Add API session * Remove old method * Remove local vars * Replace old var * Add device config * Add device configs * Fix paths * Change MAC address * Revert mac * Fix copy path * Debug loading devices * Remove reference * Changes * Re-add checks to prevent null values * Fix variable * Fix * Use dict instead of string * Try without json conversion * Container output to log * Undo changes to nmap module * Add post devices route --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Dhcp tests (#81) * Separate dhcp control methods into their own module Implement ip change test Add place holder for dhcp failover test * Stabilize network before leaving ip_change test Add dhcp_failover test * fix regression issue with individual test enable/disable setting * fix gitignore * Merge tls tests into dev (#80) * initial add of security module and tls tests * Fix server test and implement 1.3 version * pylinting * More work on client tests * Add client tls tests Add unit tets Add common python code to base test module * re-enable dhcp unit tests disab…
hitnik
pushed a commit
that referenced
this pull request
Jul 22, 2024
* Merge dev into main (Sprint 10 and 11) (#86) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) * Sync dev to main (#56) * Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix missing results on udp tests when tcp ports are also defined (#59) * Add licence header (#61) * Resolve merge conflict * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS * Connection private address (#71) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * formatting * Change isc-dhcp service setup Fix dhcpd logging Add start and stop methods to grpc dhcp client Add dhcp2 client Inttial private_addr test * Add max lease time Add unit tests * fix last commit * finish initial work on test * pylinting * Breakup test and allow better failure reporting * restore network after test * Wait for device to get a lease from original dhcp range after network restore * pylinting * Fix ipv6 tests --------- Co-authored-by: Jacob Boddey <boddey@google.com> * fix windows line ending * Fix python import * move isc-dhcp service commands to their own class update logging pylinting * fix dhcp1 * Initial CI testing for tests (#72) * Fix radvd conf * Fix individual test disable * Add NTP Pass CI test (#76) * add shared address test (#75) * Fix single ip test (#58) * Fix single ip test from detecting faux-device during validation as a failure * remove dhcp server capture file from scan --------- Co-authored-by: J Boddey <boddey@google.com> * Merge API into dev (#70) * Start API * Write interfaces * Get current configuration * Set versions * Add more API methods * Correct no-ui flag * Do not launch API on baseline test * Move loading devices back to Test Run core * Merge dev into api (#74) * Merge dev into main (Add license header) (#62) Add license header * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS * Merge dev into main (Sprint 9) (#66) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) * Sync dev to main (#56) * Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix missing results on udp tests when tcp ports are also defined (#59) * Add licence header (#61) * Resolve merge conflict * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Connection private address (#71) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * formatting * Change isc-dhcp service setup Fix dhcpd logging Add start and stop methods to grpc dhcp client Add dhcp2 client Inttial private_addr test * Add max lease time Add unit tests * fix last commit * finish initial work on test * pylinting * Breakup test and allow better failure reporting * restore network after test * Wait for device to get a lease from original dhcp range after network restore * pylinting * Fix ipv6 tests --------- Co-authored-by: Jacob Boddey <boddey@google.com> * fix windows line ending * Fix python import * move isc-dhcp service commands to their own class update logging pylinting * fix dhcp1 * Initial CI testing for tests (#72) * Fix radvd conf --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix testing command * Disable API on testing * Add API session * Remove old method * Remove local vars * Replace old var * Add device config * Add device configs * Fix paths * Change MAC address * Revert mac * Fix copy path * Debug loading devices * Remove reference * Changes * Re-add checks to prevent null values * Fix variable * Fix * Use dict instead of string * Try without json conversion * Container output to log * Undo changes to nmap module * Add post devices route --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Dhcp tests (#81) * Separate dhcp control methods into their own module Implement ip change test Add place holder for dhcp failover test * Stabilize network before leaving ip_change test Add dhcp_failover test * fix regression issue with individual test enable/disable setting * fix gitignore * Merge tls tests into dev (#80) * initial add of security module and tls tests * Fix server test and implement 1.3 version * pylinting * More work on client tests * Add client tls tests Add unit tets Add common python code to base test module …
hitnik
pushed a commit
that referenced
this pull request
Jul 22, 2024
* Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) * Sync dev to main (#56) * Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix missing results on udp tests when tcp ports are also defined (#59) * Add licence header (#61) * Resolve merge conflict * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS * Connection private address (#71) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * formatting * Change isc-dhcp service setup Fix dhcpd logging Add start and stop methods to grpc dhcp client Add dhcp2 client Inttial private_addr test * Add max lease time Add unit tests * fix last commit * finish initial work on test * pylinting * Breakup test and allow better failure reporting * restore network after test * Wait for device to get a lease from original dhcp range after network restore * pylinting * Fix ipv6 tests --------- Co-authored-by: Jacob Boddey <boddey@google.com> * fix windows line ending * Fix python import * move isc-dhcp service commands to their own class update logging pylinting * fix dhcp1 * Initial CI testing for tests (#72) * Fix radvd conf * Fix individual test disable * Add NTP Pass CI test (#76) * add shared address test (#75) * Fix single ip test (#58) * Fix single ip test from detecting faux-device during validation as a failure * remove dhcp server capture file from scan --------- Co-authored-by: J Boddey <boddey@google.com> * Merge API into dev (#70) * Start API * Write interfaces * Get current configuration * Set versions * Add more API methods * Correct no-ui flag * Do not launch API on baseline test * Move loading devices back to Test Run core * Merge dev into api (#74) * Merge dev into main (Add license header) (#62) Add license header * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS * Merge dev into main (Sprint 9) (#66) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) * Sync dev to main (#56) * Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix missing results on udp tests when tcp ports are also defined (#59) * Add licence header (#61) * Resolve merge conflict * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Connection private address (#71) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * formatting * Change isc-dhcp service setup Fix dhcpd logging Add start and stop methods to grpc dhcp client Add dhcp2 client Inttial private_addr test * Add max lease time Add unit tests * fix last commit * finish initial work on test * pylinting * Breakup test and allow better failure reporting * restore network after test * Wait for device to get a lease from original dhcp range after network restore * pylinting * Fix ipv6 tests --------- Co-authored-by: Jacob Boddey <boddey@google.com> * fix windows line ending * Fix python import * move isc-dhcp service commands to their own class update logging pylinting * fix dhcp1 * Initial CI testing for tests (#72) * Fix radvd conf --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix testing command * Disable API on testing * Add API session * Remove old method * Remove local vars * Replace old var * Add device config * Add device configs * Fix paths * Change MAC address * Revert mac * Fix copy path * Debug loading devices * Remove reference * Changes * Re-add checks to prevent null values * Fix variable * Fix * Use dict instead of string * Try without json conversion * Container output to log * Undo changes to nmap module * Add post devices route --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Dhcp tests (#81) * Separate dhcp control methods into their own module Implement ip change test Add place holder for dhcp failover test * Stabilize network before leaving ip_change test Add dhcp_failover test * fix regression issue with individual test enable/disable setting * fix gitignore * Merge tls tests into dev (#80) * initial add of security module and tls tests * Fix server test and implement 1.3 version * pylinting * More work on client tests * Add client tls tests Add unit tets Add common python code to base test module * re-enable dhcp unit tests disabled during dev …
hitnik
pushed a commit
that referenced
this pull request
Jul 22, 2024
* Merge dev into main (#117) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) * Sync dev to main (#56) * Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix missing results on udp tests when tcp ports are also defined (#59) * Add licence header (#61) * Resolve merge conflict * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS * Connection private address (#71) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * formatting * Change isc-dhcp service setup Fix dhcpd logging Add start and stop methods to grpc dhcp client Add dhcp2 client Inttial private_addr test * Add max lease time Add unit tests * fix last commit * finish initial work on test * pylinting * Breakup test and allow better failure reporting * restore network after test * Wait for device to get a lease from original dhcp range after network restore * pylinting * Fix ipv6 tests --------- Co-authored-by: Jacob Boddey <boddey@google.com> * fix windows line ending * Fix python import * move isc-dhcp service commands to their own class update logging pylinting * fix dhcp1 * Initial CI testing for tests (#72) * Fix radvd conf * Fix individual test disable * Add NTP Pass CI test (#76) * add shared address test (#75) * Fix single ip test (#58) * Fix single ip test from detecting faux-device during validation as a failure * remove dhcp server capture file from scan --------- Co-authored-by: J Boddey <boddey@google.com> * Merge API into dev (#70) * Start API * Write interfaces * Get current configuration * Set versions * Add more API methods * Correct no-ui flag * Do not launch API on baseline test * Move loading devices back to Test Run core * Merge dev into api (#74) * Merge dev into main (Add license header) (#62) Add license header * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS * Merge dev into main (Sprint 9) (#66) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) * Sync dev to main (#56) * Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix missing results on udp tests when tcp ports are also defined (#59) * Add licence header (#61) * Resolve merge conflict * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Connection private address (#71) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * formatting * Change isc-dhcp service setup Fix dhcpd logging Add start and stop methods to grpc dhcp client Add dhcp2 client Inttial private_addr test * Add max lease time Add unit tests * fix last commit * finish initial work on test * pylinting * Breakup test and allow better failure reporting * restore network after test * Wait for device to get a lease from original dhcp range after network restore * pylinting * Fix ipv6 tests --------- Co-authored-by: Jacob Boddey <boddey@google.com> * fix windows line ending * Fix python import * move isc-dhcp service commands to their own class update logging pylinting * fix dhcp1 * Initial CI testing for tests (#72) * Fix radvd conf --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix testing command * Disable API on testing * Add API session * Remove old method * Remove local vars * Replace old var * Add device config * Add device configs * Fix paths * Change MAC address * Revert mac * Fix copy path * Debug loading devices * Remove reference * Changes * Re-add checks to prevent null values * Fix variable * Fix * Use dict instead of string * Try without json conversion * Container output to log * Undo changes to nmap module * Add post devices route --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Dhcp tests (#81) * Separate dhcp control methods into their own module Implement ip change test Add place holder for dhcp failover test * Stabilize network before leaving ip_change test Add dhcp_failover test * fix regression issue with individual test enable/disable setting * fix gitignore * Merge tls tests into dev (#80) * initial add of security module and tls tests * Fix server test and implement 1.3 version * pylinting * More work on client tests * Add client tls tests Add unit tets Add common python code to base test module * re-enable dhcp uni…
hitnik
pushed a commit
that referenced
this pull request
Jul 22, 2024
* Merge dev into main (#117) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) * Sync dev to main (#56) * Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix missing results on udp tests when tcp ports are also defined (#59) * Add licence header (#61) * Resolve merge conflict * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS * Connection private address (#71) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * formatting * Change isc-dhcp service setup Fix dhcpd logging Add start and stop methods to grpc dhcp client Add dhcp2 client Inttial private_addr test * Add max lease time Add unit tests * fix last commit * finish initial work on test * pylinting * Breakup test and allow better failure reporting * restore network after test * Wait for device to get a lease from original dhcp range after network restore * pylinting * Fix ipv6 tests --------- Co-authored-by: Jacob Boddey <boddey@google.com> * fix windows line ending * Fix python import * move isc-dhcp service commands to their own class update logging pylinting * fix dhcp1 * Initial CI testing for tests (#72) * Fix radvd conf * Fix individual test disable * Add NTP Pass CI test (#76) * add shared address test (#75) * Fix single ip test (#58) * Fix single ip test from detecting faux-device during validation as a failure * remove dhcp server capture file from scan --------- Co-authored-by: J Boddey <boddey@google.com> * Merge API into dev (#70) * Start API * Write interfaces * Get current configuration * Set versions * Add more API methods * Correct no-ui flag * Do not launch API on baseline test * Move loading devices back to Test Run core * Merge dev into api (#74) * Merge dev into main (Add license header) (#62) Add license header * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS * Merge dev into main (Sprint 9) (#66) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) * Sync dev to main (#56) * Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix missing results on udp tests when tcp ports are also defined (#59) * Add licence header (#61) * Resolve merge conflict * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Connection private address (#71) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * formatting * Change isc-dhcp service setup Fix dhcpd logging Add start and stop methods to grpc dhcp client Add dhcp2 client Inttial private_addr test * Add max lease time Add unit tests * fix last commit * finish initial work on test * pylinting * Breakup test and allow better failure reporting * restore network after test * Wait for device to get a lease from original dhcp range after network restore * pylinting * Fix ipv6 tests --------- Co-authored-by: Jacob Boddey <boddey@google.com> * fix windows line ending * Fix python import * move isc-dhcp service commands to their own class update logging pylinting * fix dhcp1 * Initial CI testing for tests (#72) * Fix radvd conf --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix testing command * Disable API on testing * Add API session * Remove old method * Remove local vars * Replace old var * Add device config * Add device configs * Fix paths * Change MAC address * Revert mac * Fix copy path * Debug loading devices * Remove reference * Changes * Re-add checks to prevent null values * Fix variable * Fix * Use dict instead of string * Try without json conversion * Container output to log * Undo changes to nmap module * Add post devices route --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Dhcp tests (#81) * Separate dhcp control methods into their own module Implement ip change test Add place holder for dhcp failover test * Stabilize network before leaving ip_change test Add dhcp_failover test * fix regression issue with individual test enable/disable setting * fix gitignore * Merge tls tests into dev (#80) * initial add of security module and tls tests * Fix server test and implement 1.3 version * pylinting * More work on client tests * Add client tls tests Add unit tets Add common python code to base test module * re-enable …
hitnik
pushed a commit
that referenced
this pull request
Jul 22, 2024
* Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) * Sync dev to main (#56) * Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix missing results on udp tests when tcp ports are also defined (#59) * Add licence header (#61) * Resolve merge conflict * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS * Connection private address (#71) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * formatting * Change isc-dhcp service setup Fix dhcpd logging Add start and stop methods to grpc dhcp client Add dhcp2 client Inttial private_addr test * Add max lease time Add unit tests * fix last commit * finish initial work on test * pylinting * Breakup test and allow better failure reporting * restore network after test * Wait for device to get a lease from original dhcp range after network restore * pylinting * Fix ipv6 tests --------- Co-authored-by: Jacob Boddey <boddey@google.com> * fix windows line ending * Fix python import * move isc-dhcp service commands to their own class update logging pylinting * fix dhcp1 * Initial CI testing for tests (#72) * Fix radvd conf * Fix individual test disable * Add NTP Pass CI test (#76) * add shared address test (#75) * Fix single ip test (#58) * Fix single ip test from detecting faux-device during validation as a failure * remove dhcp server capture file from scan --------- Co-authored-by: J Boddey <boddey@google.com> * Merge API into dev (#70) * Start API * Write interfaces * Get current configuration * Set versions * Add more API methods * Correct no-ui flag * Do not launch API on baseline test * Move loading devices back to Test Run core * Merge dev into api (#74) * Merge dev into main (Add license header) (#62) Add license header * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS * Merge dev into main (Sprint 9) (#66) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) * Sync dev to main (#56) * Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix missing results on udp tests when tcp ports are also defined (#59) * Add licence header (#61) * Resolve merge conflict * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Connection private address (#71) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * formatting * Change isc-dhcp service setup Fix dhcpd logging Add start and stop methods to grpc dhcp client Add dhcp2 client Inttial private_addr test * Add max lease time Add unit tests * fix last commit * finish initial work on test * pylinting * Breakup test and allow better failure reporting * restore network after test * Wait for device to get a lease from original dhcp range after network restore * pylinting * Fix ipv6 tests --------- Co-authored-by: Jacob Boddey <boddey@google.com> * fix windows line ending * Fix python import * move isc-dhcp service commands to their own class update logging pylinting * fix dhcp1 * Initial CI testing for tests (#72) * Fix radvd conf --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix testing command * Disable API on testing * Add API session * Remove old method * Remove local vars * Replace old var * Add device config * Add device configs * Fix paths * Change MAC address * Revert mac * Fix copy path * Debug loading devices * Remove reference * Changes * Re-add checks to prevent null values * Fix variable * Fix * Use dict instead of string * Try without json conversion * Container output to log * Undo changes to nmap module * Add post devices route --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Dhcp tests (#81) * Separate dhcp control methods into their own module Implement ip change test Add place holder for dhcp failover test * Stabilize network before leaving ip_change test Add dhcp_failover test * fix regression issue with individual test enable/disable setting * fix gitignore * Merge tls tests into dev (#80) * initial add of security module and tls tests * Fix server test and implement 1.3 version * pylinting * More work on client tests * Add client tls tests Add unit tets Add common python code to base test module * re-enable dhcp unit tests disabled dur…
hitnik
pushed a commit
that referenced
this pull request
Jul 22, 2024
* Merge dev into main (Release 1.1-alpha) (#182) * Update README.md (#232) Signed-off-by: J Boddey <boddey@google.com> * Merge release v1.1.1 into main (#229) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) * Sync dev to main (#56) * Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix missing results on udp tests when tcp ports are also defined (#59) * Add licence header (#61) * Resolve merge conflict * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS * Connection private address (#71) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * formatting * Change isc-dhcp service setup Fix dhcpd logging Add start and stop methods to grpc dhcp client Add dhcp2 client Inttial private_addr test * Add max lease time Add unit tests * fix last commit * finish initial work on test * pylinting * Breakup test and allow better failure reporting * restore network after test * Wait for device to get a lease from original dhcp range after network restore * pylinting * Fix ipv6 tests --------- Co-authored-by: Jacob Boddey <boddey@google.com> * fix windows line ending * Fix python import * move isc-dhcp service commands to their own class update logging pylinting * fix dhcp1 * Initial CI testing for tests (#72) * Fix radvd conf * Fix individual test disable * Add NTP Pass CI test (#76) * add shared address test (#75) * Fix single ip test (#58) * Fix single ip test from detecting faux-device during validation as a failure * remove dhcp server capture file from scan --------- Co-authored-by: J Boddey <boddey@google.com> * Merge API into dev (#70) * Start API * Write interfaces * Get current configuration * Set versions * Add more API methods * Correct no-ui flag * Do not launch API on baseline test * Move loading devices back to Test Run core * Merge dev into api (#74) * Merge dev into main (Add license header) (#62) Add license header * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS * Merge dev into main (Sprint 9) (#66) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) * Sync dev to main (#56) * Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix missing results on udp tests when tcp ports are also defined (#59) * Add licence header (#61) * Resolve merge conflict * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Connection private address (#71) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * formatting * Change isc-dhcp service setup Fix dhcpd logging Add start and stop methods to grpc dhcp client Add dhcp2 client Inttial private_addr test * Add max lease time Add unit tests * fix last commit * finish initial work on test * pylinting * Breakup test and allow better failure reporting * restore network after test * Wait for device to get a lease from original dhcp range after network restore * pylinting * Fix ipv6 tests --------- Co-authored-by: Jacob Boddey <boddey@google.com> * fix windows line ending * Fix python import * move isc-dhcp service commands to their own class update logging pylinting * fix dhcp1 * Initial CI testing for tests (#72) * Fix radvd conf --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix testing command * Disable API on testing * Add API session * Remove old method * Remove local vars * Replace old var * Add device config * Add device configs * Fix paths * Change MAC address * Revert mac * Fix copy path * Debug loading devices * Remove reference * Changes * Re-add checks to prevent null values * Fix variable * Fix * Use dict instead of string * Try without json conversion * Container output to log * Undo changes to nmap module * Add post devices route --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Dhcp tests (#81) * Separate dhcp control methods into their own module Implement ip change test Add place holder for dhcp failover test * Stabilize network before leaving ip_change test Add dhcp_failover test * fix regression issue with individual test enable/disable setting * fix gitignore * Merge tls tests into dev (#80) * initial add of security module and tls tests * Fix server test and implement 1.3 version *…
hitnik
pushed a commit
that referenced
this pull request
Jul 22, 2024
* Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) * Sync dev to main (#56) * Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix missing results on udp tests when tcp ports are also defined (#59) * Add licence header (#61) * Resolve merge conflict * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS * Connection private address (#71) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * formatting * Change isc-dhcp service setup Fix dhcpd logging Add start and stop methods to grpc dhcp client Add dhcp2 client Inttial private_addr test * Add max lease time Add unit tests * fix last commit * finish initial work on test * pylinting * Breakup test and allow better failure reporting * restore network after test * Wait for device to get a lease from original dhcp range after network restore * pylinting * Fix ipv6 tests --------- Co-authored-by: Jacob Boddey <boddey@google.com> * fix windows line ending * Fix python import * move isc-dhcp service commands to their own class update logging pylinting * fix dhcp1 * Initial CI testing for tests (#72) * Fix radvd conf * Fix individual test disable * Add NTP Pass CI test (#76) * add shared address test (#75) * Fix single ip test (#58) * Fix single ip test from detecting faux-device during validation as a failure * remove dhcp server capture file from scan --------- Co-authored-by: J Boddey <boddey@google.com> * Merge API into dev (#70) * Start API * Write interfaces * Get current configuration * Set versions * Add more API methods * Correct no-ui flag * Do not launch API on baseline test * Move loading devices back to Test Run core * Merge dev into api (#74) * Merge dev into main (Add license header) (#62) Add license header * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS * Merge dev into main (Sprint 9) (#66) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) * Sync dev to main (#56) * Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix missing results on udp tests when tcp ports are also defined (#59) * Add licence header (#61) * Resolve merge conflict * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Connection private address (#71) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * formatting * Change isc-dhcp service setup Fix dhcpd logging Add start and stop methods to grpc dhcp client Add dhcp2 client Inttial private_addr test * Add max lease time Add unit tests * fix last commit * finish initial work on test * pylinting * Breakup test and allow better failure reporting * restore network after test * Wait for device to get a lease from original dhcp range after network restore * pylinting * Fix ipv6 tests --------- Co-authored-by: Jacob Boddey <boddey@google.com> * fix windows line ending * Fix python import * move isc-dhcp service commands to their own class update logging pylinting * fix dhcp1 * Initial CI testing for tests (#72) * Fix radvd conf --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix testing command * Disable API on testing * Add API session * Remove old method * Remove local vars * Replace old var * Add device config * Add device configs * Fix paths * Change MAC address * Revert mac * Fix copy path * Debug loading devices * Remove reference * Changes * Re-add checks to prevent null values * Fix variable * Fix * Use dict instead of string * Try without json conversion * Container output to log * Undo changes to nmap module * Add post devices route --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Dhcp tests (#81) * Separate dhcp control methods into their own module Implement ip change test Add place holder for dhcp failover test * Stabilize network before leaving ip_change test Add dhcp_failover test * fix regression issue with individual test enable/disable setting * fix gitignore * Merge tls tests into dev (#80) * initial add of security module and tls tests * Fix server test and implement 1.3 version * pylinting * More work on client tests * Add client tls tests Add unit tets Add common python code to base test module * re-enable dhcp unit tests disabled…
hitnik
pushed a commit
that referenced
this pull request
Jul 22, 2024
* Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) * Sync dev to main (#56) * Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix missing results on udp tests when tcp ports are also defined (#59) * Add licence header (#61) * Resolve merge conflict * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS * Connection private address (#71) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * formatting * Change isc-dhcp service setup Fix dhcpd logging Add start and stop methods to grpc dhcp client Add dhcp2 client Inttial private_addr test * Add max lease time Add unit tests * fix last commit * finish initial work on test * pylinting * Breakup test and allow better failure reporting * restore network after test * Wait for device to get a lease from original dhcp range after network restore * pylinting * Fix ipv6 tests --------- Co-authored-by: Jacob Boddey <boddey@google.com> * fix windows line ending * Fix python import * move isc-dhcp service commands to their own class update logging pylinting * fix dhcp1 * Initial CI testing for tests (#72) * Fix radvd conf * Fix individual test disable * Add NTP Pass CI test (#76) * add shared address test (#75) * Fix single ip test (#58) * Fix single ip test from detecting faux-device during validation as a failure * remove dhcp server capture file from scan --------- Co-authored-by: J Boddey <boddey@google.com> * Merge API into dev (#70) * Start API * Write interfaces * Get current configuration * Set versions * Add more API methods * Correct no-ui flag * Do not launch API on baseline test * Move loading devices back to Test Run core * Merge dev into api (#74) * Merge dev into main (Add license header) (#62) Add license header * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS * Merge dev into main (Sprint 9) (#66) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) * Sync dev to main (#56) * Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix missing results on udp tests when tcp ports are also defined (#59) * Add licence header (#61) * Resolve merge conflict * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Connection private address (#71) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * formatting * Change isc-dhcp service setup Fix dhcpd logging Add start and stop methods to grpc dhcp client Add dhcp2 client Inttial private_addr test * Add max lease time Add unit tests * fix last commit * finish initial work on test * pylinting * Breakup test and allow better failure reporting * restore network after test * Wait for device to get a lease from original dhcp range after network restore * pylinting * Fix ipv6 tests --------- Co-authored-by: Jacob Boddey <boddey@google.com> * fix windows line ending * Fix python import * move isc-dhcp service commands to their own class update logging pylinting * fix dhcp1 * Initial CI testing for tests (#72) * Fix radvd conf --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix testing command * Disable API on testing * Add API session * Remove old method * Remove local vars * Replace old var * Add device config * Add device configs * Fix paths * Change MAC address * Revert mac * Fix copy path * Debug loading devices * Remove reference * Changes * Re-add checks to prevent null values * Fix variable * Fix * Use dict instead of string * Try without json conversion * Container output to log * Undo changes to nmap module * Add post devices route --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Dhcp tests (#81) * Separate dhcp control methods into their own module Implement ip change test Add place holder for dhcp failover test * Stabilize network before leaving ip_change test Add dhcp_failover test * fix regression issue with individual test enable/disable setting * fix gitignore * Merge tls tests into dev (#80) * initial add of security module and tls tests * Fix server test and implement 1.3 version * pylinting * More work on client tests * Add client tls tests Add unit tets Add common python code to base test module * re-enable dhcp unit tests disabled during dev * rename module to tls * fix renaming * Fix unit tests broken by module rename Add TLS 1.3 tests to config * Add TLS 1.3 tests to config fix unit tests * Add certificate signature checks * Add local cert mounting for signature validatoin Fix test results * Update tls 1.2 server to pass with tls 1.3 compliance Add unit tests around tls 1.2 server Misc updates and cleanup * pylinting * Update cipher checks and add test * Fix test results when None is returned with details * Fix duplicate results --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Test output restructure (#79) * Change runtime test structure to allow for multiple old tests * fix current test move * logging changes * Add device test count to device config * Change max report naming Add optional default value to system.json * Copy current test instead of moving to keep a consistent location of the most recent test * fix merge issue * pylint * Use local device folder and use session for config --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Keep test results in memory (#82) * Keep results in memory * More useful debug message * Fix file path * Result descriptions (#92) * Add short descriptions to conn module Add result_description to results with shorter wording for UI usage * Add result details to ipv6 tests * Update test descriptions in baseline module * Update dns test result details * Update skip results to include details when present * dns module formatting * add result details to nmap tests * add result details to ntp tests * Add short descriptions to tls module and formatting * misc test module formatting * fix typo * Misc cleanup (#93) * Fix network request from module config Misc formatting issues in test orchestrator * fix misc network orchestrator formatting issues * fix misc ovs control formatting issues * fix misc ip control formatting issues * Allow CORS (#91) * Allow CORS * Fix add device * Configurable API port * Add /history and device config endpoints (#88) * Add /history and device config endpoints * Add total tests * Add report to device * Only run tests if baseline passes * Re-enable actions, fix conn module (#89) * Re-enable actions, fix conn module * Fix net_orc init * Update report file name in testing * Add required result to module configs (#95) * Fix DNS test name (#110) * Clear runtime folder on start (#111) * Initial work on pdf report output and format via html (#103) * Add user interface (#98) * Add UI, small changes * Update UI * Update expected tests * Add some documentation * Save test modules (#115) * Expand testing to include API and more testing (#96) * Create Testrun package (#114) * Merge dev into main (Sprint 10 and 11) (#86) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test con…
hitnik
pushed a commit
that referenced
this pull request
Jul 22, 2024
* Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) * Sync dev to main (#56) * Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix missing results on udp tests when tcp ports are also defined (#59) * Add licence header (#61) * Resolve merge conflict * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS * Connection private address (#71) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * formatting * Change isc-dhcp service setup Fix dhcpd logging Add start and stop methods to grpc dhcp client Add dhcp2 client Inttial private_addr test * Add max lease time Add unit tests * fix last commit * finish initial work on test * pylinting * Breakup test and allow better failure reporting * restore network after test * Wait for device to get a lease from original dhcp range after network restore * pylinting * Fix ipv6 tests --------- Co-authored-by: Jacob Boddey <boddey@google.com> * fix windows line ending * Fix python import * move isc-dhcp service commands to their own class update logging pylinting * fix dhcp1 * Initial CI testing for tests (#72) * Fix radvd conf * Fix individual test disable * Add NTP Pass CI test (#76) * add shared address test (#75) * Fix single ip test (#58) * Fix single ip test from detecting faux-device during validation as a failure * remove dhcp server capture file from scan --------- Co-authored-by: J Boddey <boddey@google.com> * Merge API into dev (#70) * Start API * Write interfaces * Get current configuration * Set versions * Add more API methods * Correct no-ui flag * Do not launch API on baseline test * Move loading devices back to Test Run core * Merge dev into api (#74) * Merge dev into main (Add license header) (#62) Add license header * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS * Merge dev into main (Sprint 9) (#66) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) * Sync dev to main (#56) * Merge dev into main (Sprint 7 and 8) (#33) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Pylint (#32) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add license header (#36) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Ovs (#35) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * remove ovs files added back in during merge * Nmap (#38) * More formatting fixes * More formatting fixes * More formatting fixes * More formatting fixes * Misc pylint fixes Fix test module logger * remove unused files * more formatting * revert breaking pylint changes * more formatting * fix results file * More formatting * ovs module formatting * Add ovs control into network orchestrator * Add verification methods for the base network * Add network validation and misc logging updates * remove ovs module * add license header to all python files * Update tcp scans to speed up full port range scan Add version checking Implement ssh version checking * Add unknown port checks Match unknown ports to existing services Add unknown ports without existing services to results file --------- Co-authored-by: Jacob Boddey <boddey@google.com> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Create startup capture (#37) * Connection (#40) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Conn mac oui (#42) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Con mac address (#43) * Initial add of connection test module with ping test * Update host user resolving * Update host user resolving for validator * add get user method to validator * Add mac_oui test Add option to return test result and details of test for reporting * Add connection.mac_address test * Dns (#44) * Add MDNS test * Update existing mdns logging to be more consistent with other tests * Add startup and monitor captures * File permissions (#45) * Fix validator file permissions * Fix test module permissions * Fix device capture file permissions * Fix device results permissions * Add connection single ip test (#47) * Nmap results (#49) * Update processing of nmap results to use xml output and json conversions for stability * Update matching with regex to prevent wrong service matches and duplicate processing for partial matches * Update max port scan range * Framework restructure (#50) * Restructure framework and modules * Fix CI paths * Fix base module * Add build script * Remove build logs * Update base and template docker files to fit the new format Implement a template option on network modules Fix skipping of base image build * remove base image build in ci * Remove group from chown --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Ip control (#51) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * Move config to /local (#52) * Move config to /local * Fix testing config * Fix ovs_control config location * Fix faux dev config location * Add documentation (#53) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Sprint 8 Hotfix (#54) * Fix connection results.json * Re add try/catch * Fix log level * Debug test module load order * Add depends on to nmap module * Remove logging change --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix missing results on udp tests when tcp ports are also defined (#59) * Add licence header (#61) * Resolve merge conflict * Add network docs (#63) * Add network docs * Rename to readme * Add link to template module * Dhcp (#64) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Dhcp (#67) * Add initial work for ip control module * Implement ip control module with additional cleanup methods * Update link check to not use error stream * Add error checking around container network configurations * Add network cleanup for namespaces and links * formatting * initial work on adding grpc functions for dhcp tests * rework code to allow for better usage and unit testing * working poc for test containers and grpc client to dhcp-1 * Move grpc client code into base image * Move grpc proto builds outside of dockerfile into module startup script * Setup pythonpath var in test module base startup process misc cleanup * pylinting and logging updates * Add python path resolving to network modules Update grpc path to prevent conflicts misc pylinting * Change lease resolving method to fix pylint issue * cleanup unit tests * cleanup unit tests * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * Add grpc updates to dhcp2 module Update dhcp_config to deal with missing optional variables * fix line endings * misc cleanup * Move isc-dhcp-server and radvd to services Move DHCP server monitoring and booting to python script * Add grpc methods to interact with dhcp_server module Update dhcp_server to control radvd server directly from calls Fix radvd service status method * Add updates to dhcp2 module Update radvd service * Add license headers * Add connection.dhcp_address test (#68) * Add NTP tests (#60) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * Pylint (#69) --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> * Add ipv6 tests (#65) * Add ipv6 tests * Check for ND_NS --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Connection private address (#71) * Add ntp support test * Add extra log message * Modify descriptions * Pylint * formatting * Change isc-dhcp service setup Fix dhcpd logging Add start and stop methods to grpc dhcp client Add dhcp2 client Inttial private_addr test * Add max lease time Add unit tests * fix last commit * finish initial work on test * pylinting * Breakup test and allow better failure reporting * restore network after test * Wait for device to get a lease from original dhcp range after network restore * pylinting * Fix ipv6 tests --------- Co-authored-by: Jacob Boddey <boddey@google.com> * fix windows line ending * Fix python import * move isc-dhcp service commands to their own class update logging pylinting * fix dhcp1 * Initial CI testing for tests (#72) * Fix radvd conf --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Fix testing command * Disable API on testing * Add API session * Remove old method * Remove local vars * Replace old var * Add device config * Add device configs * Fix paths * Change MAC address * Revert mac * Fix copy path * Debug loading devices * Remove reference * Changes * Re-add checks to prevent null values * Fix variable * Fix * Use dict instead of string * Try without json conversion * Container output to log * Undo changes to nmap module * Add post devices route --------- Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com> Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> Co-authored-by: Noureddine <noureddine@saidi.uk> Co-authored-by: SuperJonotron <jphughes4433@gmail.com> * Dhcp tests (#81) * Separate dhcp control methods into their own module Implement ip change test Add place holder for dhcp failover test * Stabilize network before leaving ip_change test Add dhcp_failover test * fix regression issue with individual test enable/disable setting * fix gitignore * Merge tls tests into dev (#80) * initial add of security module and tls tests * Fix server test and implement 1.3 version * pylinting * More work on client tests * Add client tls tests Add unit tets Add common python code to base test module * re-enable dhcp unit tests disabled during dev * rename module to tls * fix renaming * Fix unit tests broken by module rename Add TLS 1.3 tests to config * Add TLS 1.3 tests to config fix unit tests * Add certificate signature checks * Add local cert mounting for signature validatoin Fix test results * Update tls 1.2 server to pass with tls 1.3 compliance Add unit tests around tls 1.2 server Misc updates and cleanup * pylinting * Update cipher checks and add test * Fix test results when None is returned with details * Fix duplicate results --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Test output restructure (#79) * Change runtime test structure to allow for multiple old tests * fix current test move * logging changes * Add device test count to device config * Change max report naming Add optional default value to system.json * Copy current test instead of moving to keep a consistent location of the most recent test * fix merge issue * pylint * Use local device folder and use session for config --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Keep test results in memory (#82) * Keep results in memory * More useful debug message * Fix file path * Result descriptions (#92) * Add short descriptions to conn module Add result_description to results with shorter wording for UI usage * Add result details to ipv6 tests * Update test descriptions in baseline module * Update dns test result details * Update skip results to include details when present * dns module formatting * add result details to nmap tests * add result details to ntp tests * Add short descriptions to tls module and formatting * misc test module formatting * fix typo * Misc cleanup (#93) * Fix network request from module config Misc formatting issues in test orchestrator * fix misc network orchestrator formatting issues * fix misc ovs control formatting issues * fix misc ip control formatting issues * Allow CORS (#91) * Allow CORS * Fix add device * Configurable API port * Add /history and device config endpoints (#88) * Add /history and device config endpoints * Add total tests * Add report to device * Only run tests if baseline passes * Re-enable actions, fix conn module (#89) * Re-enable actions, fix conn module * Fix net_orc init * Update report file name in testing * Add required result to module configs (#95) * Fix DNS test name (#110) * Clear runtime folder on start (#111) * Initial work on pdf report output and format via html (#103) * Add user interface (#98) * Add UI, small changes * Update UI * Update expected tests * Add some documentation * Save test modules (#115) * Expand testing to include API and more testing (#96) * Create Testrun package (#114) * Merge dev into main (Sprint 10 and 11) (#86) * Implement test orchestrator (#4) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level --------- Co-authored-by: Jacob Boddey <boddey@google.com> * Add issue report templates (#7) * Add issue templates * Update README.md * Discover devices on the network (#5) * Test run sync (#8) * Initial work on test-orchestrator * Ignore runtime folder * Update runtime directory for test modules * Fix logging Add initial framework for running tests * logging and misc cleanup * logging changes * Add a stop hook after all tests complete * Refactor test_orc code * Add arg passing Add option to use locally cloned via install or remote via main project network orchestrator * Fix baseline module Fix orchestrator exiting only after timeout * Add result file to baseline test module Change result format to match closer to design doc * Refactor pylint * Skip test module if it failed to start * Refactor * Check for valid log level * Add config file arg Misc changes to network start procedure * fix merge issues * Update runner and test orch procedure Add useful runtiem args * Restructure test run startup process Misc updates to work with net orch updates * Refactor --------- * Quick refactor (#9) * Fix duplicate sleep calls * Add net orc (#11) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add the DNS test module (#12) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Refactor --------- * Add baseline and pylint tests (#25) * Discover devices on the network (#22) * Discover devices on the network * Add defaults when missing from config Implement monitor wait period from config * Add steady state monitor Remove duplicate callback registrations * Load devices into network orchestrator during testrun start --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Build dependencies first (#21) * Build dependencies first * Remove debug message * Add depend on option to test modules * Re-add single interface option * Import subprocess --------- Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com> * Port scan test module (#23) * Add network orchestrator repository * cleanup duplicate start and install scripts * Temporary fix for python dependencies * Remove duplicate python requirements * remove duplicate conf files * remove remote-net option * cleanp unecessary files * Add dns test module Fix test module build process * Add mac address of device under test to test container Update dns test to use mac address filter * Update dns module tests * Change result output * logging update * Update test module for better reusability * Load in module config to test module * logging cleanup * Update baseline module to new template Misc cleanup * Add ability to disable individual tests * remove duplicate readme * Update device directories * Remove local folder * Update device template Update test module to work with new device config file format * Change test module network config options Do not start network services for modules not configured for network * Initial nmap test module add Add device ip resolving to base module Add network mounting for test modules * Update ipv4 device resolving in test modules * Map in ip subnets and remove hard coded references * Add ftp port test * Add ability to pass config for individual tests within a module Update nmap module scan to run tests based on config * Add full module check for compliance * Add all tcp port scans to config * Update nmap commands to match existing DAQ tests Add udp scanning and tests * logging cleanup * Update TCP port scanning range Update logging * Merge device config into module config Update device template * fix merge issues * Update timeouts Add multi-threading for multiple scanns to run simultaneously Add option to use scan scripts for services * Fix merge issues * Fix device configs * Remove unecessary files * Cleanup duplicate properties * Cleanup install script * Formatting (#26) * Fix pylint issues in net orc * more pylint fixes * fix listener lint issues * fix logger lint issues * fix validator lint issues * fix util lint issues * Update base network module linting issues * Cleanup linter issues for dhcp modules Remove old code testing code * change to single quote delimeter * Cleanup linter issues for ntp module * Cleanup linter issues for radius module * Cleanup linter issues for template module * fix linter issues with faux-dev * Test results (#27) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Test results (#28) * Collect all module test results * Fix test modules without config options * Add timestamp to test results * Add attempt timing and device info to test results * Ignore disabled test containers when generating results * Fully skip modules that are disabled * Fix pylint test and skip internet tests so CI passes (#29) * disable internet checks for pass * fix pylint test * Increase pylint score (#31) *…
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.