-
Notifications
You must be signed in to change notification settings - Fork 2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
riotnode: node abstraction package #10949
Changes from all commits
602aef1
475e389
557994a
2150c67
2a52293
80fe6db
824aeff
8c1d915
267860d
0485a49
1a048ba
52f8712
4defc19
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,2 @@ | ||
[run] | ||
omit = riotnode/tests/* |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,112 @@ | ||
# Manually added: | ||
# All xml reports | ||
*.xml | ||
|
||
#### joe made this: http://goel.io/joe | ||
|
||
#####=== Python ===##### | ||
|
||
# Byte-compiled / optimized / DLL files | ||
__pycache__/ | ||
*.py[cod] | ||
*$py.class | ||
|
||
# C extensions | ||
*.so | ||
|
||
# Distribution / packaging | ||
.Python | ||
build/ | ||
develop-eggs/ | ||
dist/ | ||
downloads/ | ||
eggs/ | ||
.eggs/ | ||
lib/ | ||
lib64/ | ||
parts/ | ||
sdist/ | ||
var/ | ||
wheels/ | ||
*.egg-info/ | ||
.installed.cfg | ||
*.egg | ||
MANIFEST | ||
|
||
# PyInstaller | ||
# Usually these files are written by a python script from a template | ||
# before PyInstaller builds the exe, so as to inject date/other infos into it. | ||
*.manifest | ||
*.spec | ||
|
||
# Installer logs | ||
pip-log.txt | ||
pip-delete-this-directory.txt | ||
|
||
# Unit test / coverage reports | ||
htmlcov/ | ||
.tox/ | ||
.coverage | ||
.coverage.* | ||
.cache | ||
nosetests.xml | ||
coverage.xml | ||
*.cover | ||
.hypothesis/ | ||
.pytest_cache/ | ||
|
||
# Translations | ||
*.mo | ||
*.pot | ||
|
||
# Django stuff: | ||
*.log | ||
local_settings.py | ||
db.sqlite3 | ||
|
||
# Flask stuff: | ||
instance/ | ||
.webassets-cache | ||
|
||
# Scrapy stuff: | ||
.scrapy | ||
|
||
# Sphinx documentation | ||
docs/_build/ | ||
|
||
# PyBuilder | ||
target/ | ||
|
||
# Jupyter Notebook | ||
.ipynb_checkpoints | ||
|
||
# pyenv | ||
.python-version | ||
|
||
# celery beat schedule file | ||
celerybeat-schedule | ||
|
||
# SageMath parsed files | ||
*.sage.py | ||
|
||
# Environments | ||
.env | ||
.venv | ||
env/ | ||
venv/ | ||
ENV/ | ||
env.bak/ | ||
venv.bak/ | ||
|
||
# Spyder project settings | ||
.spyderproject | ||
.spyproject | ||
|
||
# Rope project settings | ||
.ropeproject | ||
|
||
# mkdocs documentation | ||
/site | ||
|
||
# mypy | ||
.mypy_cache/ |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,28 @@ | ||
RIOT Node abstraction | ||
===================== | ||
|
||
This provides python object abstraction of a node. | ||
The first goal is to be the starting point for the serial abstraction and | ||
build on top of that to provide higher level abstraction like over the shell. | ||
|
||
It could provide an RPC interface to a node in Python over the serial port | ||
and maybe also over network. | ||
|
||
The goal is here to be test environment agnostic and be usable in any test | ||
framework and also without it. | ||
|
||
|
||
Testing | ||
------- | ||
|
||
Run `tox` to run the whole test suite: | ||
|
||
:: | ||
|
||
tox | ||
... | ||
________________________________ summary ________________________________ | ||
test: commands succeeded | ||
lint: commands succeeded | ||
flake8: commands succeeded | ||
congratulations :) |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,24 @@ | ||
TODO list | ||
========= | ||
|
||
Some list of things I would like to do but not for first publication. | ||
|
||
|
||
Legacy handling | ||
--------------- | ||
|
||
Some handling was directly taken from ``testrunner``, without a justified/tested | ||
reason. I just used it to not break existing setup for nothing. | ||
I have more details in the code. | ||
|
||
* Ignoring reset return value and error message | ||
* Use killpg(SIGKILL) to kill terminal | ||
|
||
|
||
Testing | ||
------- | ||
|
||
The current 'node' implementation is an ideal node where all output is captured | ||
and reset directly resets. Having wilder implementations with output loss (maybe | ||
as a deamon with a ``flash`` pre-requisite and sometime no ``reset`` would be | ||
interesting. |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,2 @@ | ||
# Use the current setup.py for requirements | ||
. |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
"""RIOT Node abstraction. | ||
|
||
This prodives python object abstraction of a node. | ||
The first goal is to be the starting point for the serial abstraction and | ||
build on top of that to provide higher level abstraction like over the shell. | ||
|
||
It could provide an RPC interface to a node in Python over the serial port | ||
and maybe also over network. | ||
""" | ||
|
||
__version__ = '0.1.0' |
Original file line number | Diff line number | Diff line change | ||||||||
---|---|---|---|---|---|---|---|---|---|---|
@@ -0,0 +1,205 @@ | ||||||||||
"""RIOTNode abstraction. | ||||||||||
|
||||||||||
Define class to abstract a node over the RIOT build system. | ||||||||||
""" | ||||||||||
|
||||||||||
import os | ||||||||||
import time | ||||||||||
import logging | ||||||||||
import subprocess | ||||||||||
import contextlib | ||||||||||
|
||||||||||
import pexpect | ||||||||||
|
||||||||||
from . import utils | ||||||||||
|
||||||||||
DEVNULL = open(os.devnull, 'w') | ||||||||||
|
||||||||||
|
||||||||||
class TermSpawn(pexpect.spawn): | ||||||||||
"""Subclass to adapt the behaviour to our need. | ||||||||||
|
||||||||||
* change default `__init__` values | ||||||||||
* disable local 'echo' to not match send messages | ||||||||||
* 'utf-8/replace' by default | ||||||||||
* default timeout | ||||||||||
* tweak exception: | ||||||||||
* replace the value with the called pattern | ||||||||||
* remove exception context from inside pexpect implementation | ||||||||||
""" | ||||||||||
|
||||||||||
def __init__(self, # pylint:disable=too-many-arguments | ||||||||||
command, timeout=10, echo=False, | ||||||||||
encoding='utf-8', codec_errors='replace', **kwargs): | ||||||||||
super().__init__(command, timeout=timeout, echo=echo, | ||||||||||
encoding=encoding, codec_errors=codec_errors, | ||||||||||
**kwargs) | ||||||||||
|
||||||||||
def expect(self, pattern, *args, **kwargs): | ||||||||||
# pylint:disable=arguments-differ | ||||||||||
try: | ||||||||||
return super().expect(pattern, *args, **kwargs) | ||||||||||
except (pexpect.TIMEOUT, pexpect.EOF) as exc: | ||||||||||
raise self._pexpect_exception(exc, pattern) | ||||||||||
|
||||||||||
def expect_exact(self, pattern, *args, **kwargs): | ||||||||||
# pylint:disable=arguments-differ | ||||||||||
try: | ||||||||||
return super().expect_exact(pattern, *args, **kwargs) | ||||||||||
except (pexpect.TIMEOUT, pexpect.EOF) as exc: | ||||||||||
raise self._pexpect_exception(exc, pattern) | ||||||||||
|
||||||||||
@staticmethod | ||||||||||
def _pexpect_exception(exc, pattern): | ||||||||||
"""Tweak pexpect exception. | ||||||||||
|
||||||||||
* Put the calling 'pattern' as value | ||||||||||
* Remove exception context | ||||||||||
""" | ||||||||||
exc.pexpect_value = exc.value | ||||||||||
exc.value = pattern | ||||||||||
|
||||||||||
# Remove exception context | ||||||||||
exc.__cause__ = None | ||||||||||
exc.__traceback__ = None | ||||||||||
return exc | ||||||||||
|
||||||||||
|
||||||||||
class RIOTNode(): | ||||||||||
"""Class abstracting a RIOTNode in an application. | ||||||||||
|
||||||||||
This should abstract the build system integration. | ||||||||||
|
||||||||||
:param application_directory: relative directory to the application. | ||||||||||
:param env: dictionary of environment variables that should be used. | ||||||||||
These overwrites values coming from `os.environ` and can help | ||||||||||
define factories where environment comes from a file or if the | ||||||||||
script is not executed from the build system context. | ||||||||||
|
||||||||||
Environment variable configuration | ||||||||||
|
||||||||||
:environment BOARD: current RIOT board type. | ||||||||||
:environment RIOT_TERM_START_DELAY: delay before `make term` is said to be | ||||||||||
ready after calling. | ||||||||||
""" | ||||||||||
|
||||||||||
TERM_SPAWN_CLASS = TermSpawn | ||||||||||
TERM_STARTED_DELAY = int(os.environ.get('RIOT_TERM_START_DELAY') or 3) | ||||||||||
|
||||||||||
MAKE_ARGS = () | ||||||||||
RESET_TARGETS = ('reset',) | ||||||||||
|
||||||||||
def __init__(self, application_directory='.', env=None): | ||||||||||
self._application_directory = application_directory | ||||||||||
|
||||||||||
# TODO I am not satisfied by this, but would require changing all the | ||||||||||
# environment handling, just put a note until I can fix it. | ||||||||||
# I still want to show a PR before this | ||||||||||
# I would prefer getting either no environment == os.environ or the | ||||||||||
# full environment to use. | ||||||||||
# It should also change the `TERM_STARTED_DELAY` thing. | ||||||||||
self.env = os.environ.copy() | ||||||||||
self.env.update(env or {}) | ||||||||||
|
||||||||||
self.term = None # type: pexpect.spawn | ||||||||||
|
||||||||||
self.logger = logging.getLogger(__name__) | ||||||||||
|
||||||||||
@property | ||||||||||
def application_directory(self): | ||||||||||
"""Absolute path to the current directory.""" | ||||||||||
return os.path.abspath(self._application_directory) | ||||||||||
|
||||||||||
def board(self): | ||||||||||
"""Return board type.""" | ||||||||||
return self.env['BOARD'] | ||||||||||
|
||||||||||
def reset(self): | ||||||||||
"""Reset current node.""" | ||||||||||
# Ignoring 'reset' return value was taken from `testrunner`. | ||||||||||
# For me it should not be done for all boards as it should be an error. | ||||||||||
# I would rather fix it in the build system or have a per board | ||||||||||
# configuration. | ||||||||||
|
||||||||||
# Make reset yields error on some boards even if successful | ||||||||||
# Ignore printed errors and returncode | ||||||||||
self.make_run(self.RESET_TARGETS, stdout=DEVNULL, stderr=DEVNULL) | ||||||||||
|
||||||||||
@contextlib.contextmanager | ||||||||||
def run_term(self, reset=True, **startkwargs): | ||||||||||
"""Terminal context manager.""" | ||||||||||
try: | ||||||||||
self.start_term(**startkwargs) | ||||||||||
if reset: | ||||||||||
self.reset() | ||||||||||
yield self.term | ||||||||||
finally: | ||||||||||
self.stop_term() | ||||||||||
|
||||||||||
def start_term(self, **spawnkwargs): | ||||||||||
"""Start the terminal. | ||||||||||
|
||||||||||
The function is blocking until it is ready. | ||||||||||
It waits some time until the terminal is ready and resets the node. | ||||||||||
""" | ||||||||||
self.stop_term() | ||||||||||
|
||||||||||
term_cmd = self.make_command(['term']) | ||||||||||
self.term = self.TERM_SPAWN_CLASS(term_cmd[0], args=term_cmd[1:], | ||||||||||
env=self.env, **spawnkwargs) | ||||||||||
|
||||||||||
# on many platforms, the termprog needs a short while to be ready | ||||||||||
time.sleep(self.TERM_STARTED_DELAY) | ||||||||||
|
||||||||||
def _term_pid(self): | ||||||||||
"""Terminal pid or None.""" | ||||||||||
return getattr(self.term, 'pid', None) | ||||||||||
|
||||||||||
def stop_term(self): | ||||||||||
"""Stop the terminal.""" | ||||||||||
with utils.ensure_all_subprocesses_stopped(self._term_pid(), | ||||||||||
self.logger): | ||||||||||
self._safe_term_close() | ||||||||||
|
||||||||||
def _safe_term_close(self): | ||||||||||
"""Safe 'term.close'. | ||||||||||
|
||||||||||
Handles possible exceptions. | ||||||||||
""" | ||||||||||
try: | ||||||||||
self.term.close() | ||||||||||
except AttributeError: | ||||||||||
# Not initialized | ||||||||||
pass | ||||||||||
except ProcessLookupError: | ||||||||||
self.logger.warning('Process already stopped') | ||||||||||
except pexpect.ExceptionPexpect: | ||||||||||
# Not sure how to cover this in a test | ||||||||||
# 'make term' is not killed by 'term.close()' | ||||||||||
self.logger.critical('Could not close make term') | ||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. How about
Suggested change
? Thes way one can check (e.g. in a subclass) if the terminal is running or not. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I may even maybe put a However I did not really see a use case for calling |
||||||||||
|
||||||||||
def make_run(self, targets, *runargs, **runkwargs): | ||||||||||
"""Call make `targets` for current RIOTNode context. | ||||||||||
|
||||||||||
It is using `subprocess.run` internally. | ||||||||||
|
||||||||||
:param targets: make targets | ||||||||||
:param *runargs: args passed to subprocess.run | ||||||||||
:param *runkwargs: kwargs passed to subprocess.run | ||||||||||
:return: subprocess.CompletedProcess object | ||||||||||
""" | ||||||||||
command = self.make_command(targets) | ||||||||||
return subprocess.run(command, env=self.env, *runargs, **runkwargs) | ||||||||||
|
||||||||||
def make_command(self, targets): | ||||||||||
"""Make command for current RIOTNode context. | ||||||||||
|
||||||||||
:return: list of command arguments (for example for subprocess) | ||||||||||
""" | ||||||||||
command = ['make'] | ||||||||||
command.extend(self.MAKE_ARGS) | ||||||||||
if self._application_directory != '.': | ||||||||||
dir_cmd = '--no-print-directory', '-C', self.application_directory | ||||||||||
command.extend(dir_cmd) | ||||||||||
command.extend(targets) | ||||||||||
return command |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
"""riotnode.tests directory.""" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
so it doesn't return a key error, or is this on purpose?