Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add frontend module in launch, launch_xml and launch_yaml packages #226

Merged
merged 75 commits into from
Jul 11, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
75 commits
Select commit Hold shift + click to select a range
1090efa
Added launch_fronted abstract entity
ivanpauno Apr 15, 2019
5a894f2
Add launch_xml entity implementation
ivanpauno Apr 15, 2019
3958430
Add parse_executable function and test
ivanpauno Apr 15, 2019
f23772d
Corrected with PR comments
ivanpauno Apr 22, 2019
df8c439
Corrected with PR comments
ivanpauno Apr 25, 2019
471e406
Add expose_action and expose_substitution functions
ivanpauno Apr 16, 2019
9807d4d
Add parser
ivanpauno Apr 22, 2019
60e28d0
Add interpolate substitution function
ivanpauno Apr 22, 2019
9e8fac7
Updated xml 'executable' action test. Corrected parser module
ivanpauno Apr 22, 2019
43d87c7
Corrected error checking in decorators
ivanpauno Apr 22, 2019
7a03121
Correct launch_frontend.Entity
ivanpauno Apr 23, 2019
e2c50dd
Load entry_points which add parsing methods
ivanpauno Apr 23, 2019
68cba5a
Modified executable example
ivanpauno Apr 23, 2019
4450dd5
Used parse_substitution in parse_executable
ivanpauno Apr 23, 2019
cc0af6e
Changed entity 'frontend' staticmethod to a property
ivanpauno Apr 25, 2019
3ddbe05
Deleted list substitution
ivanpauno Apr 25, 2019
4a347fe
Updated cleaner executable args handling using list
ivanpauno Apr 25, 2019
3dd5c31
Using a grammar for parsing substitutions. Moved to frontend specific…
ivanpauno May 9, 2019
2e281f4
Corrected type annotations in substitution_parse_methods
ivanpauno May 10, 2019
9d4bca7
* Updated entity abstraction to allow get typed values.
ivanpauno May 10, 2019
bf2ad68
Added yaml launch frontend implementation
ivanpauno May 10, 2019
e5a0645
* Replaced map usage with list comprehensions.
ivanpauno May 10, 2019
43c77f5
Corrected xml test_list.py
ivanpauno May 10, 2019
65871e9
Updated launch xml readme
ivanpauno May 10, 2019
bc2f03c
Added readme to launch_yaml
ivanpauno May 13, 2019
7089121
Add test for let action and var substitution
ivanpauno May 13, 2019
3e1f9e2
Correct bugs in type_utils. Updated related doc.
ivanpauno May 14, 2019
0ba7e9d
Address PR comments
ivanpauno May 17, 2019
f3e08a4
Corrected substitution grammar. Completed substitution test
ivanpauno May 17, 2019
c0efe06
Add new test substitution case
ivanpauno May 17, 2019
6a30270
Update setup.py versions
ivanpauno May 17, 2019
ea58a13
Allow code reusage of base class parsing method
ivanpauno May 17, 2019
51d080e
Clearer launch_yaml.Entity constructor
ivanpauno May 17, 2019
3f13144
typo
ivanpauno May 17, 2019
595fa9c
Moved example tests in place
ivanpauno May 20, 2019
244dbbe
Corrected error in type_utils
ivanpauno May 20, 2019
a711306
Correct error with brute force load
ivanpauno May 20, 2019
f4d7ecc
Correct indentation
ivanpauno May 20, 2019
889dbdb
Add option for reading value as an string in yaml format
ivanpauno Jun 3, 2019
7d6076b
Revert "Add option for reading value as an string in yaml format"
ivanpauno Jun 3, 2019
1f6f911
Minimal correction in grammar
ivanpauno Jun 3, 2019
b1e30cd
Addressed reviewer comments
ivanpauno Jun 11, 2019
5cb565c
Address review comments
ivanpauno Jun 13, 2019
e7d190f
Add parsing function for include and group actions. Added related tests
ivanpauno Jun 18, 2019
8764df0
Address per review comments
ivanpauno Jun 25, 2019
be1e683
Address review comments
ivanpauno Jun 28, 2019
68dc487
Merged launch_frontend into launch package
ivanpauno Jun 28, 2019
740c204
Update type_utils
ivanpauno Jul 2, 2019
f0d0749
Corrected type utils again after further testing
ivanpauno Jul 2, 2019
d9a2845
Delete launch_frontend dependency
ivanpauno Jul 3, 2019
75268fc
Update readmes
ivanpauno Jul 3, 2019
f8a4b06
Test include tag properly
ivanpauno Jul 3, 2019
a64faaa
Address PR comments
ivanpauno Jul 3, 2019
fc8b2c8
Address remaining comments
ivanpauno Jul 3, 2019
18cf4a0
Restrict conversions in type_utils
ivanpauno Jul 3, 2019
ecf9a04
Correct ExecuteProcess parsing method
ivanpauno Jul 4, 2019
4478603
Allow quoted strings in substitution grammar
ivanpauno Jul 4, 2019
6e52e94
Avoid using yaml convertions in type_utils
ivanpauno Jul 4, 2019
5dd0ac0
Renamed launch_frontend folder in test to frontend. Updated test_subs…
ivanpauno Jul 4, 2019
85b39c9
Clearer grammar
ivanpauno Jul 5, 2019
2e2f5ae
Update type utils for allowing non uniform lists
ivanpauno Jul 5, 2019
afee3e6
Add a way of escaping the characters as the string passed by the subs…
ivanpauno Jul 5, 2019
ec3ce5c
Correct execute_process method for correctly escape characters
ivanpauno Jul 5, 2019
4b3ecf2
Update type_utils in favour of heavy usage of typing objects
ivanpauno Jul 8, 2019
38b8c0c
Correct error in coerce_to_str method
ivanpauno Jul 8, 2019
efaf9c3
Install grammar.lark correctly
ivanpauno Jul 8, 2019
3822fcd
Solved strange problem when installing grammar.lark when using --syml…
ivanpauno Jul 8, 2019
1b73e5d
Please flake8
ivanpauno Jul 9, 2019
f15b0f2
Avoid using issubclass with typing objects
ivanpauno Jul 9, 2019
b5f6a95
Add missing dependency with lark parcer
ivanpauno Jul 9, 2019
3c81e11
Use posix style paths in launch files
ivanpauno Jul 9, 2019
ba9ab09
Correct type_utils on windows
ivanpauno Jul 9, 2019
7b07333
Further corrections of type_utils on Windows
ivanpauno Jul 9, 2019
091ae9c
Solve path problem in test_include
ivanpauno Jul 9, 2019
deada40
Address review comments
ivanpauno Jul 10, 2019
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions launch/launch/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,9 @@
from . import actions
from . import conditions
from . import events
from . import frontend
from . import logging
from . import substitutions
from .action import Action
from .condition import Condition
from .event import Event
Expand All @@ -38,7 +40,9 @@
'actions',
'conditions',
'events',
'frontend',
'logging',
'substitutions',
'Action',
'Condition',
'Event',
Expand Down
30 changes: 30 additions & 0 deletions launch/launch/action.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,10 @@
from .launch_context import LaunchContext
from .launch_description_entity import LaunchDescriptionEntity

if False:
from .frontend import Entity
from .frontend import Parser


class Action(LaunchDescriptionEntity):
"""
Expand All @@ -44,6 +48,32 @@ def __init__(self, *, condition: Optional[Condition] = None) -> None:
"""
self.__condition = condition

@staticmethod
def parse(entity: 'Entity', parser: 'Parser'):
"""
Return the `Action` action and kwargs for constructing it.

This is only intended for code reuse.
This class is not exposed with `expose_action`.
"""
# Import here for avoiding cyclic imports.
from .conditions import IfCondition
from .conditions import UnlessCondition
if_cond = entity.get_attr('if', optional=True)
unless_cond = entity.get_attr('unless', optional=True)
kwargs = {}
if if_cond is not None and unless_cond is not None:
raise RuntimeError("if and unless conditions can't be used simultaneously")
if if_cond is not None:
kwargs['condition'] = IfCondition(
predicate_expression=parser.parse_substitution(if_cond)
)
if unless_cond is not None:
kwargs['condition'] = UnlessCondition(
predicate_expression=parser.parse_substitution(unless_cond)
)
return Action, kwargs

@property
def condition(self) -> Optional[Condition]:
"""Getter for condition."""
Expand Down
69 changes: 69 additions & 0 deletions launch/launch/actions/execute_process.py
Original file line number Diff line number Diff line change
Expand Up @@ -58,13 +58,17 @@
from ..events.process import ProcessStdout
from ..events.process import ShutdownProcess
from ..events.process import SignalProcess
from ..frontend import Entity
from ..frontend import expose_action
from ..frontend import Parser
from ..launch_context import LaunchContext
from ..launch_description import LaunchDescription
from ..some_actions_type import SomeActionsType
from ..some_substitutions_type import SomeSubstitutionsType
from ..substitution import Substitution # noqa: F401
from ..substitutions import LaunchConfiguration
from ..substitutions import PythonExpression
from ..substitutions import TextSubstitution
from ..utilities import create_future
from ..utilities import is_a_subclass
from ..utilities import normalize_to_list_of_substitutions
Expand All @@ -74,6 +78,7 @@
_global_process_counter = 0 # in Python3, this number is unbounded (no rollover)


@expose_action('executable')
class ExecuteProcess(Action):
"""Action that begins executing a process and sets up event handlers for the process."""

Expand Down Expand Up @@ -225,6 +230,65 @@ def __init__(
self.__stdout_buffer = io.StringIO()
self.__stderr_buffer = io.StringIO()

@classmethod
def parse(
cls,
entity: Entity,
parser: Parser,
cmd_arg_name: str = 'cmd'
):
"""
Return the `ExecuteProcess` action and kwargs for constructing it.

:param: cmd_arg_name Allow changing the name of `cmd` tag.
Intended for code reuse in derived classes (e.g.: launch_ros.actions.Node).
"""
_, kwargs = super().parse(entity, parser)

cmd = entity.get_attr(cmd_arg_name)
# `cmd` is supposed to be a list separated with ' '.
# All the found `TextSubstitution` items are split and
# added to the list again as a `TextSubstitution`.
cmd = parser.parse_substitution(cmd)
cmd_list = []
for arg in cmd:
if isinstance(arg, TextSubstitution):
text = arg.text
text = shlex.split(text)
text = [TextSubstitution(text=item) for item in text]
cmd_list.extend(text)
else:
cmd_list.append(arg)
kwargs[cmd_arg_name] = cmd_list

cwd = entity.get_attr('cwd', optional=True)
if cwd is not None:
kwargs['cwd'] = parser.parse_substitution(cwd)
name = entity.get_attr('name', optional=True)
if name is not None:
kwargs['name'] = parser.parse_substitution(name)
prefix = entity.get_attr('launch-prefix', optional=True)
if prefix is not None:
kwargs['prefix'] = parser.parse_substitution(prefix)
output = entity.get_attr('output', optional=True)
if output is not None:
kwargs['output'] = parser.escape_characters(output)
shell = entity.get_attr('shell', data_type=bool, optional=True)
if shell is not None:
kwargs['shell'] = shell
# Conditions won't be allowed in the `env` tag.
# If that feature is needed, `set_enviroment_variable` and
# `unset_enviroment_variable` actions should be used.
env = entity.get_attr('env', data_type=List[Entity], optional=True)
if env is not None:
env = {
tuple(parser.parse_substitution(e.get_attr('name'))):
parser.parse_substitution(e.get_attr('value')) for e in env
}
kwargs['additional_env'] = env

return cls, kwargs

@property
def output(self):
"""Getter for output."""
Expand Down Expand Up @@ -637,3 +701,8 @@ def additional_env(self):
def shell(self):
"""Getter for shell."""
return self.__shell

@property
def prefix(self):
"""Getter for prefix."""
return self.__prefix
14 changes: 14 additions & 0 deletions launch/launch/actions/group_action.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,14 @@
from .push_launch_configurations import PushLaunchConfigurations
from .set_launch_configuration import SetLaunchConfiguration
from ..action import Action
from ..frontend import Entity
from ..frontend import expose_action
from ..frontend import Parser
from ..launch_context import LaunchContext
from ..some_substitutions_type import SomeSubstitutionsType


@expose_action('group')
class GroupAction(Action):
"""
Action that yields other actions, optionally scoping launch configurations.
Expand Down Expand Up @@ -54,6 +58,16 @@ def __init__(
else:
self.__launch_configurations = {}

@classmethod
def parse(cls, entity: Entity, parser: Parser):
"""Return `GroupAction` action and kwargs for constructing it."""
_, kwargs = super().parse(entity, parser)
scoped = entity.get_attr('scoped', data_type=bool, optional=True)
if scoped is not None:
kwargs['scoped'] = scoped
kwargs['actions'] = [parser.parse_action(e) for e in entity.children]
return cls, kwargs

def execute(self, context: LaunchContext) -> Optional[List[Action]]:
"""Execute the action."""
actions = [] # type: List[Action]
Expand Down
22 changes: 22 additions & 0 deletions launch/launch/actions/include_launch_description.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,14 +22,19 @@

from .set_launch_configuration import SetLaunchConfiguration
from ..action import Action
from ..frontend import Entity
from ..frontend import expose_action
from ..frontend import Parser
from ..launch_context import LaunchContext
from ..launch_description_entity import LaunchDescriptionEntity
from ..launch_description_source import LaunchDescriptionSource
from ..launch_description_sources import AnyLaunchDescriptionSource
from ..some_substitutions_type import SomeSubstitutionsType
from ..utilities import normalize_to_list_of_substitutions
from ..utilities import perform_substitutions


@expose_action('include')
class IncludeLaunchDescription(Action):
"""
Action that includes a launch description source and yields its entities when visited.
Expand Down Expand Up @@ -69,6 +74,23 @@ def __init__(
self.__launch_description_source = launch_description_source
self.__launch_arguments = launch_arguments

@classmethod
def parse(cls, entity: Entity, parser: Parser):
"""Return `IncludeLaunchDescription` action and kwargs for constructing it."""
_, kwargs = super().parse(entity, parser)
file_path = parser.parse_substitution(entity.get_attr('file'))
kwargs['launch_description_source'] = AnyLaunchDescriptionSource(file_path)
args = entity.get_attr('arg', data_type=List[Entity], optional=True)
if args is not None:
kwargs['launch_arguments'] = [
(
parser.parse_substitution(e.get_attr('name')),
parser.parse_substitution(e.get_attr('value'))
)
for e in args
]
return cls, kwargs

@property
def launch_description_source(self) -> LaunchDescriptionSource:
"""Getter for self.__launch_description_source."""
Expand Down
14 changes: 14 additions & 0 deletions launch/launch/actions/set_launch_configuration.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,13 +17,17 @@
from typing import List

from ..action import Action
from ..frontend import Entity
from ..frontend import expose_action
from ..frontend import Parser
from ..launch_context import LaunchContext
from ..some_substitutions_type import SomeSubstitutionsType
from ..substitution import Substitution
from ..utilities import normalize_to_list_of_substitutions
from ..utilities import perform_substitutions


@expose_action('let')
class SetLaunchConfiguration(Action):
"""
Action that sets a launch configuration by name.
Expand All @@ -44,6 +48,16 @@ def __init__(
self.__name = normalize_to_list_of_substitutions(name)
self.__value = normalize_to_list_of_substitutions(value)

@classmethod
def parse(cls, entity: Entity, parser: Parser):
"""Return `SetLaunchConfiguration` action and kwargs for constructing it."""
name = parser.parse_substitution(entity.get_attr('name'))
value = parser.parse_substitution(entity.get_attr('value'))
_, kwargs = super().parse(entity, parser)
kwargs['name'] = name
kwargs['value'] = value
return cls, kwargs

@property
def name(self) -> List[Substitution]:
"""Getter for self.__name."""
Expand Down
29 changes: 29 additions & 0 deletions launch/launch/frontend/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# Copyright 2019 Open Source Robotics Foundation, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

"""Package for launch markup."""

from . import type_utils
from .entity import Entity
from .expose import expose_action, expose_substitution
from .parser import Parser


__all__ = [
'Entity',
'Parser',
'expose_action',
'expose_substitution',
'type_utils',
]
88 changes: 88 additions & 0 deletions launch/launch/frontend/entity.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
# Copyright 2019 Open Source Robotics Foundation, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

"""Module for Entity class."""

from typing import Any
from typing import List
from typing import Optional
from typing import Text
from typing import Union


class Entity:
"""Single item in the intermediate front_end representation."""

@property
def type_name(self) -> Text:
"""Get Entity type."""
raise NotImplementedError()

@property
def parent(self) -> Optional['Entity']:
"""Get Entity parent."""
raise NotImplementedError()

@property
def children(self) -> List['Entity']:
"""Get the Entity's children."""
raise NotImplementedError()

def get_attr(
self,
name: Text,
*,
data_type: Any = str,
optional: bool = False
) -> Optional[Union[
List[Union[int, str, float, bool]],
Union[int, str, float, bool],
List['Entity']
]]:
"""
Access an attribute of the entity.

By default, it will try to return it as an string.
`types` states the expected types of the attribute. Type coercion or type checking is
applied depending on the particular frontend.

The allowed types are:
- a scalar type i.e. `str`, `int`, `float`, `bool`;
- a uniform list i.e `List[str]`, `List[int]`, `List[float]`, `List[bool]`;
- a non-uniform list of known scalar types e.g. `List[Union[int, str]]`;
- a non-uniform list of any scalar type i.e. `list` or `List`;
- a `Union` of any of the above;
- `List[Entity]`, see below.

`types = None` works in the same way as:
`Union[int, float, bool, list, str]`

`List[Entity]` allows accessing a list of subentities with an specific name.
Check the documentation of each specific frontend implementation to see how `list`
and `List[Entity]` look like.

If `optional` is `True` and the attribute cannot be found, `None` will be returned
instead of raising `AttributeError`.

:param name: name of the attribute
:param types: type of the attribute to be read. Default to 'str'
:param optional: when `True`, it doesn't raise an error when the attribute is not found.
It returns `None` instead. Defaults to `False`
:raises `AttributeError`: Attribute not found. Only possible if `optional` is `False`
:raises `TypeError`: Attribute found but it is not of the correct type.
Only happens in frontend implementations that do type checking
:raises `ValueError`: Attribute found but can't be coerced to one of the types.
Only happens in frontend implementations that do type coercion
"""
raise NotImplementedError()
Loading