Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure the contents of the field are an array. #63

Merged
merged 5 commits into from
Jun 3, 2019

Conversation

clalancette
Copy link
Contributor

@clalancette clalancette commented May 31, 2019

It turns out that while all AbstractSequences are defined
to be arrays, we allow users to assign other sequences (like
lists) to those fields. Make sure that the field starts with
'array(' before trying to strip it off.

Signed-off-by: Chris Lalancette clalancette@openrobotics.org

This should fix ros2/build_farmer#212

It turns out that while all AbstractSequences are *defined*
to be arrays, we allow users to assign other sequences (like
lists) to those fields.  Make sure that the field starts with
'array(' before trying to strip it off.

Signed-off-by: Chris Lalancette <clalancette@openrobotics.org>
@clalancette
Copy link
Contributor Author

Here's a full CI run:

  • Linux Build Status
  • Linux-aarch64 Build Status
  • macOS Build Status
  • Windows Build Status

@clalancette clalancette added in progress Actively being worked on (Kanban column) in review Waiting for review (Kanban column) and removed in progress Actively being worked on (Kanban column) labels May 31, 2019
# It is possible that the user assigned a non-array type to
# the AbstractSequence field, so only remove 'array(' if it
# is in the string.
if fieldstr.startswith('array('):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please move the check into the else block since there is no reason for it on the if block.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 810ac05

@dirk-thomas
Copy link
Member

When the user assigns a list to an array field I would expect the setter function to ensure that the actually stored value is an array.array: see

@[ if isinstance(member.type, Array)]@
self._@(member.name) = numpy.array(value, dtype=@(SPECIAL_NESTED_BASIC_TYPES[member.type.value_type.typename]['dtype']))
@[ elif isinstance(member.type, AbstractSequence)]@
self._@(member.name) = array.array('@(SPECIAL_NESTED_BASIC_TYPES[member.type.value_type.typename]['type_code'])', value)
@[ end if]@

So I think before making this change we need to investigate why that is not the case (and maybe address that instead).

@clalancette
Copy link
Contributor Author

So I think before making this change we need to investigate why that is not the case (and maybe address that instead).

Yeah, that occurred to me too. I put this up so we could get it in quickly, but since this isn't a rush for release now we should probably investigate that.

Signed-off-by: Chris Lalancette <clalancette@openrobotics.org>
@clalancette
Copy link
Contributor Author

So I think before making this change we need to investigate why that is not the case (and maybe address that instead).

I see where the discrepancy is. The problem shows up for types that are not in the SPECIAL_NESTED_BASIC_TYPES. If the message has a Sequence of booleans, for instance, there is no way to directly map them to an array. So the generated setter only checks if the new value is a valid Sequence, and does a direct assignment. While we could potentially think about coercion to an array for some of the primitive types, this wouldn't work in general for Sequences of non-primitives.

I think the fallout from this is that we do need to handle regular Sequences along with array types, and hence this patch still makes sense. Thoughts?

@dirk-thomas
Copy link
Member

Based on your argument I don't think the current patch is right. I would rather change the condition isinstance(t, rosidl_parser.definition.AbstractSequence) to also check for SPECIAL_NESTED_BASIC_TYPES.

@clalancette
Copy link
Contributor Author

`` to also check for SPECIAL_NESTED_BASIC_TYPES.

The problem with that is that SPECIAL_NESTED_BASIC_TYPES comes from https://github.com/ros2/rosidl_python/blob/master/rosidl_generator_py/rosidl_generator_py/generate_py_impl.py#L37 , and we don't currently import rosidl_generator_py at runtime for the generated messages. Our options seem to be:

  1. Import rosidl_generator_py at runtime
    1. Do it at the top of the file
    2. Do it just in repr (slightly better performance if repr is never called)
  2. Move SPECIAL_NESTED_BASIC_TYPES to rosidl_parser.definition (I'm not sure what additional changes would be required with that)
  3. Keep with what this patch does and use an indirect check (presence of 'array(') to determine this

I'm not really hugely in favor of importing rosidl_generator_py at runtime, mostly because we slow down all message code with an additional import. However, I could be convinced otherwise.

@dirk-thomas
Copy link
Member

How about 4. use SPECIAL_NESTED_BASIC_TYPES in the template and expand the actual values into the generated code?

Signed-off-by: Chris Lalancette <clalancette@openrobotics.org>
@clalancette
Copy link
Contributor Author

clalancette commented May 31, 2019

How about 4. use SPECIAL_NESTED_BASIC_TYPES in the template and expand the actual values into the generated code?

Done in 0e8e345. New CI:

  • Linux Build Status
  • Linux-aarch64 Build Status
  • macOS Build Status
  • Windows Build Status

# a type that we store in an array, strip off the 'array' portion.
if isinstance(t, rosidl_parser.definition.AbstractSequence) and \
isinstance(t.value_type, rosidl_parser.definition.BasicType) and \
t.value_type.typename in @([*SPECIAL_NESTED_BASIC_TYPES]):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick: the multi line condition isn't distinguishable from the indented block. Please use the following style instead:

if (
    ... and
    ...
):
    ...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in f7b18f6

@dirk-thomas dirk-thomas added this to Proposed in Dashing Patch Release 1 via automation May 31, 2019
Signed-off-by: Chris Lalancette <clalancette@openrobotics.org>
Signed-off-by: Chris Lalancette <clalancette@openrobotics.org>
@clalancette
Copy link
Contributor Author

CI is green (Windows test failures are unrelated), so I'm going to merge.

@clalancette clalancette merged commit f692e9c into master Jun 3, 2019
Dashing Patch Release 1 automation moved this from Proposed to Needs Release Jun 3, 2019
@delete-merged-branch delete-merged-branch bot deleted the check-field-type branch June 3, 2019 15:56
clalancette added a commit that referenced this pull request Jun 12, 2019
* Ensure the contents of the field are an array.

It turns out that while all AbstractSequences are *defined*
to be arrays, we allow users to assign other sequences (like
lists) to those fields.  Make sure that the field starts with
'array(' before trying to strip it off.

Signed-off-by: Chris Lalancette <clalancette@openrobotics.org>

* Changes from review.

Signed-off-by: Chris Lalancette <clalancette@openrobotics.org>

* Look at the type.value_type to determine whether it is an array.

Signed-off-by: Chris Lalancette <clalancette@openrobotics.org>

* Review feedback.

Signed-off-by: Chris Lalancette <clalancette@openrobotics.org>

* Fix flake8 warnings.

Signed-off-by: Chris Lalancette <clalancette@openrobotics.org>
clalancette added a commit that referenced this pull request Jun 12, 2019
* Ensure the contents of the field are an array.

It turns out that while all AbstractSequences are *defined*
to be arrays, we allow users to assign other sequences (like
lists) to those fields.  Make sure that the field starts with
'array(' before trying to strip it off.

Signed-off-by: Chris Lalancette <clalancette@openrobotics.org>

* Changes from review.

Signed-off-by: Chris Lalancette <clalancette@openrobotics.org>

* Look at the type.value_type to determine whether it is an array.

Signed-off-by: Chris Lalancette <clalancette@openrobotics.org>

* Review feedback.

Signed-off-by: Chris Lalancette <clalancette@openrobotics.org>

* Fix flake8 warnings.

Signed-off-by: Chris Lalancette <clalancette@openrobotics.org>
@clalancette clalancette moved this from Needs Backport to Released in Dashing Patch Release 1 Jun 12, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
in review Waiting for review (Kanban column)
Projects
No open projects
Development

Successfully merging this pull request may close these issues.

All nightlies have a new failure in test_communication
2 participants