Unicode Caller ID prevent receiving topic messages

[asmodehn]

I am currently investigating this, while diving into rospy code, so I have only a part of the information...

It seems that, whenever rospy.names.get_name() returns a caller_id that is a unicode string, it then goes into rospy.impl.tcpros_pubsub.get_header_fields() which forwards this to tcpros_base.write_header() and to rosgraph.network.write_ros_handshake_header().
There encode_ros_handshake_header throw 'ascii' codec can't decode byte 0xc5 in position 0: ordinal not in range(128) which is caught in tcpros_base._tcp_server_callback and the following message is displayed:

[WARN] [WallTime: 1448097800.943432] Inbound TCP/IP connection failed: 'ascii' codec can't decode byte 0xc5 in position 0: ordinal not in range(128)

And since the connection doesn't happen some communication breaks somewhere.
It took me a very long time to track this down because :

• There is no information in the error message hinting at where things have gone wrong
• This doesn't break in the main node process, but in the topic subscriber thread, which means that to be able to catch it with a debugger, one needs to actually debug inside rospy itself.

Now here are my two questions :

• what can I do right now to make sure none of my node happen to set the caller_id with a unicode string ? I am currently clueless as to why that caller_id is unicode...
• what can we do to fix this / prevent the error in the long term ?

This happens for me now with https://github.com/asmodehn/rostful but the setup is a bit complicated, identifying the core issue with a simpler setup would be helpful.
I ll keep looking for more clues in the next few days...

[asmodehn]

I managed to trace down the issue to this place :

# check for name override
    mappings = rospy.names.get_mappings()
    print mappings
    if '__name' in mappings:
        name = mappings['__name']
        if anonymous:
            logdebug("[%s] WARNING: due to __name setting, anonymous setting is being changed to false"%name)
            anonymous = False

rospy.names.get_mappings() is actually : {u'__name': u'rostful', u'__log': u'/home/alexv/.ros/log/39179d0e-9181-11e5-aa39-0862664558c8/rostful-3.log'}
So after this section, name changed from raw string r'rostful' (from function param) to unicode u'rostful' (from mappings).

This mapping is unicode because the argument is passed from command line directly ( started with roslaunch ).

So It looks like :

1. the name from command line argument overwrite the name passed as argument in the method.
2. the sys.argv list is not converted/asserted to rawstrings to make sure everything underneath works.
3. rospy.tcpros_pubsub implementation doesn't support unicode strings.

Looking at http://docs.ros.org/api/rospy/html/rospy-module.html#init_node It seems argv should be [str] so that means I need to make that happen on the caller code, before rospy...

[asmodehn]

In case it s of any help to someone else coming around here, this did the trick :

node_str_argv = [unicodedata.normalize('NFKD', arg).encode('ascii', 'ignore') if isinstance(arg, unicode) else str(arg) for arg in node_argv]
rospy.init_node(name=node_name, argv=node_str_argv)

[dirk-thomas]

I think there is still a piece missing here. You mentioned that the variable was a raw string before and afterwards is a unicode string. While you mentioned that the value was both time just rostful the original error message indicates otherwise:

'ascii' codec can't decode byte 0xc5

Unicode or not Python does convert the string automatically. The problem is that the character is not a character with ascii value 0-127. So the question is what actual value was the problem and where did it come from? Maybe you can provide a reproducible example to answer both questions.

[asmodehn]

The string is not converted.
The argument ( unicode ) replaces the function param ( str ) as the node name.
Then rospy.tcpros_pubsub assume the name is str ( ascii ), but it is not.

Doing something like rospy.init_node(r'node_name', argv={u'__name': u'node_name'}) should be enough to trigger the problem...
I ll come up with a detailed example if needed, when i get some time for it.

[dirk-thomas]

Using the single line works fine for me. Please provide a SSCCE.

[asmodehn]

Sorry my argv syntax was wrong.
To be detected as mapping by init_node, it should have been : rospy.init_node(r'node_name', argv={u'__name:=node_name'})

Full Talker node triggering the problem ( plus some boilerplate code to get ROS to initialize from inside python debugger ) : https://gist.github.com/asmodehn/92ff6f9f0cf35a65533c
The warning is displayed and if you plug a listener to it, you will see nothing gets through.
As soon as you remove that unicode arg, things will work.

[dirk-thomas]

Thank you for the reproducible example.

Can you please try the patch from PR #717 and report back if it fixes the problem for you.

[asmodehn]

I tried with this modified talker node and indeed the problem seems fixed !
I don't get the warning anymore, and the message is received by the listener as expected.
The name of the node is still changed to the argv unicode string, but this doesn't prevent the communication to happen as expected.

[dirk-thomas]

Thank you for confirming.