[BUG] Unable to setup the persistent firewall rules: 'ascii' codec can't encode character

[taoyama]

This issue is similar to #2228

If system locale is other than English (ex. Japanese), ascii encoding fails because command result of journalctl contains multibyte characters.

ex: ascii codec can't encode character '水'

[root@rhel76vm ~]# LANG=ja_JP.UTF8 journalctl -u waagent-network-setup.service
-- Logs begin at 水 2021-06-02 00:46:26 UTC, end at 水 2021-06-02 00:50:01 UTC. --
 6月 02 00:48:07 rhel76vm systemd[1]: Starting Setup network rules for WALinuxAgent...
 6月 02 00:48:07 rhel76vm python[11877]: Setting up firewall for the WALinux Agent with args: {'dst_ip': '168.63.129.16', 'uid': '0', 'wait': '-w'}
 6月 02 00:48:07 rhel76vm python[11877]: Successfully set the firewall rules
 6月 02 00:48:07 rhel76vm systemd[1]: Started Setup network rules for WALinuxAgent.

repro steps

1. Create VM

$ az group create -g myResourceGroup -l japaneast
$ az vm create -g myResourceGroup -n rhel7 --image RedHat:RHEL:7.6:7.6.2021051101

2. SSH to the VM
3. Stop firewall and enable waagent-network-setup.service

# systemctl stop firewalld
# systemctl disable firewalld
# iptables -F -t security
# localectl set-locale LANG=ja_JP.UTF8
# systemctl restart waagent
# systemctl start waagent-network-setup.service
# systemctl restart waagent

4. encoding error is logged

# grep ascii /var/log/waagent.log
2021-06-02T00:48:13.879608Z ERROR ExtHandler ExtHandler Unable to setup the persistent firewall rules: 'ascii' codec can't encode character u'\u6c34' in position 81: ordinal not in range(128)

Distro and WALinuxAgent details (please complete the following information):

waagent -version

WALinuxAgent-2.2.46 running on redhat 7.6
Python: 2.7.5
Goal state agent: 2.3.0.2

[kevinclark19a]

Some cursory debugging led me to some code in azurelinuxagent.common.event:EventLogger.add_event:
(( this code starts at azurelinuxagent/common/event.py#L486 on current develop ))

event.parameters.append(TelemetryEventParam(GuestAgentExtensionEventsSchema.Name, str(name)))
event.parameters.append(TelemetryEventParam(GuestAgentExtensionEventsSchema.Version, str(version)))
event.parameters.append(TelemetryEventParam(GuestAgentExtensionEventsSchema.Operation, str(op)))
event.parameters.append(TelemetryEventParam(GuestAgentExtensionEventsSchema.OperationSuccess, bool(is_success)))
event.parameters.append(TelemetryEventParam(GuestAgentExtensionEventsSchema.Message, str(message)))
event.parameters.append(TelemetryEventParam(GuestAgentExtensionEventsSchema.Duration, int(duration)))

The str(...) should instead use azurelinuxagent.common.future.ustr for compatibility with python2.

Should be a quick fix, and I already have a repro machine up and running, so I'll test it and update here.

[kevinclark19a]

Quick update: str -> ustr seems to fix the persistent firewall rules error; I applied a quick patch to my repro machine, and now get

2021-06-08T20:07:21.664990Z INFO ExtHandler ExtHandler Persistent firewall rules setup successfully

instead of the above error. Unfortunately, this isn't quite the end of things, as we get

2021-06-08T20:07:26.706095Z WARNING SendTelemetryHandler ExtHandler Unexpected error when generating Events: 'ascii' codec can't encode character u'\u706b' in position 86: ordinal not in range(128), Traceback (most recent call last):
File "build/bdist.linux-x86_64/egg/azurelinuxagent/common/protocol/wire.py", line 1182, in report_event
event_str = event_to_v1(event)
File "build/bdist.linux-x86_64/egg/azurelinuxagent/common/protocol/wire.py", line 564, in event_to_v1
params += event_param_to_v1(param)
File "build/bdist.linux-x86_64/egg/azurelinuxagent/common/protocol/wire.py", line 558, in event_param_to_v1
attr_type)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u706b' in position 86: ordinal not in range(128)

This is a previously known issue, I believe.