Patched googleads suds package causes UnicodeDecodeError when trying to add non-ascii keyword criteria to Ad Group

[nitzanmargalit]

Hi,
configs:
os == ubuntu 16.04
googleads == 5.0.0
python == 2.7.12

request body XML:

<ns0:Body>
      <ns1:mutate>
         <ns1:operations>
            <ns1:operator>ADD</ns1:operator>
            <ns1:operand xsi:type="ns1:BiddableAdGroupCriterion">
               <ns1:adGroupId>41453028391</ns1:adGroupId>
               <ns1:criterion xsi:type="ns1:Keyword">
                  <ns1:Criterion.Type>Keyword</ns1:Criterion.Type>
                  <ns1:text>שלום</ns1:text>
                  <ns1:matchType>EXACT</ns1:matchType>
               </ns1:criterion>
               <ns1:AdGroupCriterion.Type>BiddableAdGroupCriterion</ns1:AdGroupCriterion.Type>
               <ns1:userStatus>ENABLED</ns1:userStatus>
            </ns1:operand>
         </ns1:operations>
      </ns1:mutate>
   </ns0:Body>

stack trace:

  File "/usr/local/lib/python2.7/dist-packages/googleads/common.py", line 700, in MakeSoapRequest
    *[_PackForSuds(arg, self.suds_client.factory) for arg in args])
  File "/usr/local/lib/python2.7/dist-packages/suds/client.py", line 521, in __call__
    return client.invoke(args, kwargs)
  File "/usr/local/lib/python2.7/dist-packages/suds/client.py", line 581, in invoke
    result = self.send(soapenv)
  File "/usr/local/lib/python2.7/dist-packages/suds/client.py", line 613, in send
    reply = self.options.transport.send(request)
  File "/usr/local/lib/python2.7/dist-packages/googleads/util.py", line 98, in PatchedHttpTransportSend
    fp = self.u2open(u2request)
  File "/usr/local/lib/python2.7/dist-packages/suds/transport/http.py", line 132, in u2open
    return url.open(u2request, timeout=tm)
  File "/usr/lib/python2.7/urllib2.py", line 429, in open
    response = self._open(req, data)
  File "/usr/lib/python2.7/urllib2.py", line 447, in _open
    '_open', req)
  File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 1241, in https_open
    context=self._context)
  File "/usr/lib/python2.7/urllib2.py", line 1195, in do_open
    h.request(req.get_method(), req.get_selector(), req.data, headers)
  File "/usr/lib/python2.7/httplib.py", line 1057, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib/python2.7/httplib.py", line 1097, in _send_request
    self.endheaders(body)
  File "/usr/lib/python2.7/httplib.py", line 1053, in endheaders
    self._send_output(message_body)
  File "/usr/lib/python2.7/httplib.py", line 895, in _send_output
    msg += message_body
UnicodeDecodeError: 'ascii' codec can't decode byte 0xd7 in position 979: ordinal not in range(128)

As you can see, the problem is that in when the code tries to execute 'msg += message_body' ("/usr/lib/python2.7/httplib.py", line 895, in _send_output), 'msg' is unicode object and 'message_body' is str object with non-ascii chars ('שלום').
The problem is that when creating 'msg' by 'msg = "\r\n".join(self._buffer)', one of the elemnt in the buffer is unicode object and thus 'msg' wil also be unicode object. The elemnt in questionn is the uri u'POST /api/adwords/cm/v201609/AdGroupCriterionService HTTP/1.1'.

I've tried to send the keyword text as str/unicode/suds.sax.text.Text object and all of them failed with the same issue described above.
Please let me know if I'm doing something wrong or how to fix this issue.

Thanks in Advance,
Nitzan Margalit

[parander]

Had the same problem here. Tracked it down to
util.py:93
u2request = urllib2.Request(url, msg, headers)

Problem being that url will become a unicode string in the Request object, causing a concatenation later (msg += message_body) to be a unicode += str, which will raise the exception you mentioned (when message_body contains non-ascii).

I patched this locally by just adding str() around the url when the Request object is created:
u2request = urllib2.Request(str(url), msg, headers)

I'm not sure if this is the proper solution though. Better have someone in the dev team to answer that.

[nitzanmargalit]

Hi Par,
I also thought about patching it (since it is such a simple sulotion), but didn't liked the idea of patching an already patched package.
Still waiting for someone from the dev team to pick this issue.

[msaniscalchi]

Hello,

Thanks for the report! I can replicate your issue. It basically comes down to the line setting the url, which should be something like:

url = self.__get_request_url(request)

It appears that suds-jurko's development branch was used as a reference when this patch was written. In that version, suds.transport.Request has been updated to do some sanity checking and this logic was effectively replaced by:

url = request.url

Of course, that nifty sanity checking isn't available in the version we're actually using, so the result is this issue. A temporary work-around would be to change the line setting the URL to something like:

url = suds.transport.http.HttpTransport._HttpTransport__get_request_url(request.url)

This will be fixed in the next release.

Regards,
Mark

[nitzanmargalit]

@msaniscalchi
Just wanted to update that the correct temp fix should be:

url = suds.transport.http.HttpTransport._HttpTransport__get_request_url(request)

since HttpTransport._HttpTransport__get_request_url expects Request object, not a Text object.

Thanks,
Nitzan Margalit