AdWords API - Parallel report download - Getting Errors

[chaitu1124]

Hi, 

I have more than 270 advertiser accounts. Right now pulling the reports one by one by looping. I come to know we can use parallel report download inorder to download all together. 

I tried to execute the sample code in the following URL.

https://developers.google.com/adwords/api/docs/samples/python/reporting

I have placed googleads.yaml in home directory and changed File directory in the script. When I ran the script got the following errors. 

Please help me to understand these errors.

Error Message:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Python27\lib\multiprocessing\forking.py", line 381, in main
    self = load(from_parent)
  File "C:\Python27\lib\pickle.py", line 1384, in load
    return Unpickler(file).load()
  File "C:\Python27\lib\pickle.py", line 864, in load
    dispatch[key](self)
  File "C:\Python27\lib\pickle.py", line 1089, in load_newobj
    obj = cls.__new__(cls, *args)
TypeError: __new__() takes at least 2 arguments (1 given)
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Python27\lib\multiprocessing\forking.py", line 381, in main
    self = load(from_parent)
  File "C:\Python27\lib\pickle.py", line 1384, in load
    return Unpickler(file).load()
  File "C:\Python27\lib\pickle.py", line 864, in load
    dispatch[key](self)
  File "C:\Python27\lib\pickle.py", line 1089, in load_newobj
    obj = cls.__new__(cls, *args)
TypeError: __new__() takes at least 2 arguments (1 given)
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Python27\lib\multiprocessing\forking.py", line 381, in main
    self = load(from_parent)
  File "C:\Python27\lib\pickle.py", line 1384, in load
    return Unpickler(file).load()
  File "C:\Python27\lib\pickle.py", line 864, in load
    dispatch[key](self)
  File "C:\Python27\lib\pickle.py", line 1089, in load_newobj
    obj = cls.__new__(cls, *args)
TypeError: __new__() takes at least 2 arguments (1 given)
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Python27\lib\multiprocessing\forking.py", line 381, in main
    self = load(from_parent)
  File "C:\Python27\lib\pickle.py", line 1384, in load
    return Unpickler(file).load()
  File "C:\Python27\lib\pickle.py", line 864, in load
    dispatch[key](self)
  File "C:\Python27\lib\pickle.py", line 1089, in load_newobj
    obj = cls.__new__(cls, *args)
TypeError: __new__() takes at least 2 arguments (1 given)
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Python27\lib\multiprocessing\forking.py", line 381, in main
    self = load(from_parent)
  File "C:\Python27\lib\pickle.py", line 1384, in load
    return Unpickler(file).load()
  File "C:\Python27\lib\pickle.py", line 864, in load
    dispatch[key](self)
  File "C:\Python27\lib\pickle.py", line 1089, in load_newobj
    obj = cls.__new__(cls, *args)
TypeError: __new__() takes at least 2 arguments (1 given)
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Python27\lib\multiprocessing\forking.py", line 381, in main
    self = load(from_parent)
  File "C:\Python27\lib\pickle.py", line 1384, in load
    return Unpickler(file).load()
  File "C:\Python27\lib\pickle.py", line 864, in load
    dispatch[key](self)
  File "C:\Python27\lib\pickle.py", line 1089, in load_newobj
    obj = cls.__new__(cls, *args)
TypeError: __new__() takes at least 2 arguments (1 given)
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Python27\lib\multiprocessing\forking.py", line 381, in main
    self = load(from_parent)
  File "C:\Python27\lib\pickle.py", line 1384, in load
    return Unpickler(file).load()
  File "C:\Python27\lib\pickle.py", line 864, in load
    dispatch[key](self)
  File "C:\Python27\lib\pickle.py", line 1089, in load_newobj
    obj = cls.__new__(cls, *args)
TypeError: __new__() takes at least 2 arguments (1 given)
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Python27\lib\multiprocessing\forking.py", line 381, in main
    self = load(from_parent)
  File "C:\Python27\lib\pickle.py", line 1384, in load
    return Unpickler(file).load()
  File "C:\Python27\lib\pickle.py", line 864, in load
    dispatch[key](self)
  File "C:\Python27\lib\pickle.py", line 1089, in load_newobj
    obj = cls.__new__(cls, *args)
TypeError: __new__() takes at least 2 arguments (1 given)
Finished downloading reports with the following results:

[msaniscalchi]

Hello,

Thanks for the report! I just ran this sample in Python 2 and 3 and didn't see a similar error in either case. I don't think the error you're reporting is directly related to parallel_report_download.py because the stacktrace you provided indicates that the error consistently occurs on line 381, where pickle is being used to deserialize something. The sample is well under 381 lines, and doesn't make use of pickle.

The error indicates that something is going wrong in the deserialization of whatever is being loaded on line 381.

Regards,
Mark

[msaniscalchi]

Actually, I've found a very similar report indicating that this is very likely due to differences in Windows. I'll investigate what needs to be changed for compatibility on Windows--it sounds like ReportWorker would need to be pickleable.

-- Mark

[chaitu1124]

is there any update on this?

[msaniscalchi]

As stated in my prior comment, it looks like the underlying issue with the sample is that there are some differences in a Windows environment. It appears that there are some changes we can make that would resolve that. As for the timing of the fix, we generally work to include fixes for lower-priority issues with Sunset or new API version releases.

While I can't guarantee it, I expect the fix for this to be packaged with either the upcoming DFP release / AdWords sunset, or the next AdWords release.

Regards,
Mark

[msaniscalchi]

This will be resolved alongside the upcoming DFP release.

The core issue here is that AdWordsClient and ReportDownloader are not serializable with pickle. A multiprocessing.Process instance (or subclass instance, in this case) must be serializable when used on a Windows environment, so the inclusion of an AdWordsClient instance in the ReportWorker's initializer (and usage in _DownloadReport) is the underlying cause. To work around this for now, you could have a top-level function that effectively replaces _DownloadReport, which is called in ReportWorker.run.

Regards,
Mark

[msaniscalchi]

This was fixed in the last release.