Error when running in a google cloud function

[jmolinap]

Hi

I have a python based (python 3.7 runtime) google cloud function and when trying to get info from an url i get the following error

TypeError: string argument expected, got 'bytes'

The full log is:

packages/google/cloud/functions/worker.py", line 346, in run_http_function result = _function_handler.invoke_user_function(flask.request) File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker.py", line 217, in invoke_user_function return call_user_function(request_or_event) File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker.py", line 210, in call_user_function return self._user_function(request_or_event) File "/user_code/main.py", line 30, in get_info info_dict = ydl.extract_info(url, download=False) File "/env/local/lib/python3.7/site-packages/youtube_dl/YoutubeDL.py", line 796, in extract_info ie_result = ie.extract(url) File "/env/local/lib/python3.7/site-packages/youtube_dl/extractor/common.py", line 530, in extract ie_result = self._real_extract(url) File "/env/local/lib/python3.7/site-packages/youtube_dl/extractor/youtube.py", line 1677, in _real_extract video_webpage = self._download_webpage(url, video_id) File "/env/local/lib/python3.7/site-packages/youtube_dl/extractor/common.py", line 794, in _download_webpage expected_status=expected_status) File "/env/local/lib/python3.7/site-packages/youtube_dl/extractor/youtube.py", line 273, in _download_webpage_handle *args, **compat_kwargs(kwargs)) File "/env/local/lib/python3.7/site-packages/youtube_dl/extractor/common.py", line 660, in _download_webpage_handle urlh = self._request_webpage(url_or_request, video_id, note, errnote, fatal, data=data, headers=headers, query=query, expected_status=expected_status) File "/env/local/lib/python3.7/site-packages/youtube_dl/extractor/common.py", line 602, in _request_webpage self.report_download_webpage(video_id) File "/env/local/lib/python3.7/site-packages/youtube_dl/extractor/common.py", line 922, in report_download_webpage self.to_screen('%s: Downloading webpage' % video_id) File "/env/local/lib/python3.7/site-packages/youtube_dl/extractor/common.py", line 914, in to_screen self._downloader.to_screen('[%s] %s' % (self.IE_NAME, msg)) File "/env/local/lib/python3.7/site-packages/youtube_dl/YoutubeDL.py", line 502, in to_screen return self.to_stdout(message, skip_eol, check_quiet=True) File "/env/local/lib/python3.7/site-packages/youtube_dl/YoutubeDL.py", line 516, in to_stdout self._write_string(output, self._screen_file) File "/env/local/lib/python3.7/site-packages/youtube_dl/YoutubeDL.py", line 505, in _write_string write_string(s, out=out, encoding=self.params.get('encoding')) File "/env/local/lib/python3.7/site-packages/youtube_dl/utils.py", line 3084, in write_string out.buffer.write(byt) TypeError: string argument expected, got 'bytes'

Any advice?

Thanks.

[dstftw]

Looks like you have sys.stdout and/or sys.stderr replaced with custom non-conforming implementations that have buffer.write expecting string instead of bytes-like object.

[davidmarcos98]

Hi,

Same thing happens now to me using Google Cloud Functions with Python 3.7, when running extract_info. Any solution for being able to use it?

Full error log:

Traceback (most recent call last):
File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker.py", line 346, in run_http_function
result = _function_handler.invoke_user_function(flask.request)
File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker.py", line 217, in invoke_user_function
return call_user_function(request_or_event)
File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker.py", line 210, in call_user_function
return self._user_function(request_or_event)
File "/user_code/main.py", line 12, in geturls
info_dict = ydl.extract_info(video, download=False)
File "/env/local/lib/python3.7/site-packages/youtube_dl/YoutubeDL.py", line 797, in extract_info
ie_result = ie.extract(url)
File "/env/local/lib/python3.7/site-packages/youtube_dl/extractor/common.py", line 530, in extract
ie_result = self._real_extract(url)
File "/env/local/lib/python3.7/site-packages/youtube_dl/extractor/skylinewebcams.py", line 26, in _real_extract
webpage = self._download_webpage(url, video_id)
File "/env/local/lib/python3.7/site-packages/youtube_dl/extractor/common.py", line 794, in _download_webpage
expected_status=expected_status)
File "/env/local/lib/python3.7/site-packages/youtube_dl/extractor/common.py", line 660, in _download_webpage_handle
urlh = self._request_webpage(url_or_request, video_id, note, errnote, fatal, data=data, headers=headers, query=query, expected_status=expected_status)
File "/env/local/lib/python3.7/site-packages/youtube_dl/extractor/common.py", line 602, in _request_webpage
self.report_download_webpage(video_id)
File "/env/local/lib/python3.7/site-packages/youtube_dl/extractor/common.py", line 922, in report_download_webpage
self.to_screen('%s: Downloading webpage' % video_id)
File "/env/local/lib/python3.7/site-packages/youtube_dl/extractor/common.py", line 914, in to_screen
self._downloader.to_screen('[%s] %s' % (self.IE_NAME, msg))
File "/env/local/lib/python3.7/site-packages/youtube_dl/YoutubeDL.py", line 503, in to_screen
return self.to_stdout(message, skip_eol, check_quiet=True)
File "/env/local/lib/python3.7/site-packages/youtube_dl/YoutubeDL.py", line 517, in to_stdout
self._write_string(output, self._screen_file)
File "/env/local/lib/python3.7/site-packages/youtube_dl/YoutubeDL.py", line 506, in _write_string
write_string(s, out=out, encoding=self.params.get('encoding'))
File "/env/local/lib/python3.7/site-packages/youtube_dl/utils.py", line 3102, in write_string
out.buffer.write(byt)
TypeError: string argument expected, got 'bytes'

[nitinmishra27]

Hi @davidmarcos98 have you tried fixing the issue?

[x955199]

Hi, any update on this? Getting the exact same issue using Google Cloud Functions with Python 3.7.

Traceback (most recent call last): File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker.py", line 346, in run_http_function result = _function_handler.invoke_user_function(flask.request) File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker.py", line 217, in invoke_user_function return call_user_function(request_or_event) File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker.py", line 210, in call_user_function return self._user_function(request_or_event) File "/user_code/main.py", line 68, in main videoId = videoToAudio(request_json['url']) File "/user_code/main.py", line 54, in videoToAudio r = ydl.extract_info(url, download=False) File "/env/local/lib/python3.7/site-packages/youtube_dl/YoutubeDL.py", line 797, in extract_info ie_result = ie.extract(url) File "/env/local/lib/python3.7/site-packages/youtube_dl/extractor/common.py", line 530, in extract ie_result = self._real_extract(url) File "/env/local/lib/python3.7/site-packages/youtube_dl/extractor/youtube.py", line 1681, in _real_extract video_webpage = self._download_webpage(url, video_id) File "/env/local/lib/python3.7/site-packages/youtube_dl/extractor/common.py", line 794, in _download_webpage expected_status=expected_status) File "/env/local/lib/python3.7/site-packages/youtube_dl/extractor/youtube.py", line 272, in _download_webpage_handle *args, **compat_kwargs(kwargs)) File "/env/local/lib/python3.7/site-packages/youtube_dl/extractor/common.py", line 660, in _download_webpage_handle urlh = self._request_webpage(url_or_request, video_id, note, errnote, fatal, data=data, headers=headers, query=query, expected_status=expected_status) File "/env/local/lib/python3.7/site-packages/youtube_dl/extractor/common.py", line 602, in _request_webpage self.report_download_webpage(video_id) File "/env/local/lib/python3.7/site-packages/youtube_dl/extractor/common.py", line 922, in report_download_webpage self.to_screen('%s: Downloading webpage' % video_id) File "/env/local/lib/python3.7/site-packages/youtube_dl/extractor/common.py", line 914, in to_screen self._downloader.to_screen('[%s] %s' % (self.IE_NAME, msg)) File "/env/local/lib/python3.7/site-packages/youtube_dl/YoutubeDL.py", line 503, in to_screen return self.to_stdout(message, skip_eol, check_quiet=True) File "/env/local/lib/python3.7/site-packages/youtube_dl/YoutubeDL.py", line 517, in to_stdout self._write_string(output, self._screen_file) File "/env/local/lib/python3.7/site-packages/youtube_dl/YoutubeDL.py", line 506, in _write_string write_string(s, out=out, encoding=self.params.get('encoding')) File "/env/local/lib/python3.7/site-packages/youtube_dl/utils.py", line 3102, in write_string out.buffer.write(byt) TypeError: string argument expected, got 'bytes'

[x955199]

@dstftw @davidmarcos98 @jmolinap Seems like for Python 2.x it works cause there is a special condition for it. Not sure what is the expected behavior for Python 3.7 and if I am missing some parameters?

youtube-dl/youtube_dl/utils.py

Line 3096 in 75294a5

or sys.version_info[0] < 3): # Python 2 lies about mode of sys.stderr

[steveobd]

the workaround is to create a logger. (e.g. logger = logging.getLogger('assphinctersayswhat') ) and then pass it into the options
ydl_opts = {
'outtmpl': filename,
'format': 'bestvideo[width<=1280]+bestaudio'
'/bestvideo[height<=720]+bestaudio'
'/bestvideo[ext=mp4]+bestaudio'
'/best',
'retries': 5,
'continuedl': True,
'quiet': True,
'logger': logger
}

[x955199]

thanks @steveobd my workaround was to overwrite sys.stdout

import sys
from io import TextIOWrapper, BytesIO
sys.stdout = TextIOWrapper(BytesIO(), sys.stdout.encoding)