'HTTP Error 400: Bad Request' when setting token_file while using po_token_verifier on AWS Lambda

[yannickErp]

I need to change the token file path, because when using po_token_verifier on AWS Lambda, the file is created in a non-writable directory. But when changing it, I always get Bad Request 400.
This is my code:

tmp_cache_dir = f'{tmp_folder}/pytubefix_cache'
       yt = YouTube(
            video_link, use_po_token=True,
            token_file=os.path.join(tmp_cache_dir, 'tokens.json'),
            po_token_verifier=po_token_verifier_function
       )

It also doesn't matter if I put poToken and visitor_data manually in the console, I get the same result.

Complete Error:

ERROR Error downloading video: HTTP Error 400: Bad Request
Traceback (most recent call last):
  File "/Users/yannickerpelding/Documents/ClipMania/Code/ClipNow/ClipNow-Restructured/Services/Coordinator/src/local.py", line 52, in <module>
    output = main.run(**event)
  File "/Users/yannickerpelding/Documents/ClipMania/Code/ClipNow/ClipNow-Restructured/Services/Coordinator/src/main.py", line 35, in run
    download.download_from_youtube(video_id=global_vars.video_id, max_resolution=global_vars.max_resolution,
  File "/Users/yannickerpelding/Documents/ClipMania/Code/ClipNow/ClipNow-Restructured/Services/Coordinator/src/download.py", line 53, in download_from_youtube
    download_video_from_youtube(
  File "/Users/yannickerpelding/Documents/ClipMania/Code/ClipNow/ClipNow-Restructured/Services/Coordinator/src/download.py", line 158, in download_video_from_youtube
    raise e
  File "/Users/yannickerpelding/Documents/ClipMania/Code/ClipNow/ClipNow-Restructured/Services/Coordinator/src/download.py", line 143, in download_video_from_youtube
    logging.info(f"Looking for best stream to download video in yt streams: {yt.streams}")
  File "/Users/yannickerpelding/anaconda3/envs/clipnow_lambda/lib/python3.10/site-packages/pytubefix/__main__.py", line 600, in streams
    self.check_availability()
  File "/Users/yannickerpelding/anaconda3/envs/clipnow_lambda/lib/python3.10/site-packages/pytubefix/__main__.py", line 311, in check_availability
    status, messages = extract.playability_status(self.vid_info)
  File "/Users/yannickerpelding/anaconda3/envs/clipnow_lambda/lib/python3.10/site-packages/pytubefix/__main__.py", line 413, in vid_info
    innertube_response = innertube.player(self.video_id)
  File "/Users/yannickerpelding/anaconda3/envs/clipnow_lambda/lib/python3.10/site-packages/pytubefix/innertube.py", line 731, in player
    return self._call_api(endpoint, query, self.base_data)
  File "/Users/yannickerpelding/anaconda3/envs/clipnow_lambda/lib/python3.10/site-packages/pytubefix/innertube.py", line 674, in _call_api
    response = request._execute_request(
  File "/Users/yannickerpelding/anaconda3/envs/clipnow_lambda/lib/python3.10/site-packages/pytubefix/request.py", line 35, in _execute_request
    return urlopen(request, timeout=timeout)  # nosec
  File "/Users/yannickerpelding/anaconda3/envs/clipnow_lambda/lib/python3.10/urllib/request.py", line 216, in urlopen
    return opener.open(url, data, timeout)
  File "/Users/yannickerpelding/anaconda3/envs/clipnow_lambda/lib/python3.10/urllib/request.py", line 525, in open
    response = meth(req, response)
  File "/Users/yannickerpelding/anaconda3/envs/clipnow_lambda/lib/python3.10/urllib/request.py", line 634, in http_response
    response = self.parent.error(
  File "/Users/yannickerpelding/anaconda3/envs/clipnow_lambda/lib/python3.10/urllib/request.py", line 563, in error
    return self._call_chain(*args)
  File "/Users/yannickerpelding/anaconda3/envs/clipnow_lambda/lib/python3.10/urllib/request.py", line 496, in _call_chain
    result = func(*args)
  File "/Users/yannickerpelding/anaconda3/envs/clipnow_lambda/lib/python3.10/urllib/request.py", line 643, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 400: Bad Request

[yannickErp]

@JuanBindez ?

[tugaykoyunkaya]

same issue 400 bad request @JuanBindez

[JuanBindez]

Please note, we need more information, the versions of Python, Pytubefix or OS were not provided, so we ask that you provide the necessary information and as much detail as possible so that everyone can check, and I recommend always using the latest versions of pytubefix

[martyna1221]

Hi, I'm having the same problem here. Using the latest version of pytubefix (7.3.0)

I want to run my code (which uses pytubefix) on the cloud (GCP) however, I was getting the bot detected error so I had to add the suggested use_po_token=True parameter

I'm currently testing locally with the use_po_token parameter but I'm encountering the error below

I generated the tokens from the youtube-po-token-generator repository

My Python version is 3.11.9 and I have a Windows machine

This is how I'm calling the YouTube function ->

yt = YouTube(youtube_url, use_po_token=True, po_token_verifier=token_verifier)

With the token_verifier function looking like this (replace X and Y with the tokens generated from running youtube-po-token-generator) ->

def token_verifier():
    po_token = "X"
    visitor_data = "Y"
    return po_token, visitor_data

Traceback:
File "streamlit/runtime/scriptrunner/exec_code.py", line 88, in exec_func_with_error_handling
    result = func()
File "streamlit/runtime/scriptrunner/script_runner.py", line 590, in code_to_exec
    exec(code, module.__dict__)
File "streamlit-app/main.py", line 182, in <module>
    main()
File "streamlit-app/main.py", line 81, in main
    st.write("Title: ", yt.title)
File "pytubefix/__main__.py", line 674, in title
    self._author = self.vid_info.get("videoDetails", {}).get(
File "pytubefix/__main__.py", line 421, in vid_info
    innertube_response = innertube.player(self.video_id)
File "pytubefix/innertube.py", line 785, in player
    return self._call_api(endpoint, query, self.base_data)
File "pytubefix/innertube.py", line 692, in _call_api
    response = request._execute_request(
File "pytubefix/request.py", line 35, in _execute_request
    return urlopen(request, timeout=timeout)
File "urllib/request.py", line 216, in urlopen
    return opener.open(url, data, timeout)
File "urllib/request.py", line 525, in open
    response = meth(req, response)
File "urllib/request.py", line 634, in http_response
    response = self.parent.error(
File "urllib/request.py", line 563, in error
    return self._call_chain(*args)
File "urllib/request.py", line 496, in _call_chain
    result = func(*args)
File "urllib/request.py", line 643, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)

[felipeucelli]

def token_verifier():
   po_token = "X"
   visitor_data = "Y"
   return po_token, visitor_data

@martyna1221
The visitor_data should be returned first and then the po_token. Try it and let us know the result.

[tugaykoyunkaya]

if i use the same method in local server everything works fine, but deploy to heroku, aws etc, i get error
"Error: SPfkZPT1DzM This request was detected as a bot. Use use_po_token=True to view. See more details at https://github.com/JuanBindez/pytubefix/pull/209\n\nOS: linux\nPython: 3.12.7 (main, Oct 1 2024, 06:10:35) [GCC 11.4.0]\nPytubefix: 7.2.2\n"

yt = YouTube(video_url, use_po_token=True, po_token_verifier=po_token_verifier)
def po_token_verifier():
return "xx", "yy"

@JuanBindez 
can we use any server and how?

[trojkat]

This is probably because YT will generate different po_token and visitor_data when visiting YT from a different IP so the data generated using a home IP won't match. In my case, it's not even working when I generate po_token and visitor_data at the same remote server using the docker

docker run quay.io/invidious/youtube-trusted-session-generator
[INFO] internally launching GUI (X11 environment)
[INFO] starting Xvfb
[INFO] launching chromium instance
[INFO] launching browser.
[INFO] waiting 10 seconds for the page to fully load.
visitor_data: <hidden>
po_token: <hidden>
successfully removed temp profile /tmp/uc_q5q8dmg2

>>> from pytubefix import YouTube
>>> url = "<hidden>" 
>>> YouTube(url, use_po_token=True).title
You can use the tool: https://github.com/YunzheZJU/youtube-po-token-generator, to get the token
Enter with your visitorData: <hidden>
Enter with your po_token: <hidden>
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/pytubefix/__main__.py", line 692, in title
    self._title = self.vid_info['videoDetails']['title']
KeyError: 'videoDetails'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.7/site-packages/pytubefix/__main__.py", line 696, in title
    self.check_availability()
  File "/usr/local/lib/python3.7/site-packages/pytubefix/__main__.py", line 349, in check_availability
    raise exceptions.BotDetection(video_id=self.video_id)
pytubefix.exceptions.BotDetection: <hidden> This request was detected as a bot. Use `use_po_token=True` to view. See more details at https://github.com/JuanBindez/pytubefix/pull/209

[arkaydeus]

Is there any way to just get the lowest resolution while hosting on a cloud environment? This would be fine for extracting audio. Currently even with the visitorData and poToken, the bot detection is in force.

[noahdasanaike]

Running

! node youtube-po-token-generator/examples/one-shot.js

then pulling the poToken and visitorData and feeding like this (with some amendments):

from pytubefix import Channel, YouTube

def token_verifier():
    po_token = [hard code]
    visitor_data = [hard code]
    return visitor_data, po_token

yt = YouTube(f"https://www.youtube.com/watch?v=8R19I8rdyR4", 'WEB_EMBED', use_po_token = True, po_token_verifier = token_verifier())

still returns

HTTPError: HTTP Error 400: Bad Request

[arkaydeus]

Yeah, seems like they've upped their game and are blocking datacenter ip ranges.

[AmerAnsari]

Yeah, seems like they've upped their game and are blocking datacenter ip ranges.

That's so true, tried on Frankfurt and Washington with po token, there was no luck.

I thing the edge is to use proxies but even not sure what proxies will work! is there any suggestions?