Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HTTP Error 404: Not Found when trying get stream #1060

Closed
oleksandr-shvab opened this issue Jul 21, 2021 · 82 comments · Fixed by #1067
Closed

HTTP Error 404: Not Found when trying get stream #1060

oleksandr-shvab opened this issue Jul 21, 2021 · 82 comments · Fixed by #1067
Labels

Comments

@oleksandr-shvab
Copy link

oleksandr-shvab commented Jul 21, 2021

Describe the bug
I need to get URL to upload the video on AWS. The code is simplest as possible

yt = YouTube(video_url)
streams = yt.streams

The video that I want to download: http://youtube.com/watch?v=2lAe1cqCOXo
On the line where we get streams, it returns the error urllib.error.HTTPError: HTTP Error 404: Not Found
It's happening not always around once at 2-3 requests.

Full error list:

Internal Server Error: /api/external-video-upload/
Traceback (most recent call last):
  File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/django/core/handlers/base.py", line 179, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/sentry_sdk/integrations/django/views.py", line 67, in sentry_wrapped_callback
    return callback(request, *args, **kwargs)
  File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
    return view_func(*args, **kwargs)
  File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/django/views/generic/base.py", line 70, in view
    return self.dispatch(request, *args, **kwargs)
  File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/rest_framework/views.py", line 509, in dispatch
    response = self.handle_exception(exc)
  File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/rest_framework/views.py", line 469, in handle_exception
    self.raise_uncaught_exception(exc)
  File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
    raise exc
  File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/rest_framework/views.py", line 506, in dispatch
    response = handler(request, *args, **kwargs)
  File "/home/alex/Developer/blossom-backend/blossom/recordings/views.py", line 819, in get
    youtube_video_upload(video_url, recording)
  File "/home/alex/Developer/blossom-backend/blossom/recordings/utils.py", line 791, in youtube_video_upload
    streams = yt.streams
  File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/pytube/__main__.py", line 321, in streams
    return StreamQuery(self.fmt_streams)
  File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/pytube/__main__.py", line 214, in fmt_streams
    if "adaptive_fmts" in self.player_config_args:
  File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/pytube/__main__.py", line 188, in player_config_args
    self._player_config_args = self.vid_info
  File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/pytube/__main__.py", line 291, in vid_info
    return dict(parse_qsl(self.vid_info_raw))
  File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/pytube/__main__.py", line 109, in vid_info_raw
    self._vid_info_raw = request.get(self.vid_info_url)
  File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/pytube/request.py", line 53, in get
    response = _execute_request(url, headers=extra_headers, timeout=timeout)
  File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/pytube/request.py", line 37, in _execute_request
    return urlopen(request, timeout=timeout)  # nosec
  File "/home/alex/.pyenv/versions/3.9.0/lib/python3.9/urllib/request.py", line 214, in urlopen
    return opener.open(url, data, timeout)
  File "/home/alex/.pyenv/versions/3.9.0/lib/python3.9/urllib/request.py", line 523, in open
    response = meth(req, response)
  File "/home/alex/.pyenv/versions/3.9.0/lib/python3.9/urllib/request.py", line 632, in http_response
    response = self.parent.error(
  File "/home/alex/.pyenv/versions/3.9.0/lib/python3.9/urllib/request.py", line 561, in error
    return self._call_chain(*args)
  File "/home/alex/.pyenv/versions/3.9.0/lib/python3.9/urllib/request.py", line 494, in _call_chain
    result = func(*args)
  File "/home/alex/.pyenv/versions/3.9.0/lib/python3.9/urllib/request.py", line 641, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found
Internal Server Error: /api/external-video-upload/
Traceback (most recent call last):
  File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/django/core/handlers/base.py", line 179, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/sentry_sdk/integrations/django/views.py", line 67, in sentry_wrapped_callback
    return callback(request, *args, **kwargs)
  File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
    return view_func(*args, **kwargs)
  File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/django/views/generic/base.py", line 70, in view
    return self.dispatch(request, *args, **kwargs)
  File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/rest_framework/views.py", line 509, in dispatch
    response = self.handle_exception(exc)
  File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/rest_framework/views.py", line 469, in handle_exception
    self.raise_uncaught_exception(exc)
  File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
    raise exc
  File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/rest_framework/views.py", line 506, in dispatch
    response = handler(request, *args, **kwargs)
  File "/home/alex/Developer/blossom-backend/blossom/recordings/views.py", line 819, in get
    youtube_video_upload(video_url, recording)
  File "/home/alex/Developer/blossom-backend/blossom/recordings/utils.py", line 791, in youtube_video_upload
    streams = yt.streams
  File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/pytube/__main__.py", line 321, in streams
    return StreamQuery(self.fmt_streams)
  File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/pytube/__main__.py", line 214, in fmt_streams
    if "adaptive_fmts" in self.player_config_args:
  File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/pytube/__main__.py", line 188, in player_config_args
    self._player_config_args = self.vid_info
  File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/pytube/__main__.py", line 291, in vid_info
    return dict(parse_qsl(self.vid_info_raw))
  File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/pytube/__main__.py", line 109, in vid_info_raw
    self._vid_info_raw = request.get(self.vid_info_url)
  File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/pytube/request.py", line 53, in get
    response = _execute_request(url, headers=extra_headers, timeout=timeout)
  File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/pytube/request.py", line 37, in _execute_request
    return urlopen(request, timeout=timeout)  # nosec
  File "/home/alex/.pyenv/versions/3.9.0/lib/python3.9/urllib/request.py", line 214, in urlopen
    return opener.open(url, data, timeout)
  File "/home/alex/.pyenv/versions/3.9.0/lib/python3.9/urllib/request.py", line 523, in open
    response = meth(req, response)
  File "/home/alex/.pyenv/versions/3.9.0/lib/python3.9/urllib/request.py", line 632, in http_response
    response = self.parent.error(
  File "/home/alex/.pyenv/versions/3.9.0/lib/python3.9/urllib/request.py", line 561, in error
    return self._call_chain(*args)
  File "/home/alex/.pyenv/versions/3.9.0/lib/python3.9/urllib/request.py", line 494, in _call_chain
    result = func(*args)
  File "/home/alex/.pyenv/versions/3.9.0/lib/python3.9/urllib/request.py", line 641, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found

System information

  • Python version 3.9
  • Pytube version 10.9.3
@github-actions
Copy link

Thank you for contributing to PyTube. Please remember to reference Contributing.md

@AbdullahElrubi
Copy link

I'm getting this error too now, I've an up and running app that has errors with all videos.
@tfdahlin please propose a fix for this issue

@rishabh3354
Copy link

rishabh3354 commented Jul 21, 2021

Same error happening for me as well
urllib.error.HTTPError: HTTP Error 404: Not Found After trying 2-5 times, it will return 200. This is very annoying..Please help @tfdahlin

@AbdullahElrubi
Copy link

@rishabh3354 actually this is expected since YouTube always block apis like these and make updates on them from time to time.

@rishabh3354
Copy link

@rishabh3354 actually this is expected since YouTube always block apis like these and make updates on them from time to time.

So there must be a solution for this.. Actually it is quite irritating.. please suggest a solution.

@gustav-py
Copy link

@ rishabh3354 na verdade, isso é esperado, já que o YouTube sempre bloqueia aplicativos como esses e faz atualizações neles de vez em quando.

Portanto, deve haver uma solução para isso .. Na verdade, é bastante irritante .. por favor, sugira uma solução.

This error already happened in the past, basically youtube updated the URLs, so PyTube couldn't download the video, at the time the solution was a new Pytube update, at the moment I'm not having problems in my application with this error, maybe that's just it is happening to some people.

@tfdahlin
Copy link
Collaborator

I can't fix this tonight, I will try to work on it tomorrow

@oleksandr-shvab
Copy link
Author

If yesterday one of the 2-3 requests was a success, today all requests failed.
I tried to handle it in the cycle, and if the request fails then repeat the request. Else return stream, but now all requests fail.

@KoubaPetr
Copy link

Same here, yesterday happening with every second video, so I was bombing it in a loop till it worked out. Helped for few hours, but today I am unable to download a single video, not even from a different IP address.

Grateful for this active community, authors and contributors! Hope we can find a workaround/fix.

@rishabh3354
Copy link

If yesterday one of the 2-3 requests was a success, today all requests failed.
I tried to handle it in the cycle, and if the request fails then repeat the request. Else return stream, but now all requests fail.

Yes now all requests is failing

@chandeepadissanayake
Copy link

Failing for all of the videos now.

@Pw-Wolf
Copy link

Pw-Wolf commented Jul 22, 2021

If yesterday one of the 2-3 requests was a success, today all requests failed.
I tried to handle it in the cycle, and if the request fails then repeat the request. Else return stream, but now all requests fail.

Yes now all requests is failing

Same happening here.

@ByTE1974
Copy link

Same error

@bigbear22941
Copy link

dito, failing all videos

@wes1993
Copy link

wes1993 commented Jul 22, 2021

Same for me.. some solutions?

@aviallon
Copy link

I can't fix this tonight, I will try to work on it tomorrow

https://github.com/ytdl-org/youtube-dl are able to do it by parsing the webpage.

@joon612
Copy link

joon612 commented Jul 22, 2021

I can't fix this tonight, I will try to work on it tomorrow

https://github.com/ytdl-org/youtube-dl are able to do it by parsing the webpage.

I saw same issue recently in it, are you sure about this can work?

@aviallon
Copy link

I can't fix this tonight, I will try to work on it tomorrow

https://github.com/ytdl-org/youtube-dl are able to do it by parsing the webpage.

I saw same issue recently in it, are you sure about this can work?

I just used it right now to download an entire playlist, and it worked (while pytube didn't).

@Zeecka
Copy link
Contributor

Zeecka commented Jul 23, 2021

Pytube is using the /get_video_info endpoint which seems to have change in the past days/hours: https://stackoverflow.com/questions/67615278/get-video-info-youtube-endpoint-suddenly-returning-404-not-found (note: the &html5=1&c=TVHTML5&cver=6.20180913 parameters are already provided by pytube)

This was referenced Jul 23, 2021
@tfdahlin tfdahlin linked a pull request Jul 24, 2021 that will close this issue
@shreyesh0610
Copy link

Do we have an update on this, since this being an awesome library and with a critical issue like this one, I am having an impact on an already running app. Request to please kindly provide an update on this issue. Also, thanks for the awesome work you guys are doing!

@shohamtal
Copy link

shohamtal commented Jul 29, 2021

You can test Zeecka's experimental solution. Worked great for me:
pip install git+https://github.com/Zeecka/pytube@fix_1060

Had to change my code to explicit download mp4 format, otherwise it downloaded 3gp format:
YouTube("https://www.youtube.com/watch?v=7kZqas35FTA").streams.filter(progressive=True, file_extension='mp4').first().download()

@tfdahlin
Copy link
Collaborator

Update: I made a couple slight changes to Zeecka's solution that should help with the age restrictions. I believe the current version will now work correctly for most age-restricted videos without needing to authenticate. There are certain age-restricted videos that are even more restricted than others that will require authentication, but these videos are very uncommon.

If you are able to, please install the experimental version, test it out, and let us know if there are any unexpected errors. If you run into an AgeRestrictedError exception, then you are probably testing against a video that has the higher level of age-restriction, and will need to authenticate. You can do that as follows:

yt = YouTube(url, use_oauth=True)

The first time that you run this code with the use_oauth argument, it will prompt you to visit a URL and input a code to authorize access to your account. This generates an authorization and refresh OAuth token, which are then stored on the machine running the code. Doing this introduces a potential security risk. Specifically, those tokens are scoped to YouTube, which means that if they get exposed, somebody could perform certain actions as though they were your account. Although this does not give direct access to your account, it does allow access to certain features. I'm a little fuzzy on exactly what this scope allows, but YouTube's public API states that the scope allows the token to Manage your YouTube account. If you do NOT want to store these locally, you can pass another argument, allow_oauth_cache=False to prevent them from being stored on disk.

You can install the experimental fix to test it with python -m pip install git+https://github.com/Zeecka/pytube@fix_1060, as @shohamtal says above.

@gustav-py
Copy link

Atualização: fiz algumas pequenas alterações na solução do Zeecka que devem ajudar com as restrições de idade. Acredito que a versão atual agora funcionará corretamente para a maioria dos vídeos com restrição de idade, sem a necessidade de autenticação. Há alguns vídeos com restrição de idade que são ainda mais restritas do que outros que vai exigem autenticação, mas esses vídeos são muito incomuns.

Se puder, instale a versão experimental, teste-a e informe-nos se houver algum erro inesperado. Se você encontrar uma AgeRestrictedErrorexceção, provavelmente está testando um vídeo que tem o nível mais alto de restrição de idade e precisará ser autenticado. Você pode fazer isso da seguinte maneira:

yt  =  YouTube ( url , use_oauth = True )

Na primeira vez que você executar este código com o use_oauthargumento, ele solicitará que você visite um URL e insira um código para autorizar o acesso à sua conta. Isso gera uma autorização e um token OAuth de atualização , que são armazenados na máquina que executa o código. Fazer isso apresenta um risco potencial de segurança . Especificamente, esses tokens têm como escopo o YouTube , o que significa que, se forem expostos, alguém pode realizar certas ações como se fossem sua conta. Embora isso não conceda acesso direto à sua conta, permite o acesso a certos recursos. Estou um pouco confuso sobre exatamente o que esse escopo permite, mas a API pública do YouTube afirma que o escopo permite que o tokenManage your YouTube account. Se você NÃO deseja armazená-los localmente, pode passar outro argumento, allow_oauth_cache=Falsepara evitar que sejam armazenados no disco.

Você pode instalar a correção experimental para testá-la python -m pip install git+https://github.com/Zeecka/pytube@fix_1060, como @shohamtal disse acima.

I'm glad you had a solution for the error 🙌🙌, I just have a doubt that is: If in my code I don't put the use_oauth, and put it to download normally nothing will be stored on my machine right?

@tfdahlin
Copy link
Collaborator

I'm glad you had a solution for the error 🙌🙌, I just have a doubt that is: If in my code I don't put the use_oauth, and put it to download normally nothing will be stored on my machine right?

That is correct. Nothing is stored on your machine if you don't include the use_oauth argument.

@partriv
Copy link

partriv commented Jul 30, 2021

python -m pip install git+https://github.com/Zeecka/pytube@fix_1060 worked for me, thank you SO MUCH @tfdahlin. I will keep testing it out and let you know if i encounter any issues.

@rishabh3354
Copy link

rishabh3354 commented Jul 30, 2021

python -m pip install git+https://github.com/Zeecka/pytube@fix_1060 worked for me, thank you SO MUCH @tfdahlin. I will keep testing it out and let you know if i encounter any issues.

i downloaded dash stream video and found extension is missing in the downloaded files. Previously extension (for eg .mp4 or.webm ) is automatically included in the downloaded videos. I dont know if its a bug or new feature but my project breaks with this.
i am using ubuntu 20.04 @Zeecka

@VellichorPrism
Copy link

VellichorPrism commented Jul 30, 2021

python -m pip install git+https://github.com/Zeecka/pytube@fix_1060 Worked for me. Using the regular pip install pytube isn't. Will the fix be implemented into the main repo?

@Loki-Hornsby
Copy link

@LeopardsLeone ah sorry, my bad thought the traceback might provide some help incase it was different to any others

@tfdahlin
Copy link
Collaborator

i downloaded dash stream video and found extension is missing in the downloaded files. Previously extension (for eg .mp4 or.webm ) is automatically included in the downloaded videos. I dont know if its a bug or new feature but my project breaks with this.
i am using ubuntu 20.04 @Zeecka

@rishabh3354 This behavior changed in v10.9.3, and should only happen if you're using the filename argument when downloading. When you specify a custom filename, you should now include the extension as part of the filename (which you can access with stream.subtype). If you do not use the filename argument, a filename will be automatically generated that correctly includes the extension.

python -m pip install git+https://github.com/Zeecka/pytube@fix_1060 Worked for me. Using the regular pip install pytube isn't. Will the fix be implemented into the main repo?

Yes, it will, but it needs to be tested more thoroughly before I push it there, hence why I'm asking for help testing.

@Zeecka
Copy link
Contributor

Zeecka commented Jul 30, 2021

Just so you know, even if I did the initial PR, @tfdahlin was already aware of the workaround and worked on the biggest part of the fix (oauth / Age restriction bypass). Thank you for your work.

@kettenbach-it
Copy link

Yes, it will, but it needs to be tested more thoroughly before I push it there, hence why I'm asking for help testing.

Since my comment #1060 (comment) I downloaded a couple of more videos with the patch 1060 applied and all worked fine!

However, I just noticed something else on this topic:
As my code snippet already posted here #1060 (comment) shows:

videostreams = ytb.streams.filter(adaptive=True, only_video=True).order_by('resolution').desc()
for videostream in videostreams:
   try: 
     [...]
     videodownloadresult = videostream.download(gettempdir(), filename_prefix="video_")
   except urllib.error.HTTPError as error:
     [...] -> try next stream
[...]
-> same with audio

I use "try" and "except" when downloading streams.
The background is that I have noticed in the past that not all streams that streams.filter delivers are actually available. For whatever reason this is the case - maybe YT still processes these formats or similar - my code simply uses the stream with the next-lower resolution in this case.
I have always done well with this and my programme works very reliably.
This may be a tip for all those who are struggling with 404 errors....

@rishabh3354
Copy link

i downloaded dash stream video and found extension is missing in the downloaded files. Previously extension (for eg .mp4 or.webm ) is automatically included in the downloaded videos. I dont know if its a bug or new feature but my project breaks with this.
i am using ubuntu 20.04 @Zeecka

@rishabh3354 This behavior changed in v10.9.3, and should only happen if you're using the filename argument when downloading. When you specify a custom filename, you should now include the extension as part of the filename (which you can access with stream.subtype). If you do not use the filename argument, a filename will be automatically generated that correctly includes the extension.

python -m pip install git+https://github.com/Zeecka/pytube@fix_1060 Worked for me. Using the regular pip install pytube isn't. Will the fix be implemented into the main repo?

Yes, it will, but it needs to be tested more thoroughly before I push it there, hence why I'm asking for he

i downloaded dash stream video and found extension is missing in the downloaded files. Previously extension (for eg .mp4 or.webm ) is automatically included in the downloaded videos. I dont know if its a bug or new feature but my project breaks with this.
i am using ubuntu 20.04 @Zeecka

@rishabh3354 This behavior changed in v10.9.3, and should only happen if you're using the filename argument when downloading. When you specify a custom filename, you should now include the extension as part of the filename (which you can access with stream.subtype). If you do not use the filename argument, a filename will be automatically generated that correctly includes the extension.

python -m pip install git+https://github.com/Zeecka/pytube@fix_1060 Worked for me. Using the regular pip install pytube isn't. Will the fix be implemented into the main repo?

Yes, it will, but it needs to be tested more thoroughly before I push it there, hence why I'm asking for help testing.

Thanks for the information. I am testing it thoroughly, if any bug comes up, will let you know. Thanks @tfdahlin

@partriv
Copy link

partriv commented Jul 30, 2021

Just so you know, even if I did the initial PR, @tfdahlin was already aware of the workaround and worked on the biggest part of the fix (oauth / Age restriction bypass). Thank you for your work.

Thank you @Zeecka as well! I saw all the work you and @tfdahlin put into the PR. You are inspiring me to start contributing to this project as well.

@rishabh3354
Copy link

rishabh3354 commented Jul 31, 2021

git+https://github.com/Zeecka/pytube@fix_1060 this is working quite smoothly in my project. i havenot found any bugs till now. Merge it to main soon. Thanks, Fantastic work @tfdahlin, @Zeecka
One thing i noted in this fix. Youtube video info parsing is 2x faster than previous version (in my case it is, i am not sure about the changes but it is way faster) or am i the only one who is experiencing it?

@VisionsOfDrifting
Copy link

VisionsOfDrifting commented Jul 31, 2021

Zeecka's fix branch is working for me too.

pip install git+https://github.com/Zeecka/pytube@fix_1060 

Downloaded 150 videos no issues. It also seems faster to me.

@must-defend-500
Copy link

must-defend-500 commented Jul 31, 2021

Hey guys, I'm getting HTTP 410 Gone but the video is definitely not gone. Maybe a bug?

Traceback (most recent call last):
  File "get_youtube_video.py", line 57, in <module>
    main()
  File "get_youtube_video.py", line 42, in main
    stream = yt.streams.filter(progressive=True, file_extension='mp4').first()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pytube/__main__.py", line 321, in streams
    return StreamQuery(self.fmt_streams)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pytube/__main__.py", line 214, in fmt_streams
    if "adaptive_fmts" in self.player_config_args:
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pytube/__main__.py", line 188, in player_config_args
    self._player_config_args = self.vid_info
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pytube/__main__.py", line 291, in vid_info
    return dict(parse_qsl(self.vid_info_raw))
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pytube/__main__.py", line 109, in vid_info_raw
    self._vid_info_raw = request.get(self.vid_info_url)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pytube/request.py", line 53, in get
    response = _execute_request(url, headers=extra_headers, timeout=timeout)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pytube/request.py", line 37, in _execute_request
    return urlopen(request, timeout=timeout)  # nosec
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 531, in open
    response = meth(req, response)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 641, in http_response
    'http', request, response, code, msg, hdrs)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 569, in error
    return self._call_chain(*args)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 503, in _call_chain
    result = func(*args)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 410: Gone

To clarify, this is with pip install git+https://github.com/Zeecka/pytube@fix_1060

@tfdahlin
Copy link
Collaborator

Hey guys, I'm getting HTTP 410 Gone but the video is definitely not gone. Maybe a bug?

To clarify, this is with pip install git+https://github.com/Zeecka/pytube@fix_1060

It looks like the 404 has been changed to a 410. You're seeing this issue because you do not have the experimental branch correctly installed (In your traceback, I see references to self.vid_info_url, which is no longer in the codebase in the experimental branch)

@must-defend-500
Copy link

@tfdahlin good catch, Ive got python and python3 issues on my computer. I don't know why I can't intsall it from Github. I'll just wait for you guys to merge with master. Any sense when that might happen?

@tfdahlin
Copy link
Collaborator

tfdahlin commented Aug 1, 2021

@must-defend-500 I'm hoping to get to it tomorrow. I need to fix a bunch of unit tests, and it's taking a while (most of the old ones relied on data generated by the old get_vid_info url, so now that objects are being created in a different way, its causing issues with our mock objects)

@rishabh3354
Copy link

One thing i notice, i am able to download age restricted video from pytube. And when i am trying to download same video from youtube-dl, it is giving me error. youtube-dl currently not handling age restricted videos. but pytube does. great job @tfdahlin

@HACKER097
Copy link

You can test Zeecka's experimental solution. Worked great for me:
pip install git+https://github.com/Zeecka/pytube@fix_1060

Had to change my code to explicit download mp4 format, otherwise it downloaded 3gp format:
YouTube("https://www.youtube.com/watch?v=7kZqas35FTA").streams.filter(progressive=True, file_extension='mp4').first().download()

I've been testing it a bit, and captions do not work with the experimental solution.

>>> from pytube import YouTube
>>> yt = YouTube('http://youtube.com/watch?v=2lAe1cqCOXo')
>>> yt.captions
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/hacker097/.local/lib/python3.8/site-packages/pytube/__main__.py", line 299, in captions
    return pytube.CaptionQuery(self.caption_tracks)
  File "/home/hacker097/.local/lib/python3.8/site-packages/pytube/__main__.py", line 291, in caption_tracks
    return [pytube.Caption(track) for track in raw_tracks]
  File "/home/hacker097/.local/lib/python3.8/site-packages/pytube/__main__.py", line 291, in <listcomp>
    return [pytube.Caption(track) for track in raw_tracks]
  File "/home/hacker097/.local/lib/python3.8/site-packages/pytube/captions.py", line 22, in __init__
    self.name = caption_track["name"]["simpleText"]
KeyError: 'simpleText'
>>>

@dogproblems
Copy link

@must-defend-500

I don't know why I can't intsall it from Github.

try pip uninstall pytube first, make sure the old version is gone. worked for me

tfdahlin pushed a commit that referenced this issue Aug 3, 2021
* User InnerTube in place of `get_video_info` url

* Added some additional base parameters for innertube requests.

* Added Oauth support for innertube client

* Add exception for age-restricted videos which can no longer be accessed without using auth.

* Carved out and simplified code where possible due to API changes.

* Added renderer catch -- fixes #1068

* Additional channel name support for URL-encoded names.

* Updated test mocks, removed region-locked test because that functionality no longer works.
@tfdahlin
Copy link
Collaborator

tfdahlin commented Aug 3, 2021

The fix for this has been pushed to pypi as v11.0.0, so you should be able to install it normally with python -m pip install --upgrade pytube

@must-defend-500
Copy link

@tfdahlin you rock, thank you for the lib and the fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.