Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for authentication on globo.com #4189

Closed
alessanderbotti opened this issue Nov 13, 2014 · 20 comments
Closed

Support for authentication on globo.com #4189

alessanderbotti opened this issue Nov 13, 2014 · 20 comments

Comments

@alessanderbotti
Copy link

@alessanderbotti alessanderbotti commented Nov 13, 2014

Thank you very much for supporting globo.com videos (#3823). Can you please support also the authentication options --username and --password on globo.com? It is not working on version 2014.11.13.3 (the newest by now).

An example URL would be:

http://globotv.globo.com/rede-globo/globo-reporter/t/para-assinantes/v/globo-reporter-amazonia-nas-alturas-07112014/3751209/

And running the following command gives the output:
youtube-dl -u username -p password -t --console-title -v http://globotv.globo.com/rede-globo/globo-reporter/t/para-assinantes/v/globo-reporter-amazonia-nas-alturas-07112014/3751209/

[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['-u', u'PRIVATE', '-p', u'PRIVATE', '-t', '--console-title', '-v', 'http://globotv.globo.com/rede-globo/globo-reporter/t/para-assinantes/v/globo-reporter-amazonia-nas-alturas-07112014/3751209/']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2014.11.13.3
[debug] Python version 2.7.8 - Linux-3.16.0-24-generic-x86_64-with-Ubuntu-14.10-utopic
[debug] exe versions: avconv 11-6, avprobe 11-6, ffmpeg 2.0.1, ffprobe 2.0.1
[debug] Proxy map: {}
[Globo] rede-globo/globo-reporter/t/para-assinantes/v/globo-reporter-amazonia-nas-alturas-07112014/3751209/: Downloading webpage
[Globo] 3751209: Downloading JSON metadata
[Globo] 3751209: Downloading security hash for 3751209_wmsm
ERROR: Globo returned error: Video is for subscribers only. Authorization required.
Traceback (most recent call last):
File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 536, in extract_info
ie_result = ie.extract(url)
File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 199, in extract
return self._real_extract(url)
File "/usr/local/bin/youtube-dl/youtube_dl/extractor/globo.py", line 370, in _real_extract
'%s returned error: %s' % (self.IE_NAME, message), expected=True)
ExtractorError: Globo returned error: Video is for subscribers only. Authorization required.

@naglis naglis added the request label Nov 13, 2014
@dstftw
Copy link
Collaborator

@dstftw dstftw commented Dec 19, 2014

globo subscription is paid-only. We can add support for it only if you are willing to provide valid account credentials.

@wbecher
Copy link

@wbecher wbecher commented Sep 10, 2017

If anyone is going to add support to this, I can borrow my account info... just message me: wbecher @ gmail.com

@adamstraube
Copy link

@adamstraube adamstraube commented Sep 14, 2017

There is a Kodi plugin written that allows streaming of pay walled Globo movies by vitorhirota here: https://bitbucket.org/vitorhirota/repository.brazilian.xbmc-addons/overview

I believe this part of the code does the authentication: https://bitbucket.org/vitorhirota/repository.brazilian.xbmc-addons/src/29ce020aea66cb281c3bf55607a6f528ee60b3ee/plugin.video.globotv/resources/lib/globo.py?at=master#globo.py-157

I tried getting it to work in youtube_dl but the _download... functions were not working for me. I couldn't get it to work with requests.post either (might need to check if the URL works?). I would need to read into how youtube_dl requests stuff before going further.

With that being said anyone else want to have a go?

@adamstraube
Copy link

@adamstraube adamstraube commented Sep 15, 2017

OK so now I have an issue submitting to the authentication script. It gives me a 400 error. I suspect it is looking for a cookie variable, specifically 'glb_uid' but could not work out how it is generated.

@adamstraube
Copy link

@adamstraube adamstraube commented Oct 3, 2017

I did not have any luck getting youtube-dl to authenticate. However I had some luck in copying the cookies out of a logged in Globo session in browser into cookies.txt and adding it to the youtube-dl command eg. youtube-dl --verbose --cookies ~/cookies.txt "https://globoplay.globo.com/v/...".
It does seem to time out and give access denied after some period of time in my experience though which doesn't really make sense as re-running the command straight after works again.

@arbarbosa1979
Copy link

@arbarbosa1979 arbarbosa1979 commented Nov 9, 2017

I can provide valid credentials for you to test this site.

@dstftw dstftw mentioned this issue Jan 8, 2018
4 of 8 tasks complete
@Troyciv
Copy link

@Troyciv Troyciv commented Feb 18, 2018

Authentification is now mandatory even for free (non-paid) videos (like this one).

youtube-dl -u email -p password -v https://globoplay.globo.com/v/6475372/programa/

gives the following error:


[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-u', 'PRIVATE', '-p', 'PRIVATE', '-v', 'https://globoplay.globo.com/v/6475372/programa/']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2017.09.24
[debug] Python version 3.6.3 - Linux-4.13.0-32-generic-x86_64-with-Ubuntu-17.10-artful
[debug] exe versions: ffmpeg 3.3.4-2, ffprobe 3.3.4-2, rtmpdump 2.4
[debug] Proxy map: {}
[Globo] 6475372: Downloading JSON metadata
[Globo] 6475372: Downloading security hash for 6475372_90_720_collection
ERROR: Globo returned error: Video is for subscribers only. Authorization required.
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/youtube_dl/YoutubeDL.py", line 777, in extract_info
    ie_result = ie.extract(url)
  File "/usr/lib/python3/dist-packages/youtube_dl/extractor/common.py", line 434, in extract
    ie_result = self._real_extract(url)
  File "/usr/lib/python3/dist-packages/youtube_dl/extractor/globo.py", line 360, in _real_extract
    '%s returned error: %s' % (self.IE_NAME, message), expected=True)
youtube_dl.utils.ExtractorError: Globo returned error: Video is for subscribers only. Authorization required.

OS: Ubuntu 17.10

edit: @adamstraube could you tell me the name of the correct cookie, so I can try to run youtube-dl with --cookies?

@Troyciv
Copy link

@Troyciv Troyciv commented Feb 20, 2018

I managed to use cookies for globo as well. As @adamstraube mentioned it seems to time out pretty quickly. I managed to download the first few fragments of this video and then got the following error.

...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 167 (attempt 10 of 10)...
[download] Skipping fragment 167...
...

Interestingly though, I can start downloading another video with the same cookie and get the same error again after a few dozens fragments.

@saitodisse
Copy link

@saitodisse saitodisse commented Mar 18, 2018

Hello guys, the cookies.txt trick did worked for me.
I've used the cookies.txt Google Chrome extension (https://chrome.google.com/webstore/detail/cookiestxt/njabckikapfpffapmjgojcnbfjonfjfg?hl=pt-BR) and I've used the command --cookies option like @adamstraube suggested. Just works.

youtube-dl.exe https://globoplay.globo.com/v/6475372/programa/ --cookies "./cookies.txt"

@Troyciv you have to login on globo and generate your own cookies

@farrista
Copy link

@farrista farrista commented Mar 19, 2018

Here, even adding the cookies, still does not work.

$ youtube-dl -v --cookies ~/Downloads/cookies.txt https://globosatplay.globo.com/globonews/v/6570759/

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'--cookies', u'/home/raul/Downloads/cookies.txt', u'https://globosatplay.globo.com/globonews/v/6570759/']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2018.03.14
[debug] Python version 2.7.12 (CPython) - Linux-4.13.0-37-generic-x86_64-with-LinuxMint-18.3-sylvia
[debug] exe versions: ffmpeg 2.8.11-0ubuntu0.16.04.1, ffprobe 2.8.11-0ubuntu0.16.04.1
[debug] Proxy map: {}
[Globo] 6570759: Downloading JSON metadata
[Globo] 6570759: Downloading security hash for 5aa60bba636d61cf40000000_90_720_m3u8
ERROR: Globo returned error: Video is for subscribers only. Authorization required.
Traceback (most recent call last):
File "/home/raul/.local/lib/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 785, in extract_info
ie_result = ie.extract(url)
File "/home/raul/.local/lib/python2.7/site-packages/youtube_dl/extractor/common.py", line 440, in extract
ie_result = self._real_extract(url)
File "/home/raul/.local/lib/python2.7/site-packages/youtube_dl/extractor/globo.py", line 360, in _real_extract
'%s returned error: %s' % (self.IE_NAME, message), expected=True)
ExtractorError: Globo returned error: Video is for subscribers only. Authorization required.

@marcusmuller
Copy link

@marcusmuller marcusmuller commented Apr 19, 2018

Alguém conseguiu com Globosatplay? O Globo play já consegui!

Globoplay:

youtube-dl.exe -u usuario@exemplo.com -p senha --external-downloader aria2c.exe https://globoplay.globo.com/v/6670684/programa/

*lembra só de instalar o aria2c.exe e inserir no PATH

@arbarbosa1979
Copy link

@arbarbosa1979 arbarbosa1979 commented Apr 19, 2018

Globosatplay tem um intermediário, no caso você faz login primeiro no site da operadora, e depois é redirecionado para o globosatplay, então teria que capturar o método usado nesse meio para conseguir acessar os videos. Eu não tenho conhecimento necessário para isso, mas acho que olhando nas ferramentas do desenvolvedor é possível identificar que página é usada para solicitar o login na operadora e depois repassar para o globosatplay

@Troyciv
Copy link

@Troyciv Troyciv commented May 20, 2018

Youtube-dl still doesn't work for me.
I am situated in Europe, if that matters.
I'll give a short summary here of what I've tried so far.

OS: Ubuntu 17.10

I do have a globoplay account (free/non-paid).
I tested a few videos on their websites e.g. "jornal nacional" or "fantástico".
They do work inside the browser with my free account.
I am aware that some videos need a paid subscription but for the videos I've chosen this is not the case.

I tried to use youtube-dl with the --cookies option without success. First I logged in via Firefox and then exported the cookies with the extension "cookies.txt". I opened the file and verified my globo cookies are in there.

Next I fired up youtube-dl --verbose --cookies cookies.txt https://globoplayglobo.com/v/6748891/programa/

I get the following error:

[debug] System config: []                                                                                                                      
[debug] User config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'--verbose', u'--cookies', u'/home/user/Downloads/cookies.txt', u'https://globoplay.globo.com/v/6748891/programa/']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8       
[debug] youtube-dl version 2018.05.18                                  
[debug] Python version 2.7.14 (CPython) - Linux-4.13.0-41-generic-x86_64-with-Ubuntu-17.10-artful
[debug] exe versions: ffmpeg 3.3.4-2, ffprobe 3.3.4-2, rtmpdump 2.4    
[debug] Proxy map: {}              
[Globo] 6748891: Downloading JSON metadata                             
[Globo] 6748891: Downloading security hash for 6748891_90_720_collection
ERROR: Globo returned error: Video is for subscribers only. Authorization required.
Traceback (most recent call last): 
  File "/home/user/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 792, in extract_info
    ie_result = ie.extract(url)    
  File "/home/user/bin/youtube-dl/youtube_dl/extractor/common.py", line 503, in extract
    ie_result = self._real_extract(url)                                
  File "/home/user/bin/youtube-dl/youtube_dl/extractor/globo.py", line 360, in _real_extract
    '%s returned error: %s' % (self.IE_NAME, message), expected=True)  
ExtractorError: Globo returned error: Video is for subscribers only. Authorization required. 

Interesting to say is that globoplay seems to send set-cookie headers.
After I run youtube-dl with --cookies all cookies of my cookies.txt got deleted.
This was not the case a few months ago.
This is how my cookies.txt looks like after I ran youtube-dl --cookies:

# Netscape HTTP Cookie File
# http://curl.haxx.se/rfc/cookie_spec.html
# This is a generated file!  Do not edit.

Next, I tried youtube-dl --verbose -u user -p password https://globoplay.globo.com/v/6748891/programa/

I got the same error:

[debug] System config: []                                                                                                                      
[debug] User config: []                                                                                                                        
[debug] Custom config: []
[debug] Command-line args: [u'--verbose', u'-u', u'PRIVATE', u'-p', u'PRIVATE', u'https://globoplay.globo.com/v/6748891/programa/']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8       
[debug] youtube-dl version 2018.05.18                                  
[debug] Python version 2.7.14 (CPython) - Linux-4.13.0-41-generic-x86_64-with-Ubuntu-17.10-artful
[debug] exe versions: ffmpeg 3.3.4-2, ffprobe 3.3.4-2, rtmpdump 2.4    
[debug] Proxy map: {}              
[Globo] 6748891: Downloading JSON metadata                             
[Globo] 6748891: Downloading security hash for 6748891_90_720_collection
ERROR: Globo returned error: Video is for subscribers only. Authorization required.
Traceback (most recent call last): 
  File "/home/user/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 792, in extract_info
    ie_result = ie.extract(url)    
  File "/home/user/bin/youtube-dl/youtube_dl/extractor/common.py", line 503, in extract
    ie_result = self._real_extract(url)                                
  File "/home/user/bin/youtube-dl/youtube_dl/extractor/globo.py", line 360, in _real_extract
    '%s returned error: %s' % (self.IE_NAME, message), expected=True)  
ExtractorError: Globo returned error: Video is for subscribers only. Authorization required.

@remitamine
Copy link
Collaborator

@remitamine remitamine commented May 20, 2018

cookies-txt add-on prefix HTTP only cookies with #HttpOnly_, make sure that you remove the prefix before using the cookies.txt file with youtube-dl.

@Troyciv
Copy link

@Troyciv Troyciv commented May 20, 2018

@remitamine There were no #HttpOnly_ prefixes to the globo cookies. I removed them from the whole file anyway, just in case. No difference, same error.

@remitamine
Copy link
Collaborator

@remitamine remitamine commented May 20, 2018

works fine with cookies(globo is looking for GLBID cookie):

youtube-dl --cookies ~/Downloads/cookies.txt https://globoplay.globo.com/v/6748891
[Globo] 6748891: Downloading JSON metadata
[Globo] 6748891: Downloading security hash for 6748891_90_720_collection
[Globo] 6748891_90_720_collection: Downloading m3u8 information
[Globo] 6748891: Downloading security hash for 6748891_90_720_l_collection
[Globo] 6748891_90_720_l_collection: Downloading m3u8 information
[Globo] 6748891: Downloading security hash for 6748891_360_1080_collection
[Globo] 6748891_360_1080_collection: Downloading m3u8 information
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 276
[download] Destination: Jornal Nacional - Íntegra 19 Maio 2018-6748891.mp4
[download]   0.2% of ~700.00MiB at 130.74KiB/s ETA 01:55:37
@Troyciv
Copy link

@Troyciv Troyciv commented May 21, 2018

I do have these two GLBID cookies:

.globo.com	TRUE	/	FALSE	0	GLBID	126005[...]
.globoplay.globo.com	TRUE	/	FALSE	0	GLBID	126005a1[...]

Still get the error. Could it have to do something with geo-restriction or user-agent?

@remitamine
Copy link
Collaborator

@remitamine remitamine commented May 21, 2018

Still get the error. Could it have to do something with geo-restriction or user-agent?

no, it's not, most likely it's caused by the expiration date in you're file(0), it should look like:

.globo.com	TRUE	/	FALSE	3674449688	GLBID	<hash>
@Troyciv
Copy link

@Troyciv Troyciv commented May 21, 2018

Yes, that's it. Omg, thank you so much. I just replaced the 0 with your number, now it works.

I checked my other cookies. All (!) of them have a 0. Any idea why that could be?

Edit: I think I found the issue. It's a browser setting. In Firefox I had set the cookie setting to "keep cookies until I close Firefox". I set them back to "until they expire", now the exported cookies.txt have expiration dates as well.

@Troyciv
Copy link

@Troyciv Troyciv commented May 21, 2018

I'm afraid the problem is not completely solved yet. The download starts but after ~20min I get 403: forbitten errors:

[debug] System config: []                                                                                                                      
[debug] User config: []                                                                                                                        
[debug] Custom config: []                                                                                                                      
[debug] Command-line args: [u'--cookies', u'/home/user/Downloads/cookies.txt', u'--verbose', u'https://globoplay.globo.com/v/6749576/programa/'
]                                                                                                                                              
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8                                                                               
[debug] youtube-dl version 2018.05.18                                                                                                          
[debug] Python version 2.7.14 (CPython) - Linux-4.13.0-41-generic-x86_64-with-Ubuntu-17.10-artful                                              
[debug] exe versions: ffmpeg 3.3.4-2, ffprobe 3.3.4-2, rtmpdump 2.4                                                                            
[debug] Proxy map: {}                                                                                                                          
[Globo] 6749576: Downloading JSON metadata                                                                                                     
[Globo] 6749576: Downloading security hash for 6749576_90_720_collection                                                                       
[Globo] 6749576_90_720_collection: Downloading m3u8 information                                                                                
[Globo] 6749576: Downloading security hash for 6749576_90_720_l_collection                                                                     
[Globo] 6749576_90_720_l_collection: Downloading m3u8 information                                                                              
[Globo] 6749576: Downloading security hash for 6749576_360_1080_collection                                                                     
[Globo] 6749576_360_1080_collection: Downloading m3u8 information 
[debug] Default format spec: bestvideo+bestaudio/best                  
[debug] Invoking downloader on u'https://vodstreaming01a.video.globo.com/r360_1080/v30/dd/c3/09/6749576_09baffdf32e1c03d2b94c541ab31670f24a2cb36/6749576-manifest.smil.ism/6749576-manifest.smil-audio_por_1=128000-video_por=3538222.m3u8?hls_client_manifest_version=4'
[hlsnative] Downloading m3u8 manifest                                  
[hlsnative] Total fragments: 785   
[download] Destination: Fantástico – Edição de 20_05_2018-6749576.mp4  
[download]  23.7% of ~3.43GiB at  1.03MiB/s ETA 01:02:56[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 187 (attempt 1 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 187 (attempt 2 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 187 (attempt 3 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 187 (attempt 4 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 187 (attempt 5 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 187 (attempt 6 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 187 (attempt 7 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 187 (attempt 8 of 10)...
^C               
ERROR: Interrupted by user

If I abort the download with Ctrl-c and restart (with the same cookie) it it will continue for another ~20min until I get the same error again.

@dstftw dstftw mentioned this issue Jun 1, 2018
5 of 9 tasks complete
@Troyciv Troyciv mentioned this issue Jun 16, 2018
5 of 5 tasks complete
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
You can’t perform that action at this time.