cannot login to coursera.org

[shirishag75]

Once I did as shared at #339 I get the following now :-

[$] python coursera-dl -n --on-demand --path=/home/shirish/Videos/Coursera/Classes/ recommender-systems                              
Downloading class: recommender-systems
Starting new HTTPS connection (1): www.coursera.org
Could not authenticate: Cannot login on coursera.org.

The class can be seen at https://www.coursera.org/learn/recommender-systems/

trying out the --debug version gives the following :-

[$] python coursera-dl -n --on-demand --path=/home/shirish/Videos/Coursera/Classes/ recommender-systems --debug                     
root[authenticate_through_netrc] Trying netrc file None
root[main] Downloading class: recommender-systems
root[login] Initiating login.
root[login] There were no .coursera.org cookies to be cleared.
root[login] Forging cookie header: csrftoken=xxxxxxxxxxxx; csrf2_token_xxxxxxxx=xxxxxxxxxxxxxxxx.
urllib3.connectionpool[_new_conn] Starting new HTTPS connection (1): www.coursera.org
urllib3.connectionpool[_make_request] "POST /api/login/v3 HTTP/1.1" 400 None
root[main] Could not authenticate: Cannot login on coursera.org.

The xxxxxx are there just for protecting privacy.

[rbrito]

What environment are you using? More details about Python interpreter and so on would be helpful. The packages installed would also be nice to know.

All that I can say here is that it works for me with my Debian sid system and the dependencies all pulled from sid (yes, I don't use pip myself). :)

[rbrito]

BTW, what is the output of git show? Doing a git pull is almost always a good thing, as I try to be very careful (read: "things have to work for me at least") when committing things.

[shirishag75]

Hi @rbrito ,
Python is :-

python --version                                                                                                               
Python 2.7.9

These are the various dependencies which I have. You will notice that python-agparse is not there but then it is supposed to be installed only with python 2.6 and I'm on python 2.7

[$] dpkg -l python-bs4  python-requests python-six python-html5lib python-setuptools                                               
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                           Version              Architecture         Description
+++-==============================-====================-====================-=================================================================
ii  python-bs4                     4.3.2-2              all                  error-tolerant HTML parser for Python
ii  python-html5lib                0.999-3              all                  HTML parser/tokenizer based on the WHATWG HTML5 specification (Py
ii  python-requests                2.4.3-6              all                  elegant and simple HTTP library for Python2, built for human bein
ii  python-setuptools              12.2-1               all                  Python Distutils Enhancements
ii  python-six                     1.9.0-1              all                  Python 2 and 3 compatibility library (Python 2 interface)

My ~/.netrc is like :-

[$] cat ~/.netrc                                                                                                                   
machine coursera-dl login <"myemailid"> password <"$PASSWORD">

I haven't shared my username and password, if need be will send it to your @debian.org address. I don't know whether or not machine needs to be machine only or should I be sharing the hostname there.

You can also look at #339 as it has the same details as the one shared above.

[MeghnaNatraj]

I am facing the same issue :/ @shirishag75 Did you find a fix to this issue?

[rbrito]

Hi.

On May 03 2015, Meghna Natraj wrote:

I am facing the same issue :/ @shirishag75 Did you find a fix to this issue?

One of the most important modules currently for connecting with coursera is
the requests module and its dependencies.

@shirishag75, there's no need (yet) to send me any personal information.

Can you please report the result (via pip) of the versions of the
following modules?

• requests
• urllib3
• pyasn1
• ndg-httpsclient
• pyOpenSSL

The last one is the more problematic one that involves compilation,
depending on your setup. @MeghnaNatraj, what system are you using? The same
as @shirishag75?

I would love to know if the instructions from
https://github.com/coursera-dl/coursera#alternative-installation-method-for-unix-systems
help you or not.

BTW, in Debian systems, the pip executable is available in the package
python-pip.

Thanks,

Rogério Brito.

Rogério Brito : rbrito@{ime.usp.br,gmail.com} : GPG key 4096R/BCFCAAAA
http://rb.doesntexist.org/blog : Projects : https://github.com/rbrito/
DebianQA: http://qa.debian.org/developer.php?login=rbrito%40ime.usp.br

[MeghnaNatraj]

@rbrito I am using a Mac OS. Version 10.9.5
Also could you give me more context into how to find out the version of the modules you mentioned?
I am not aware of which command i must use and how to go about it.

If you need more information let me know. I will let you know.

[rbrito]

Hi, Meghna.

On May 04 2015, Meghna Natraj wrote:

@rbrito I am using a Mac OS. Version 10.9.5
Also could you give me more context into how to find out the version of
the modules you mentioned?

Sure, no problems. These are the commands that you can use to find out the
versions of the programs that you are using:

    python --version
    openssl version
    pip freeze

The first command may only be available if you are using Python 2 on your
computer. I don't know what version of Python Apple ships with their
operating system (Python 2 or Python 3). If you only have Python 3
installed, the command to see its version is python3 --version.

The last command may generate a lot of output if you are not using a virtual
environment (which you should, as that separates the needed dependencies of
our program from other python programs that you may have installed).

If you are using a package manager like homebrew or macports, upgrading your
version of Python should be simple (and they usually do a very good job at
keeping up with the latest versions of the programs---but take this with
caution, as I have very limited experience with MacOS X).

Also in my experience, having the newest Python interpreter in the lineage
that you are using (for Python 2, that's 2.7.9, while for Python 3 it is
3.4.3) fixes all the problems that the users are having, especially now that
the world has been changing regarding HTTPS (e.g., with both sites and
cryptographic tools dropping support for SSLv3 and programs having to adapt
to this).

Also, it is important to know which method you used to install coursera-dl.
If you installed with pip, then I have released an update today (version
0.0.3). If you installed via a git clone, then always issuing a git pull
will grab my latest changes (and I usually only commit things that both
passes our automated tests and that work for me at least, because, as you
may guess, I use the script daily).

I am not aware of which command i must use and how to go about it.

I hope that the commands above help us know what packages you have
installed.

If you decide to upgrade things, please, do proceed with one step at a
time and stop at the first that make things work. Also, please report
which step made things work, so that we can document that and save the
collective knowledge for the futurer.

• First, upgrade the Python interpreter.
• Next, upgrade the Python modules. The more important ones are requests,
  urllib3, pyasn1, and ndg-httpsclient. This last one may require
  some compilation and an up-to-date OpenSSL.

If you need more information let me know. I will let you know.

Please let me know if these instructions work for you. If they do and you
don't mind, please write a few paragraphs and contribute to our
documentation (the README.md file, especially the troubleshooting section).

If the instructions don't work, let me know at which step you stopped and
what messages you get when things don't work.

Thanks,

Rogério Brito : rbrito@{ime.usp.br,gmail.com} : GPG key 4096R/BCFCAAAA
http://cynic.cc/blog/ : github.com/rbrito : profiles.google.com/rbrito
DebianQA: http://qa.debian.org/developer.php?login=rbrito%40ime.usp.br

[wang-boyu]

Same issue here, with OS X Yosemite 10.10.3

python --version:

Python 2.7.9

openssl version:

OpenSSL 0.9.8zd 8 Jan 2015

pip freeze:

beautifulsoup4==4.3.2
cffi==0.9.2
cryptography==0.9
enum34==1.0.4
html5lib==1.0b6
idna==2.0
ipaddress==1.0.7
keyring==5.3
ndg-httpsclient==0.4.0
pyasn1==0.1.7
pycparser==2.13
pyOpenSSL==0.15.1
requests==2.7.0
six==1.9.0
urllib3==1.10.4

The error I got is:

(my-coursera)XXXXX-MBP:coursera XXXXX$ ./coursera-dl -u XXXXX@gmail.com -p XXXXX --on-demand machine-learning
Downloading class: machine-learning
Starting new HTTPS connection (1): www.coursera.org
Could not authenticate: Cannot login on coursera.org.

If I add --debug:

(my-coursera) XXXXX-MBP:coursera XXXXX $ ./coursera-dl -u XXXXX@gmail.com -p XXXXX --on-demand machine-learning --debug
root[main] Downloading class: machine-learning
root[login] Initiating login.
root[login] There were no .coursera.org cookies to be cleared.
root[login] Forging cookie header: csrftoken= XXXXX; csrf2_token_ XXXXX = XXXXX.
requests.packages.urllib3.connectionpool[_new_conn] Starting new HTTPS connection (1): www.coursera.org
requests.packages.urllib3.connectionpool[_make_request] "POST /api/login/v3 HTTP/1.1" 401 None
root[main] Could not authenticate: Cannot login on coursera.org.

<-c cookies.txt> will result in the same error.

[shirishag75]

Actually, I didn't use pip but used debian in-house things. Apart from one or two dependencies, most of them are in debian.

[$] dpkg -l python-requests python-urllib3 python-pyasn1 python-openssl  python-ndg-httpsclient                                                          

Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                           Version              Architecture         Description
+++-==============================-====================-====================-=================================================================
ii  python-openssl                 0.15.1-1             all                  Python 2 wrapper around the OpenSSL library
ii  python-pyasn1                  0.1.7-1              all                  ASN.1 library for Python (Python 2 module)
ii  python-requests                2.7.0-1              all                  elegant and simple HTTP library for Python2, built for human bein
ii  python-urllib3                 1.10.4-1             all                  HTTP library with thread-safe connection pooling for Python
ii  python-ndg-httpsclient         0.3.2-1              all                  enhanced HTTPS support for httplib and urllib2 using PyOpenSSL

The only one which is not available yet in debian is python-ndg-httpclient. Those versions might be outdated but looking for an updated version of those if that might help. I don't like using pip or any of such tools as far as possible.

[rbrito]

Hi.

On May 19 2015, shirish wrote:

Actually, I didn't use pip but used debian in-house things. Apart from one
or two dependencies, most of them are in debian.

(...)

The only one which is not available yet in debian is
python-ndg-httpclient.

This is available in Debian, as I have it here, but you missed an s there:
python-ndg-httpSclient (the caps is only for emphasis, of course).

In fact, python-ndg-httpsclient, together with python-openssl is really
important here, but if the problem you are seeing here is not with SSL (it
doesn't look like it is), then I guess that it won't change much.

Does your ~/.netrc file really contain the angle brackets? They should
not be there.

Trying to debug this issue, I installed "from scratch" the package on both
an armel machine with 128MB of RAM and a powerpc machine also with 128MB of
RAM.

When I installed the program on the powerpc machine I mistyped my e-mail
address and I was getting login failures (exactly this one that you are
reporting), but that went away as I fixed the typo.

Just to isolate the netrc thing, can you pass your username and password on
the command line on your machine and double check them, please?

Those versions might be outdated but looking for an updated version of
those if that might help. I don't like using pip or any of such tools as
far as possible.

If you install a virtualenv (which I described in the README file), then all
modules will be installed in a subdirectory of your choice (almost like a
chroot) and no other files in your system will be affected.

Or, if you prefer, you can install a full blown chroot (or a docker
container) and download your videos there. But let's first start with the
simpler things first.

Regards,

Rogério Brito : rbrito@{ime.usp.br,gmail.com} : GPG key 4096R/BCFCAAAA
http://cynic.cc/blog/ : github.com/rbrito : profiles.google.com/rbrito
DebianQA: http://qa.debian.org/developer.php?login=rbrito%40ime.usp.br

[shirishag75]

Updated my previous answer to show the version of python-ndg-httpsclient as well. I indeed had the angle brackets and the quotation marks, removed both of them.

This is how my .netrc looks now :-

machine coursera-dl login xxxx@xxx.com password xxxxxxxxxxxxx

Of course, the name and password are to protect my privacy.

These are the steps I took :-

a. Updated coursera-dl to the latest master by doing

$ git pull origin master

Now I'm at 9d0ee94

Once this was done, again tried to use the same example as above.

This time hit it on the first attempt :-

[$] python coursera-dl -n --on-demand --path=/home/shirish/Videos/Coursera/Classes/ recommender-systems                             
Downloading class: recommender-systems
Starting new HTTPS connection (1): www.coursera.org
Logged in on coursera.org.
Downloaded https://www.coursera.org/api/opencourse.v1/course/recommender-systems (48160 bytes)
Parsing syllabus of on-demand course. This may take some time, be patient ...

So actually this was not an issue at your end but more about how .netrc should have looked at my end. Apparently apart from < even " is a funny character.

I would suggest adding to the notes the following so that others do not fall into my trap. If needed, wanted would do a PR although it would involve me forking the project, making a commit there and doing a PR to you, while you'll be able to do it more easily and smoothly by yourself.

On *nix platforms, the use of a ~/.netrc file is a good alternative to specifying both your username (i.e., your email address) and password every time on the command line. To use it, simply add a line like the one below to a file named .netrc in your home directory (or the equivalent, if you are using Windows) with contents like:

machine coursera-dl login abcd password xyzzyi

this is where login/username is abcd and the password is xyzzyi .

If you are successful you will give output something like this :-

[$] python coursera-dl -n --on-demand --path=/home/abcd/Videos/Coursera/Classes/ recommender-systems                            
Downloading class: recommender-systems
Starting new HTTPS connection (1): www.coursera.org
Logged in on coursera.org.

Things to note here :-
a. You need to share the path where the file will eventually download.
b. The moment it says logged in, the part about .netrc is finished.

I am keeping the bug open On *nix platforms, the use of a ~/.netrc file is a good alternative to specifying both your username (i.e., your email address) and password every time on the command line. To use it, simply add a line like the one below to a file named .netrc in your home directory (or the equivalent, if you are using Windows) with contents like:

machine coursera-dl login abcd password xyzzyi

this is where login/username is abcd and the password is xyzzyi .

If you are successful you will give output something like this :-

[$] python coursera-dl -n --on-demand --path=/home/abcd/Videos/Coursera/Classes/ recommender-systems                            
Downloading class: recommender-systems
Starting new HTTPS connection (1): www.coursera.org
Logged in on coursera.org.

Things to note here :-
a. You need to share the path where the file will eventually download.
b. The moment it says logged in, the part about .netrc is finished.

I am keeping the bug open On *nix platforms, the use of a ~/.netrc file is a good alternative to specifying both your username (i.e., your email address) and password every time on the command line. To use it, simply add a line like the one below to a file named .netrc in your home directory (or the equivalent, if you are using Windows) with contents like:

machine coursera-dl login abcd password xyzzyi

this is where login/username is abcd and the password is xyzzyi .

If you are successful you will give output something like this :-

[$] python coursera-dl -n --on-demand --path=/home/abcd/Videos/Coursera/Classes/ recommender-systems                            
Downloading class: recommender-systems
Starting new HTTPS connection (1): www.coursera.org
Logged in on coursera.org.

Things to note here :-
a. You need to share the path where the file will eventually download.
b. The moment it says logged in, the part about .netrc is finished.

I am keeping the bug open for now as there were other voices in here as well. My issue is solved and hopefully others would also learn something from the process. I would leave it to @rbrito and others @MeghnaNatraj and @boyus to either use this space or make their own tickets, either way thank you for sticking around :)

[wang-boyu]

Thank you @shirishag75 and @rbrito for your generous help. My issue was resolved, which turns out to be a simple passcode problem.

My careless mistake has cost me several days of debugging :)

[victorwestmann]

@boyus glad to hear that. :)

Are you guys still facing this issue @MeghnaNatraj and @shirishag75 ?

@rbrito how can I find out the coursera-dl version by the command line?
Is there a simple way to do this? :)

[daniel-barrows]

I had a similar error message. Then I discovered I could not use my coursera username, but only the email address associated with my coursera account.

[rbrito]

@sondra-kinsey, can you please send a change to our main document (README.md) making explicit the language that, perhaps, wasn't clear in the first place?

[kishoreo]

Please Help me , I could not able to download the courses
C:\Users\Kishore Odugu>coursera-dl -u myusername -p mypassword ml-regression
coursera_dl version 0.11.4
Traceback (most recent call last):
File "c:\python37\lib\site-packages\urllib3\connectionpool.py", line 600, in urlopen
chunked=chunked)
File "c:\python37\lib\site-packages\urllib3\connectionpool.py", line 343, in _make_request
self._validate_conn(conn)
File "c:\python37\lib\site-packages\urllib3\connectionpool.py", line 849, in validate_conn
conn.connect()
File "c:\python37\lib\site-packages\urllib3\connection.py", line 356, in connect
ssl_context=context)
File "c:\python37\lib\site-packages\urllib3\util\ssl.py", line 359, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "c:\python37\lib\ssl.py", line 412, in wrap_socket
session=session
File "c:\python37\lib\ssl.py", line 850, in _create
self.do_handshake()
File "c:\python37\lib\ssl.py", line 1108, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1045)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "c:\python37\lib\site-packages\requests\adapters.py", line 445, in send
timeout=timeout
File "c:\python37\lib\site-packages\urllib3\connectionpool.py", line 638, in urlopen
_stacktrace=sys.exc_info()[2])
File "c:\python37\lib\site-packages\urllib3\util\retry.py", line 398, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='api.coursera.org', port=443): Max retries exceeded with url: /api/login/v3 (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1045)')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "c:\python37\lib\runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "c:\python37\lib\runpy.py", line 85, in run_code
exec(code, run_globals)
File "C:\Python37\Scripts\coursera-dl.exe_main.py", line 9, in
File "c:\python37\lib\site-packages\coursera\coursera_dl.py", line 236, in main
login(session, args.username, args.password)
File "c:\python37\lib\site-packages\coursera\cookies.py", line 146, in login
headers=headers, allow_redirects=False)
File "c:\python37\lib\site-packages\requests\sessions.py", line 559, in post
return self.request('POST', url, data=data, json=json, **kwargs)
File "c:\python37\lib\site-packages\requests\sessions.py", line 512, in request
resp = self.send(prep, **send_kwargs)
File "c:\python37\lib\site-packages\requests\sessions.py", line 622, in send
r = adapter.send(request, **kwargs)
File "c:\python37\lib\site-packages\requests\adapters.py", line 511, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='api.coursera.org', port=443): Max retries exceeded with url: /api/login/v3 (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1045)')))

C:\Users\Kishore Odugu>