Issue downloading content

[mikemastercorp]

Hello there. I am testing the downloaded trying to access a course I've joined recently, however I am unable to be connected to Internet all the time so needed to download it on my PC.

I cloned your GIT repo to my local hard drive, started the latest version (0.16.1), provided the course link::

https://www.linkedin.com/learning/paths/become-a-windows-system-administrator-server-2012-r2

Set the browser agent to various options, but am getting the same error:

`DevTools listening on ws://127.0.0.1:57256/devtools/browser/dd30e0f6-fda8-4501-a587-3a7493f861ab
###############################################

LinkedIn Learning Download

@author r00tme 02/12/2019

@Version: GUI 0.16.1

##############################################

[*] Trying to login on LinkedInException in thread Thread-1:
Traceback (most recent call last):
File "threading.py", line 917, in _bootstrap_inner
File "threading.py", line 865, in run
File "gui.py", line 431, in downloader
File "site-packages\selenium\webdriver\remote\webdriver.py", line 360, in find_element_by_id
File "site-packages\selenium\webdriver\remote\webdriver.py", line 978, in find_element
File "site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
File "site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"[id="username"]"}
(Session info: chrome=79.0.3945.88)
`
Any help would be greatly appreciated.

FYI, I chose not to download the subtitles but wanted to have the exercise files so marked that option. I've read that you are planing on adding a troubleshooting part to the script but as it is not up yet, just tried to provide you with the maximum info I can get. Should you still need some extra tests or else, I am staying fully at your disposal ;)

[mikemastercorp]

Funny thing, after having another unsuccessful download of a course, it started working and got it all downloaded:
https://www.linkedin.com/learning/resume-makeover/

So I attempted to download the same course again and it gave the same error as before.

Here is another example of a course that shows the same issue even after I unchecked the option for automatic login to linked in and set every timing to 15 seconds delay. I logged on into the newly opened window and pointed to the course manually but the error still persists:

DevTools listening on ws://127.0.0.1:60701/devtools/browser/94a73e4b-ec79-4044-9624-9510594e53e0
###############################################

LinkedIn Learning Download

@author r00tme 02/12/2019

@Version: GUI 0.16.1

##############################################

[*] Working on course https://www.linkedin.com/learning/react-creating-and-hosting-a-full-stack-site/Exception in thread Thread-1:
Traceback (most recent call last):
File "threading.py", line 917, in _bootstrap_inner
File "threading.py", line 865, in run
File "gui.py", line 479, in downloader
File "site-packages\selenium\webdriver\remote\webdriver.py", line 564, in find_element_by_class_name
File "site-packages\selenium\webdriver\remote\webdriver.py", line 978, in find_element
File "site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
File "site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":".course-info__difficulty"}
(Session info: chrome=79.0.3945.88)

[fastmarketo]

I think they have made restrictions more intense. Previously this used to come after downloading 50-70 courses, but now this has come only at 5th course that too when I was downloading after 2-3 days gap.

[mikemastercorp]

Yeah. I got under the same fire last night, and I was able to download only 3 or 4 courses before I got hit. This morning I attempted several times to browse the /learning/ and was blocked for a while. Then I was asked nicely by LinkedIn to change my password since my account has been compromised and soon after my URL got unblocked.

I believe that I will have to manually download the courses by viewing the page source and downloading the video files directly to avoid any issues and still be able to get a few extra certifications that have weight nowadays for LinkedIn Recruitment...

The earlier mentioned error was (as suspected) caused by the login process. To pass around it, I removed my credentials and set the login time to 30 seconds which allowed me to manually login with the appropriate profile and wait for the script to do the rest.

[fastmarketo]

https://docs.microsoft.com/en-us/linkedin/shared/api-guide/concepts/rate-limits?context=linkedin/context

But now this limit is getting applicable after 4-5 courses only. Previously it was 50+courses a day

[r00tmebaby]

@mikemastercorp, This is exactly what you have to do if the automatic login process does not work. I have mentioned that many times that the delay options is just for this reason, so people with organisations and those who struggle to login using that time to log in manually and then restart the program.

I can not keep the automatic login updated, because LinkedIn is a big data source website and their developers work 24/7 to make sure that it's all secured and harder to crack. They simply can not stop bot like this one because it behaves like every normal person, clicking on the links.
It does not directly interact with the forms using remote requests, so it's hard to tell that it's not a normal user. The only thing that they can do is to limit the total amount of possible requests for a certain time.

The problem is that they have requests limitation for 24 hours. If you exceed these requests, you will get that error for another I think 12 hours, not really sure. Which is worst, the temporary ban is bind to the account not to particular IP, so using VPN or proxy will not helo you to continue downloading. The only option is to wait until the temporary ban expires or log in with another account.

For a reference have a look at this link point 1.4 https://legal.linkedin.com/api-terms-of-use where it says that no more than 250 000 calls per day are allowed. It's hard to tell what is their algorithm to calculate those requests, and which one of them are treated as a call, requests or just clicks we can only guess.

My advice is to try to increase the download time between the videos (here these options may come in favour) and see if there is an improvement.

[fastmarketo]

But I think recently they have put a severe ban from my a/c they allowed only 4 courses and then from my sisters account just 7 courses. Previously 50-70 courses were common in 24 hours span.

I am facing this for 2 days.

As per IST(Indian standard time), the ban gets lifted at 5:42 AM around.

[fastmarketo]

Ban is released 12:00 AM UTC time.

[mikemastercorp]

Well, considering the fact that LinkedIn has "swallowed" my favorite training site Lynda.com and the courses in both are theoretically the same, I will see how does Lynda allow the download even though I can suspect that the policies would be identical since both now would be managed by the same development team.

One thing I am thinking to try is to download 5-6 courses through their Learning application for offline use. If I am able to download more than 4-5 courses, than there is some extra mechanism to identify which one is what and then to allow or disallow it. Based on the fact that I still have some needs of certifications to pass for the fun of having them added to the profile, I will test the ability at the end of my free 30 days trial.

Funny enough, Lynda.com is allowing you to download your courses to your phone and Desktop where so far LinkedIn ONLY ALLOW the download on Android/iOS and it is not useful at all for me to watch the videos even on my Huwawei P20 Pro. So unless one has a tablet, the video download could basically only be listened to ;)