letterboxd does not work as an inserter

[frazzydee]

Describe the bug

I am running RatS on a headless machine with a clean installation of firefox.

To Reproduce
Steps to reproduce the behavior:

1. Have credentials for criticker & letterboxd in credentials.cfg
2. $ python3 transfer_ratings.py --source criticker --destination letterboxd

Expected behavior
Ratings successfully parsed from criticker (this happens OK), and inserted into letterboxd (this didn't happen)

Desktop (please complete the following information):

• Version: d18cd0a
• Platform OS: Debian 10.7 Buster
• Firefox version: Mozilla Firefox 78.6.1esr (from firefox -v)
• Geckodriver version: geckodriver 0.29.0 (cf6956a5ec8e 2021-01-14 10:31 +0200)
• Python version: Python 3.7.3
• PIP version: pip 18.1 from /usr/lib/python3/dist-packages/pip (python 3.7)
• Running in a virtualenv?: no
• Running in Docker?: no
• Executed command: python3 transfer_ratings.py --source criticker --destination letterboxd

Stacktrace

$ python3 transfer_ratings.py --source criticker --destination letterboxd
===== Criticker: Retrieving ratings XML
===== Criticker: saved 247 parsed movies to /home/faraaz/src/RatS/RatS/exports/20210126210139_Criticker.json
===== Letterboxd: performing loginTraceback (most recent call last):
  File "transfer_ratings.py", line 234, in <module>
    main()
  File "transfer_ratings.py", line 86, in main
    execute(args)
  File "transfer_ratings.py", line 136, in execute
    execute_inserting(args, movies, parser)
  File "transfer_ratings.py", line 153, in execute_inserting
    inserter = get_inserter_from_arg(destination)(args)
  File "/home/faraaz/src/RatS/RatS/letterboxd/letterboxd_ratings_inserter.py", line 24, in __init__
    super(LetterboxdRatingsInserter, self).__init__(Letterboxd(args), args)
  File "/home/faraaz/src/RatS/RatS/letterboxd/letterboxd_site.py", line 10, in __init__
    super(Letterboxd, self).__init__(args)
  File "/home/faraaz/src/RatS/RatS/base/base_site.py", line 36, in __init__
    self._init_browser()
  File "/home/faraaz/src/RatS/RatS/base/base_site.py", line 69, in _init_browser
    self.login()
  File "/home/faraaz/src/RatS/RatS/base/base_site.py", line 85, in login
    self._click_login_button()
  File "/home/faraaz/src/RatS/RatS/base/base_site.py", line 129, in _click_login_button
    login_button.click()
  File "/home/faraaz/.local/lib/python3.7/site-packages/selenium/webdriver/remote/webelement.py", line 80, in click
    self._execute(Command.CLICK_ELEMENT)
  File "/home/faraaz/.local/lib/python3.7/site-packages/selenium/webdriver/remote/webelement.py", line 633, in _execute
    return self._parent.execute(command, params)
  File "/home/faraaz/.local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/home/faraaz/.local/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.ElementClickInterceptedException: Message: Element <input class="button -action button-green" type="submit"> is not clickable at point (593,365) because another element <iframe id="zIframe_tyche_trendi_video" class="zeus_iframe" src="//cdn.playwire.com/bolt4/js/zeus/releases/5.1.2/frame/frame.html#id=tyche_trendi_video___pv=1"> obscures it

Geckodriver log

1611712899298	geckodriver	INFO	Listening on 127.0.0.1:40115
1611712901130	Marionette	WARN	TLS certificate errors will be ignored for this session
JavaScript error: https://www.criticker.com/js/javascript.js, line 33: ReferenceError: adblock is not defined
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
1611712914299	geckodriver	INFO	Listening on 127.0.0.1:47883
1611712916220	Marionette	WARN	TLS certificate errors will be ignored for this session

Additional context
I note from the stacktrace it appears that the login button may be getting blocked by a video. Letterboxd uses advertisement videos on its login page, and I don't have any adblock installed (since it is a headless machine with a clean firefox install).

[StegSchreck]

Hey @frazzydee,
Thanks for bringing this to my attention. I run a pi-hole in my local network which prevents me from running into this video you mentioned. But after deactivating that I could in fact reproduce your issue. I have created a fix for that.
Please let me know if this also works on your setup.

[frazzydee]

@StegSchreck, thank you very much for the fix.

Unfortunately I still get an error, below. Please let me know if I can provide any more information

faraaz@debian:~/src/RatS$ python3 transfer_ratings.py --source criticker --destination letterboxd
===== Criticker: Retrieving ratings XML
===== Criticker: saved 267 parsed movies to /home/faraaz/src/RatS/RatS/exports/20210211133807_Criticker.json
===== Letterboxd: performing loginTraceback (most recent call last):
  File "transfer_ratings.py", line 234, in <module>
    main()
  File "transfer_ratings.py", line 86, in main
    execute(args)
  File "transfer_ratings.py", line 136, in execute
    execute_inserting(args, movies, parser)
  File "transfer_ratings.py", line 153, in execute_inserting
    inserter = get_inserter_from_arg(destination)(args)
  File "/home/faraaz/src/RatS/RatS/letterboxd/letterboxd_ratings_inserter.py", line 24, in __init__
    super(LetterboxdRatingsInserter, self).__init__(Letterboxd(args), args)
  File "/home/faraaz/src/RatS/RatS/letterboxd/letterboxd_site.py", line 12, in __init__
    super(Letterboxd, self).__init__(args)
  File "/home/faraaz/src/RatS/RatS/base/base_site.py", line 36, in __init__
    self._init_browser()
  File "/home/faraaz/src/RatS/RatS/base/base_site.py", line 69, in _init_browser
    self.login()
  File "/home/faraaz/src/RatS/RatS/base/base_site.py", line 85, in login
    self._click_login_button()
  File "/home/faraaz/src/RatS/RatS/base/base_site.py", line 129, in _click_login_button
    login_button.click()
  File "/home/faraaz/.local/lib/python3.7/site-packages/selenium/webdriver/remote/webelement.py", line 80, in click
    self._execute(Command.CLICK_ELEMENT)
  File "/home/faraaz/.local/lib/python3.7/site-packages/selenium/webdriver/remote/webelement.py", line 633, in _execute
    return self._parent.execute(command, params)
  File "/home/faraaz/.local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/home/faraaz/.local/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.ElementClickInterceptedException: Message: Element <input class="button -action button-green" type="submit"> is not clickable at point (593,365) because another element <iframe id="zIframe_tyche_trendi_video" class="zeus_iframe" src="//cdn.playwire.com/bolt4/js/zeus/releases/4.3.40/frame/frame.html#id=tyche_trendi_video___pv=1"> obscures it

[StegSchreck]

Ah, it seems that the video didn't get dismissed properly in your case. I'll have another look.

[StegSchreck]

Hey @frazzydee
I have found a fix that worked for me. Sorry that I didn't have time to do this earlier. Could you verify that this works for you as well?