Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TimeoutError: waiting for selector "#login_form" failed: timeout 30000ms exceeded #13

Closed
wootwoot1234 opened this issue Nov 7, 2020 · 10 comments · Fixed by #14
Closed

TimeoutError: waiting for selector "#login_form" failed: timeout 30000ms exceeded #13

wootwoot1234 opened this issue Nov 7, 2020 · 10 comments · Fixed by #14
Assignees
@kaanyagci
Labels
bug Something isn't working good first issue Good for newcomers

Comments

@wootwoot1234
Copy link

wootwoot1234 commented Nov 7, 2020 •
edited
Loading

Hello.

I'm excited to try out this tool. I just installed it and ran it but got this error:

(node:47502) UnhandledPromiseRejectionWarning: TimeoutError: waiting for selector "#login_form" failed: timeout 30000ms exceeded.

This is the command I used:

fgps --group-ids ##########

I then ran it with the --headful param and can see the browser open and load the facebook.com but then doesn't fill in the username/password.
@kaanyagci
Copy link
Contributor

Hi there,

Apologies for the late response.

Did you initialize the scraper with fgps init ?

We will take a look as soon as possible.

@kaanyagci kaanyagci added bug Something isn't working good first issue Good for newcomers labels Dec 15, 2020
@kaanyagci kaanyagci self-assigned this Dec 15, 2020
@wootwoot1234
Copy link
Author

Thanks for responding. Yeah, I ran the init.

I think the issue might be with the selectors? I don't have a "#login_form". So I changed it to this:

  'login_form': {
    'email': '#email',
    'password': '#pass',
    'submit': '#u_0_b',
    'parent': '#u_0_a',
  }

But that didn't fix it so I'm not sure what's up. Here is the HTMl that I'm seeing:

<form class="_featuredLogin__formContainer" data-testid="royal_login_form" action="/login/?privacy_mutation_token=eyJ0eXBlIjowLCJjcmVhdGlvbl90sdvr6v6ssv6jY0Nzg0sdfgRRsdfgWxsc2l0ZV9pZCI6MzgxMjI5MDc5NTc1OTQ2fQ%3D%3D" method="post" onsubmit="" id="u_0_a">
    <input type="hidden" name="jazoest" value="2884" autocomplete="off"><input type="hidden" name="lsd" value="AVpha3BE7Do" autocomplete="off">
    <div>
        <div class="_6lux"><input type="text" class="inputtext _55r1 _6luy" name="email" id="email" data-testid="royal_email" placeholder="Email or Phone Number" autofocus="1" aria-label="Email or Phone Number"></div>
        <div class="_6lux"><input type="password" class="inputtext _55r1 _6luy" name="pass" id="pass" data-testid="royal_pass" placeholder="Password" aria-label="Password"></div>
    </div>
    <input type="hidden" autocomplete="off" name="login_source" value="comet_headerless_login"><input type="hidden" autocomplete="off" name="next" value="">
    <div class="_6ltg"><button value="1" class="_42ft _4jy0 _6lth _4jy6 _4jy1 selected _51sy" name="login" data-testid="royal_login_button" type="submit" id="u_0_b">Log In</button></div>
    <div class="_6ltj"><a href="https://www.facebook.com/recover/initiate/?ars=facebook_login&amp;privacy_mutation_token=eyJ0eXBlIjowLCJsdfgssdf64r5rs7l90aW1lIjoxNjA4MjY0Nzg0LCJjYWxsc2l0ZV9pZCI6Mzgx5NTc1OTQ2fQ%3D%3D">Forgot Password?</a></div>
    <div class="_8icz"></div>
    <div class="_6ltg"><a role="button" class="_42ft _4jy0 _6lti _4jy6 _4jy2 selected _51sy" href="#" ajaxify="/reg/spotlight/" id="u_0_2" data-testid="open-registration-form-button" rel="async">Create New Account</a></div>
</form>

@kaanyagci
Copy link
Contributor

Seems like the issue is related to the cookie banner.

Will make the patch to accept all cookies as soon as possible.
image

@kaanyagci kaanyagci mentioned this issue Dec 18, 2020
Merged
@kaanyagci kaanyagci linked a pull request Dec 18, 2020 that will close this issue
Facebook login issue fix #14
Merged
@kaanyagci
Copy link
Contributor

kaanyagci commented Dec 18, 2020 •
edited
Loading

@wootwoot1234 just updated for accepting all cookies and new login selectors. But I need to update selectors for the new Facebook UI. I'll try to push a new version before the end of the day.

@wootwoot1234
Copy link
Author

wootwoot1234 commented Dec 18, 2020 via email

@wootwoot1234
Copy link
Author

I downloaded the new code. I'm not seeing the cookie pop so I commented out that code because otherwise, it failed waiting for it.

Then it got stuck on this line:

https://github.com/Makepad-fr/facebook-group-posts-scraper/blob/39418c3452831d170e18de26c262ecf3c8bfcd8a/src/index.js#L358

So I commented out that line too just to see what would happen and got this error.


(node:7715) UnhandledPromiseRejectionWarning: Error: net::ERR_ABORTED at https://m.facebook.com/groups/InstantPotCommunity/
    at navigate (/Users/tom/Downloads/facebook-group-posts-scraper-facebook-login-issue-fix/node_modules/puppeteer/lib/FrameManager.js:120:37)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
    at async FrameManager.navigateFrame (/Users/tom/Downloads/facebook-group-posts-scraper-facebook-login-issue-fix/node_modules/puppeteer/lib/FrameManager.js:94:17)
    at async Frame.goto (/Users/tom/Downloads/facebook-group-posts-scraper-facebook-login-issue-fix/node_modules/puppeteer/lib/FrameManager.js:406:12)
    at async Page.goto (/Users/tom/Downloads/facebook-group-posts-scraper-facebook-login-issue-fix/node_modules/puppeteer/lib/Page.js:672:12)
    at async facebookMain (/Users/tom/Downloads/facebook-group-posts-scraper-facebook-login-issue-fix/src/index.js:433:3)
    at async main (/Users/tom/Downloads/facebook-group-posts-scraper-facebook-login-issue-fix/src/index.js:618:5)
  -- ASYNC --
    at Frame.<anonymous> (/Users/tom/Downloads/facebook-group-posts-scraper-facebook-login-issue-fix/node_modules/puppeteer/lib/helper.js:111:15)
    at Page.goto (/Users/tom/Downloads/facebook-group-posts-scraper-facebook-login-issue-fix/node_modules/puppeteer/lib/Page.js:672:49)
    at Page.<anonymous> (/Users/tom/Downloads/facebook-group-posts-scraper-facebook-login-issue-fix/node_modules/puppeteer/lib/helper.js:112:23)
    at facebookMain (/Users/tom/Downloads/facebook-group-posts-scraper-facebook-login-issue-fix/src/index.js:433:14)
    at main (/Users/tom/Downloads/facebook-group-posts-scraper-facebook-login-issue-fix/src/index.js:618:11)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
(Use `node --trace-warnings ...` to show where the warning was created)
(node:7715) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:7715) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

@kaanyagci
Copy link
Contributor

@wootwoot1234 This's just fixed. Now there are only two issues:

  • On headless mode, the Log In button is not found
  • Both on headless and headful modes, post content has ...More text, which should be replaced with an empty string.

I'm still working on it

@wootwoot1234
Copy link
Author

Ok, great!

I'm still getting this error but maybe it can be fixed by wrapping it in a try{} catch(e){}? That way if you don't have the cookie banner it continues?


(node:9742) UnhandledPromiseRejectionWarning: TimeoutError: waiting for XPath "//button[@data-cookiebanner="accept_button"]" failed: timeout 30000ms exceeded
    at new WaitTask (/Users/tom/Downloads/facebook-group-posts-scraper-facebook-login-issue-fix 2/node_modules/puppeteer/lib/DOMWorld.js:549:28)
    at DOMWorld._waitForSelectorOrXPath (/Users/tom/Downloads/facebook-group-posts-scraper-facebook-login-issue-fix 2/node_modules/puppeteer/lib/DOMWorld.js:478:22)
    at DOMWorld.waitForXPath (/Users/tom/Downloads/facebook-group-posts-scraper-facebook-login-issue-fix 2/node_modules/puppeteer/lib/DOMWorld.js:441:17)
    at Frame.waitForXPath (/Users/tom/Downloads/facebook-group-posts-scraper-facebook-login-issue-fix 2/node_modules/puppeteer/lib/FrameManager.js:642:47)
    at Frame.<anonymous> (/Users/tom/Downloads/facebook-group-posts-scraper-facebook-login-issue-fix 2/node_modules/puppeteer/lib/helper.js:112:23)
    at Page.waitForXPath (/Users/tom/Downloads/facebook-group-posts-scraper-facebook-login-issue-fix 2/node_modules/puppeteer/lib/Page.js:1131:29)
    at facebookLogIn (/Users/tom/Downloads/facebook-group-posts-scraper-facebook-login-issue-fix 2/src/index.js:331:14)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
    at async main (/Users/tom/Downloads/facebook-group-posts-scraper-facebook-login-issue-fix 2/src/index.js:600:10)
(Use `node --trace-warnings ...` to show where the warning was created)
(node:9742) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:9742) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

@wootwoot1234
Copy link
Author

wootwoot1234 commented Dec 18, 2020 •
edited
Loading

After I added the try/catch, it's working! Thanks!

I have a feature request. It would be nice to be able to limit it to a given number of posts. I don't need it to scrape all the posts and for big groups, it's going to take along time. :)

@kaanyagci
Copy link
Contributor

kaanyagci commented Dec 18, 2020 via email •
edited
Loading

@gohjiayi gohjiayi mentioned this issue Mar 8, 2021
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kaanyagci kaanyagci

Labels
bug Something isn't working good first issue Good for newcomers
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Facebook login issue fix Makepad-fr/fbjs
2 participants
@wootwoot1234 @kaanyagci