Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

request-html would might be a good and fast alternative for selenium #84

Closed
abhinavm24 opened this issue Feb 20, 2024 · 3 comments
Closed

Comments

@abhinavm24
Copy link

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

@anilabhadatta
Copy link
Owner

@abhinavm24 not possible at the moment, reason:

  1. session is stored locally using chrome user-data-dir, manages login system and stays logged in forever if the account is not authenticated in another device.
  2. the browser also helps in calling the api's that are auth protected so only logged in user can request and get an response
  3. single-file is integrated and it is well designed to scrape HTML data without any data loss and also removes js code, also the images are saved in base64 format so they are not linked to educative's servers.
  4. request-html also does the same thing by loading a browser and generating a response but doesn't help in obtaining the first 2 objectives that are mandatory
  5. also request-html is only created for python3.6, and not sure if the response if generated as good as singlefile.

@anilabhadatta
Copy link
Owner

@abhinavm24 major updates were pushed, do check out.
Ucdriver is integrated now

@abhinavm24
Copy link
Author

great work @anilabhadatta, will get scrapping then

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants