Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added Google scholar scraper #1011

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

deepashri30
Copy link

Description

The changes made in this pull request should be briefly described.
The get_scholar_results function leverages the requests library to interact with the Google Scholar web interface and BeautifulSoup for parsing the HTML content. Here’s a breakdown of how this tool works:

Query Input and Parameter Setup:
Users can input their search query and specify the number of desired results.
The function constructs the query URL with appropriate parameters.

Fetching Results:
The tool sends an HTTP GET request to Google Scholar with the specified query parameters.
The response, containing the HTML of the search results page, is parsed using BeautifulSoup.

Parsing and Extracting Data:
The function extracts relevant information from each search result, including the title, link to the article, snippet, and publication information.
These details are organized into a structured format for easy readability.

Displaying Results:
The results are printed to the console in a user-friendly format, allowing for quick review and access to the original sources.

Resolves: [#962 ]

Checklist

Before submitting this pull request, kindly verify that the ensuing checkpoints have been reached.

  • Have you adhered to the repository's defined coding convention rules?
  • [] Have you updated the 'documentation.md' file with the method/function documentation?
  • Have you sent a message along with the result or response?
  • Have you used the try-catch technique?
  • [] Has the method/class been added to the documentation (md file)?

Screenshots

image

Uploading a screenshot illustrating the approach you developed for validation is required.

Additional Notes/Comments

Any additional remarks or suggestions concerning this pull request are welcome.


I certify that I have carried out the relevant checks and provided the requisite screenshot for validation by submitting this pull request.
I appreciate your contribution.

Contribution under GSSOC_24

Copy link
Member

@nikhil25803 nikhil25803 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few more changes needed -

  1. We do not use functional approach, instead we structure our code in form of Classes and add functionality as methods (you can check modules like IMDB and GitHub to see how codebase is formatted).
  2. Add docstring to those classes and methods.
  3. Add the modules and methods you created in dev-documentation.md file

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants