Skip to content

CLI Web Scraper allows you to scrap data from different types of selectors. This system makes it possible to tailor data extraction to different site structures.

License

Notifications You must be signed in to change notification settings

codesalley/web-scraper-ruby

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

36 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

web-scraper-ruby

CLI web scraper, fetch pdf and course details from https://www.computer-pdf.com/ and https://www.classcentral.com/

Screenshot

Screenshot

Built With

  • Ruby

Getting Started

First, ensure that you have ruby installed on your local machine. and follow steps to run the program

Prerequisites

Ruby installed on your local machine. Terminal or similar to execute the program.

Gem libraries used

  • nokogiri
  • tty-prompt
  • HTTParty
  • open-uri
  • ruby_figlet
  • byebug

To install those libraries, simply run gem install [gem-name]

Install

  • clone project by running git clone https://github.com/juxsalley/web-scraper-ruby.git

  • cd web-scraper-ruby

  • install required gems bundle install

Usage

  • cd into the project by cd web-scraper-ruby
  • cd bin
  • Execute in terminal like so ruby scraper

To Test

  • After cloning the project run bundle install to install all required gems as instructed above
  • Then bundle exec rspec to run the test

Author

👤 Iddrisu salley

🤝 Contributing

Contributions, issues, and feature requests are welcome!

Feel free to check the issues page.

Show your support

Give a ⭐️ if you like this project!

📝 License

This project is MIT licensed.

About

CLI Web Scraper allows you to scrap data from different types of selectors. This system makes it possible to tailor data extraction to different site structures.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages