This project aims to develop a Web Scraper using Ruby. It is a capstone project that marks the completion of Ruby module in Microverse technical curriculum. The full project specification can be found here.
- Ruby
- Nokogiri gem
- Linter (Rubocop)
- open-uri gem
- Rspec gem
To get a local copy up and running follow these simple steps.
- On the project GitHub page, navigate to the main page of the repository this page.
- Under the repository name, locate and click on a green button named 'Clone or download'.
- Copy the project URL as displayed.
- If you're running Windows Operating System, open your command prompt. On Linux, Open your terminal.
- Change the current working directory to the location where you want the cloned directory to be made. Leave as it is if the current location is where you want the project to be.
- Type
git clone, and then paste the URL you copied in Step 3.
$ git clone https://github.com/promise-J/web-scraper - Press Enter. Your local copy will be created.
Please Note that you must have github installed on your PC, this can be done here.
Since all the code is written using ruby, Ruby Runtime >= 1.9 is required to interpret the code. If you don't have ruby runtime already setup on your computer, follow the instruction for your specific operating system on the official installation guide. Internet Connection is a must, for successful scraping
After cloning on your computer, open the folder using your terminal/command prompt window and run
bin/main.rb
Web scraping, web harvesting, or web data extraction is data scraping used for extracting data from websites. Web scraping software may access the World Wide Web directly using the Hypertext Transfer Protocol, or through a web browser. read-more.
I chose to scrape this website because i have been greatly helped by his rich courses especially for the javascript before i got engaged with microverse.
- The app scrapes a webpage consisting brad traversy website for all his courses and their title too.
- It should display the result on the terminal.
Note: The app requires internet connection for successful scraping!
👤 Promise Johnson
- GitHub: promise-J
- Twitter: @twitterhandle
- LinkedIn: LinkedIn
Contributions, issues and feature requests are welcome!
- Fork the Project
- Create your Feature Branch (git checkout -b feature-scrapper) (if you may...)
- Commit your Changes (git commit -m 'Add some GreatImprovement')
- Push to the Branch (git push origin feature-scrapper)
- Open a Pull Request.
You can also check the issues page.
Give a ⭐️ if you like this project.
This project is MIT licensed.