ruby CLI to scrape hrefs from single webpages
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
bin
lib
.gitignore
.travis.yml
Gemfile
Gemfile.lock
LICENSE
README.md
href-scraper.gemspec

README.md

href-scraper

Build Status Gem Version Gem

A ruby gem to get all the hrefs from a single URL(export as json)

Installation

Add this line to your application's Gemfile:

gem 'href_scraper'

And then execute:

$ bundle

Or install it yourself as:

$ gem install href_scraper

Usage

❯ href_scraper --url https://github.com
[
    "https://github.com/",
    "https://help.github.com/terms",
    "https://help.github.com/privacy",
    "https://enterprise.github.com/trial",
    "https://enterprise.github.com/contact",
    "https://help.github.com/terms",
    "https://help.github.com/privacy",
    "https://atom.io",
    "http://electron.atom.io/",
    "https://desktop.github.com/",
    "https://developer.github.com",
    "https://education.github.com/",
    "https://partner.github.com/",
    "https://community.github.com/",
    "https://github.com/about",
    "https://blog.github.com",
    "https://shop.github.com",
    "https://github.com/contact",
    "https://github.community",
    "https://help.github.com",
    "https://status.github.com/",
    "https://help.github.com/articles/github-security/",
    "https://services.github.com/"
]