A Ruby gem that crawls Apple's developer documentation.
Scrapple aims to crawl Apple's developer documetation as listed here. Once complete and working, it'll be released as a Ruby gem. It's starting point was based off of NSHipster's NoOverviewAvailable.com.
Download this repo locally, then navigate to it on your machine:
$ cd location/of/scrappleInstall the gem file:
$ bundle installAnd also ensure you've got geckodriver and Firefox installed. To check if you've got geckodriver installed from Homebrew, run:
$ which geckodriverThat should give the location if you've got it.
Simply run ruby scrapple.rb. Ensure you're at that location on your machine as well (i.e. ~/scrapple/lib).
- Get all frameworks listed
- Iterate through all of the frameworks to...
- Parse the framework's symbols one by one
- Output it all to .json
- Create a
CLIfor it - Release it as a Ruby gem
How does this work?
Scrapple uses Nokogiri to load up the docs are parse them. For help with getting started with it, visit this tutorial or its own docs.
What's happening in
def parse(response, url:, data: {})?
Scrapple receives the initial set of HTML of Apple's docs. It's matched here:
response.css('div.card__content p')
Which gives us a list of all of the frameworks. This is the error prone part, as Apple could change their HTML of this page at anytime. As such, this will have to be updated a few times a year most likely. The framework object we get back is a Nokogiri Node.