Permalink
Switch branches/tags
Nothing to show
Commits on Oct 20, 2009
  1. add ":allowed_urls", ":skip_urls" options

    mislav committed Oct 18, 2009
    Used to specify additional URL patterns that are permitted or blocked.
    API change: `skip_links_like` patterns are matched against the whole
    URI, not just the path + query part as it was before
  2. default ":traverse_up" to true

    mislav committed Oct 18, 2009
    references 0d498a6
Commits on Oct 5, 2009
  1. update readme

    mislav committed Oct 5, 2009
  2. no more global `Anemone.options`; options hash is one per crawl

    mislav committed Oct 3, 2009
    Anemone::HTTP is now a class that's initialized with :user_agent and :obey_robots_txt
Commits on Oct 1, 2009
  1. ensure that hyperlinks that can't be followed (non-http) are skipped

    mislav committed Oct 1, 2009
    Examples:
      - #fragment
      - mailto:mislav@example.com
      - javascript:void()
      - ftp://user:password@example.com/
      - tel:1-408-555-5555
      - sms:1-408-555-1212
  2. tweak specs

    mislav committed Oct 1, 2009
  3. simplify tentacle loop

    mislav committed Sep 30, 2009
Commits on Sep 30, 2009
Commits on Sep 7, 2009
  1. up version to 0.2.0

    chriskite committed Sep 7, 2009
  2. update Page depth doc

    chriskite committed Sep 7, 2009
  3. send referer correctly

    chriskite committed Sep 7, 2009
  4. fix PageHash#has_key?

    chriskite committed Sep 7, 2009
Commits on Sep 1, 2009
  1. option to obey robots.txt, and a spec

    Chris Kite
    Chris Kite committed Sep 1, 2009
Commits on Aug 11, 2009
  1. up version to 0.1.2

    chriskite committed Aug 11, 2009
Commits on Jul 23, 2009
  1. page specs

    chriskite committed Jul 23, 2009