Permalink
Switch branches/tags
Nothing to show
Commits on Oct 20, 2009
  1. add ":allowed_urls", ":skip_urls" options

    Used to specify additional URL patterns that are permitted or blocked.
    API change: `skip_links_like` patterns are matched against the whole
    URI, not just the path + query part as it was before
    committed Oct 18, 2009
  2. default ":traverse_up" to true

    references 0d498a6
    committed Oct 18, 2009
Commits on Oct 5, 2009
  1. update readme

    committed Oct 5, 2009
  2. no more global `Anemone.options`; options hash is one per crawl

    Anemone::HTTP is now a class that's initialized with :user_agent and :obey_robots_txt
    committed Oct 3, 2009
Commits on Oct 1, 2009
  1. ensure that hyperlinks that can't be followed (non-http) are skipped

    Examples:
      - #fragment
      - mailto:mislav@example.com
      - javascript:void()
      - ftp://user:password@example.com/
      - tel:1-408-555-5555
      - sms:1-408-555-1212
    committed Oct 1, 2009
  2. tweak specs

    committed Oct 1, 2009
  3. simplify tentacle loop

    committed Sep 30, 2009
Commits on Sep 30, 2009
Commits on Sep 7, 2009
  1. up version to 0.2.0

    chriskite committed Sep 7, 2009
  2. update Page depth doc

    chriskite committed Sep 7, 2009
  3. send referer correctly

    chriskite committed Sep 7, 2009
  4. fix PageHash#has_key?

    chriskite committed Sep 7, 2009
Commits on Sep 1, 2009
  1. option to obey robots.txt, and a spec

    Chris Kite committed Sep 1, 2009
Commits on Aug 11, 2009
  1. up version to 0.1.2

    chriskite committed Aug 11, 2009
Commits on Jul 23, 2009
  1. page specs

    chriskite committed Jul 23, 2009