Skip to content
Commits on May 25, 2015
  1. @manojlds

    Update version to 3.5.16

    manojlds committed
  2. @manojlds

    Merge pull request #7 from ind9/add-proxy-authorization

    manojlds committed
    Adding proxy authorization header
  3. @manojlds
Commits on May 11, 2015
  1. @codingnirvana

    Bump up artifact version

    codingnirvana committed
  2. @codingnirvana
Commits on Apr 8, 2015
  1. @ssesha

    Merge pull request #6 from ind9/CookieCrawlFix

    ssesha committed
    Fix for cookie-based crawling
Commits on Apr 7, 2015
  1. @manojlds

    Set cookie path

    manojlds committed
  2. @manojlds
  3. @addnab

    Fix for cookie-based crawling

    addnab committed
Commits on Mar 19, 2015
  1. @manojlds

    Bump up version to 3.5.13-indix

    manojlds committed
  2. @manojlds

    Trusting the ssl certificate by default

    manojlds committed
    Back ported from main crawler4j
    
    https://github.com/yasserg/crawler4j/blob/70fe6f1942427d2c054b50ad8a924b
    0e6c4beba3/src/main/java/edu/uci/ics/crawler4j/fetcher/PageFetcher.java#
    L91
Commits on Dec 11, 2014
  1. @vinothkr

    Updating version

    vinothkr committed
  2. @vinothkr

    Ideally we should just reuse the client. But the crawler does need a …

    vinothkr committed
    …different site-config (may be change the contract). We can atleast be good citizens and shut it down, it may reduce number of CLOSE_WAITs probably
Commits on Dec 10, 2014
  1. @ashwanthkumar

    bumping the crawler4j version

    ashwanthkumar committed
    contains the workaround for meta refresh property
  2. @ashwanthkumar

    workaround for extracting meta refresh property

    ashwanthkumar committed
    for some reason TIKA's HTMLParser is capturing http-equiv as name property in meta tags. Upgraded to latest TIKA that didn't help either.
    
    Added a test for now, will need to look into it
Commits on Dec 9, 2014
  1. @ashwanthkumar
  2. @ashwanthkumar
  3. @ashwanthkumar
Commits on Sep 6, 2014
  1. @phoenix24

    bumped up crawler4j version.

    phoenix24 committed
  2. @phoenix24
Commits on Aug 19, 2014
  1. @sattiwari

    Updated version

    sattiwari committed
  2. @sattiwari

    Revert "Revert "removed thread-pools/pools-client-manager from page-f…

    sattiwari committed
    …etcher.""
    
    This reverts commit 03072ac.
Commits on Aug 13, 2014
  1. @sattiwari

    Updating version

    sattiwari committed
  2. @sattiwari
Commits on Jul 10, 2014
  1. @sattiwari

    Updating version

    sattiwari committed
  2. @sattiwari

    Add link tag to end element

    sattiwari committed
  3. @sattiwari
  4. @sattiwari

    Updating version

    sattiwari committed
  5. @sattiwari
  6. @sattiwari
Commits on Jul 9, 2014
  1. @sattiwari

    Updaing version

    sattiwari committed
  2. @sattiwari
Commits on May 14, 2014
  1. @ashwanthkumar
  2. @ashwanthkumar
Commits on Apr 30, 2014
  1. @phoenix24

    Merge pull request #1 from ind9/refactoring

    phoenix24 committed
    removed thread-pools/pools-client-manager from page-fetcher.
Something went wrong with that request. Please try again.