Skip to content
Commits on Apr 18, 2016
  1. Report pickle errors, and add a -N option to ignore

    and re-initialize the cache in that case.
    committed Apr 17, 2016
Commits on Apr 17, 2016
  1. Don't run feedme if there's a process already running.

    Add some optional debug lines near the end that can be switched on
    in a case where feedme doesn't run and we don't know why
    (in this case, it was a pickle failure).
    committed Apr 16, 2016
Commits on Feb 25, 2016
  1. Include continue_on_timeout in the ConfigParser options,

    so we'll properly handle sites using the default (don't continue,
    skip to next site).
    When receiving a sigint while running non-interactively, catch
    exceptions.EOFError and act as if the user typed 'n', skip to
    next site.
    Also remove some spurious debug chatter regarding time comparisons
    and when to feed.
    committed Feb 24, 2016
Commits on Jan 3, 2016
Commits on Sep 30, 2015
Commits on Sep 25, 2015
  1. Add lots of documentation on config file options (new -c arg).

    Make a single VersionString live in feedmeparser module.
    Bump version to 1.0b1.
    committed Sep 25, 2015
Commits on Sep 14, 2015
  1. Add a special .EOF. sequence to the end of MANIFEST

    so the fetcher can tell when it's fully ready to download.
    Output plaintext, not html, from urlrss.
    committed Sep 13, 2015
Commits on Sep 10, 2015
  1. Remove a too-verbose message

    committed Sep 9, 2015
Commits on Sep 9, 2015
  1. If nocache is set, always save an index file

    even if we didn't fetch any content,
    so feeds like Xtraurls will warn us of failed URLs.
    committed Sep 8, 2015
Commits on Aug 27, 2015
Commits on Aug 9, 2015
  1. Print page_start/end info to stderr, not stdout,

    so it gets saved in the log in debug mode.
    committed Aug 9, 2015
Commits on Aug 4, 2015
  1. Handle gzipped http

    committed Aug 3, 2015
Commits on Jul 31, 2015
Commits on Jul 26, 2015
  1. Allow for cookies in the request: some sites, notably nytimes.com,

    degrade to an infinite redirect loop if cookies aren't enabled.
    committed Jul 26, 2015
Commits on Jun 2, 2015
  1. Minor changes

    committed Jun 1, 2015
Commits on Jun 1, 2015
  1. Save a MANIFEST file with a list of all filenames written,

    to make it easier for a downloader to fetch them.
    committed May 31, 2015
Commits on May 26, 2015
  1. Remove entire <head> section -- which means we'll

    miss meta tags, but the important thing is that we'll
    skip foreign style tags that mess up our rendering and
    cause a lot of unwanted data downloads. Try to notice
    any meta charset tags inside the old head and save them.
    committed May 25, 2015
Commits on May 21, 2015
  1. Add Los Alamos Daily Post

    committed May 20, 2015
Commits on May 1, 2015
  1. Try to guard against errors from bad unicode characters in URLs.

    Those seem to be showing up on Longreads, in particular.
    committed Apr 30, 2015
Commits on Mar 5, 2015
Commits on Feb 28, 2015
  1. Obey base href on pages that have it, for image rewriting.

    This may solve a lot of the images that weren't downloading.
    Guard against images that don't download; don't bomb out with
    errors, and rewrite the URL to an absolute one so the image
    will at least show if there's a live net connection.
    committed Feb 27, 2015
  2. Strip RSS content before deciding it's blank.

    Los Alamos Daily Post has whitespace-only content.
    committed Feb 27, 2015
Commits on Feb 27, 2015
  1. Rewrite img tags in the indexstr to use locally fetched images.

    Include a note in the indexstr for pages with blank content.
    committed Feb 27, 2015
Commits on Feb 26, 2015
  1. Add User-Agent to every urllib2.Request we make,

    including images and referred requests.
    committed Feb 25, 2015
Commits on Feb 25, 2015
  1. Set the User-Agent

    committed Feb 24, 2015
Commits on Nov 18, 2014
Commits on Oct 31, 2014
  1. Set a default socket timeout of 100 seconds.

    This is the only way to set a feedparser timeout for the RSS URLs.
    committed Oct 30, 2014
  2. Add new continue_on_timeout config parameter

    to control whether a timeout means we skip to the next site,
    or to the next story (true means stay on site, skip to next story).
    committed Oct 30, 2014
  3. Add a timeout of 100 seconds on stories.

    If the timeout is exceeded, we'll note that in the log
    and skip the rest of the site, assuming that the site is broken.
    (Hmm, but this may be wrong with Xtraurls or sites like
    Longreads where stories come from different sources.)
    committed Oct 30, 2014
Something went wrong with that request. Please try again.