JavaScript
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
bin
example
lib
test
.gitignore
.travis.yml
LICENSE
README.md
index.js
package-lock.json
package.json
repl.js

README.md

Build Status Coverage Status

pickup - transform feeds

The pickup Node package provides a Transform stream from RSS 2.0, including iTunes namespace extensions, and Atom 1.0 formatted XML to newline separated JSON strings or objects.

Usage

Command-line

If you haven't yet, first install json—a command for working with JSON on the command-line:

$ npm install -g json

Now you can pipe pickup to json:

$ export URL=www.newyorker.com/feed/posts
$ curl -sS $URL | pickup | json -g

Or, for example, to filter the titles:

$ curl -sS $URL | pickup | json -ga title

Library

Transforming from stdin to stdout

const pickup = require('pickup')

process.stdin
  .pipe(pickup())
  .pipe(process.stdout)

You can run this example from the command-line:

$ curl -sS $URL | node example/stdin.js | json -g

Proxy server

const http = require('http')
const pickup = require('pickup')

http.createServer((req, res) => {
  http.get('http:/'.concat(req.url), (feed) => {
    feed.pipe(pickup()).pipe(res)
  })
}).listen(8080)

To try the proxy server:

$ node example/proxy.js &
$ curl -sS http://localhost:8080/$URL | json -ga title

Types

str()

This can either be a String(), null, or undefined.

opts()

The options Object is passed to the Transform stream constructor.

pickup uses following additional options:

  • eventMode Boolean defaults to false, if true readable state buffers are not filled and no 'data', but 'feed' and 'entry' events are emitted.

  • charset str() | 'UTF-8' | 'ISO-8859-1' An optional string to specify the encoding of input data. In the common use case you received this string in the headers of your HTTP response before you began parsing. If you, not so commonly, cannot provide the encoding upfront, pickup tries to detect the encoding, and eventually defaults to 'UTF-8'. The charset option is corresponding to the optional charset MIME type parameter found in Content-Type HTTP headers. It's OK to pass any string, pickup will fall back on 'UTF-8' when confused.

url()

An undefined property, not populated by the parser, to identify feeds and entries in upstream systems, without prompting V8 to create new hidden classes.

feed()

  • author str()
  • copyright str()
  • id str()
  • image str()
  • language str()
  • link str()
  • originalURL url()
  • payment str()
  • subtitle str()
  • summary str()
  • title str()
  • ttl str()
  • updated str()
  • url url()

enclosure()

  • href str()
  • length str()
  • type str()

entry()

  • author str()
  • duration str()
  • enclosure enclosure() or undefined
  • id str()
  • image str()
  • link str()
  • originalURL url()
  • subtitle str()
  • summary str()
  • title str()
  • updated str()
  • url url()

Event:'feed'

feed()

Emitted when the meta information of the feed gets available.

Event:'entry'

entry()

Emitted for each entry.

Exports

pickup(opts())

pickup exports a function that returns a Transform stream which emits newline separated JSON strings, in objectMode the 'data' event contains entry() or feed() objects. As per XML's structure the last 'data' event usually contains the feed() object. In eventMode neither 'readable' nor 'data' events are emitted, instead 'feed' and 'entry' events are fired.

Installation

With npm, do:

$ npm install pickup

To use the CLI (as above):

$ npm install -g pickup

Contribute

Please create an issue if you encounter a feed that pickup fails to parse.

License

MIT license