Vine Screen Scrape

A tool for scraping public data from Vine. It can scrape any posts that a non-logged-in user can see (which is the full history of any public account). See lib/response.schema.json for the format of the response.

Example

CLI

The CLI operates entirely over STDOUT, and will output posts as it scrapes them. The following example is truncated because the output of the real command is obviously very long... it will end with a closing bracket (making it valid JSON) if you see the full output.

$ vine-screen-scrape -u 969179904094908416
[{"username":"969179904094908416","_id":"1220071062235422720","loop":9,"comment":0,"repost":0,"like":0,"time":1433861823,"text":"test2"},
{"username":"969179904094908416","_id":"1220070436260515840","loop":9,"comment":0,"repost":0,"like":0,"time":1433861673,"text":"test"}]

By default, there is 1 line per post, making it easy to pipe into other tools. The following example uses wc -l to count how many posts are returned. As you can see, I don't post much.

$ vine-screen-scrape -u 969179904094908416 | wc -l
2

JavaScript Module

The following example is in CoffeeScript.

VinePosts = require 'vine-screen-scrape'

# create the stream
streamOfPosts = new VinePosts('969179904094908416')

# do something interesting with the stream
streamOfPosts.on('readable', ->
  # since it's an object-mode stream, we get objects from it and don't need to
  # parse JSON or anything.
  post = streamOfPosts.read()

  # the time field is represented in UNIX time
  time = new Date(post.time * 1000)

  # output something like "slang800's post from 4/5/2015 got 1 like(s), and 0
  # comment(s)"
  console.log "slang800's post from #{time.toLocaleDateString()} got
  #{post.like} like(s), and #{post.comment} comment(s)"
)

The following example is the same as the last one, but in JavaScript.

var scrape, streamOfPosts;
scrape = require('vine-screen-scrape');

streamOfPosts = new VinePosts('969179904094908416');
streamOfPosts.on('readable', function() {
  var post, time;
  post = streamOfPosts.read();
  time = new Date(post.time * 1000);
  console.log([
    "slang800's post from ",
    time.toLocaleDateString(),
    " got ",
    post.like,
    " like(s), and ",
    post.comment,
    " comment(s)"
  ].join(''));
});

Why?

Vine doesn't provide an open, structured, and machine readable API, so, we're forced to scrape their user-facing site.

Caveats

• This is probably against the Vine TOS, so don't use it if that sort of thing worries you.
• Whenever Vine updates certain parts of their front-end this scraper will need to be updated to support the new API.
• You can't scrape protected accounts (cause it's not public duh).