Parse web pages using CSS query selectors
HTML JavaScript CSS
Switch branches/tags
Nothing to show
Clone or download
dinubs Merge pull request #18 from kausar007/master
Add curl example to readme
Latest commit ce8065c Nov 26, 2016
Permalink
Failed to load latest commit information.
controllers update ruby example on homepage and readme, add in replace for single… May 7, 2016
public small style changes Jun 6, 2016
services add in ability to get all html, as per issue #13 Jun 13, 2016
views remove irrelevant routes, change http in urls to https Sep 25, 2016
.DS_Store Merge remote-tracking branch 'origin/master' Sep 25, 2016
.gitignore add in gitignore and views and styles Apr 27, 2016
LICENSE add in GPL license, fixes issue #3 Apr 28, 2016
Procfile
README.md Add curl example to readme Nov 24, 2016
index.js maybe this will fix it May 31, 2016
package.json maybe cors this time May 31, 2016
routes.js remove irrelevant routes, change http in urls to https Sep 25, 2016

README.md

Jam API

Jam API is a service that allows you to turn any site into a JSON accessible api using CSS selectors. To get started simply run a post request to https://www.jamapi.xyz with formdata of "url" and "json_data", here's an example of what your data should look like:

{
  "title": "title",
  "logo": ".nav-logo img",
  "paragraphs": [{ "elem": ".home-post h1", "value": "text"}],
  "links": [{"elem": ".home-post > a:first-of-type", "location": "href"}]
}

Using API you can simply generate JSON data from any website.

Code samples

nodejs

const request = require('request');
request.post('https://www.jamapi.xyz/', {form: {url: 'http://www.gavin.codes/', json_data: '{"title": "title"}'}}, function(err, response, body) {
  console.log(body);
})  

Javascript

fetch('https://www.jamapi.xyz', {
    method: 'POST',
    headers: {
      'Accept': 'application/json',
      'Content-Type': 'application/json'
    },
    body: JSON.stringify({
      url: 'http://www.gavin.codes/',
      json_data: '{"title": "title"}'
    })
  }).then(function(response) {
    return response.json();
  }).then(function(json) {
    document.body.innerHTML = json;
  });  

Ruby

require 'httparty'
response = HTTParty.post("https://www.jamapi.xyz/",
  :body => { "url" => "http://www.gavin.codes/", "json_data" => "{'title': 'title'}"})  
puts response.to_json

Python

import requests
payload = {'url': 'http://www.gavin.codes/', 'json_data': '{"title": "title"}'}

r = requests.post("https://www.jamapi.xyz", data=payload)
print(r.json())

curl

curl -X POST \
  -F 'url=http://www.gavin.codes/' \
  -F 'json_data={"title":"title"}' \
  https://www.jamapi.xyz

Features

Will auto pull the img src on corresponding elements, will auto pull the href from links. If passing JSON, you must provide a "elem" property, and then the element attributes you want. When you pass an array with JSON you'll get a structure that looks as follows:

[
  {
      "index": 0,
      "value": {
          "value": "Porter Robinson – Sad Machine (Cosmo’s Midnight Remix)"
      }
  },
  {
      "index": 1,
      "value": {
          "value": "Listen to Rachel Platten’s “Stand By You”"
      }
  }]

All the attributes you provide as JSON will be put inside of the value property, and the index property is to be able to track what index it ocurred in the DOM. I nested JSON values into it's own so that you can still have an "index" property returned and not run into issues.

How it works

Main power of the program is in services/html_to_json.js. Start site with node index after doing npm install.

Suggested node version is at least 4.2.2