Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Uses pyquery based DSL to turn HTML into a dictionary

branch: master

Fetching latest commit…

Cannot retrieve the latest commit at this time

README.rst

Warning this is a work in progress. I wouldn't use this for anything that will kill someone if it fails.

This project defines a really simple JSON based DSL for converting (X)HTML into dictionaries. Here is an example:

from pyquery import PyQuery
from pyqueryextract import extract
d = PyQuery('<a href="http://www.google.com/">Google!</a>')
spec = {
    'hardcoded': 'text',
    'txt':  {'$query': 'a'},
    'link': {'$query': 'a', '$op': ['attr', 'href']},
}

data = extract(d, spec)

This would produce the following dictionary:

{'link': 'http://www.google.com/', 'txt': 'Google!', 'hardcoded': 'text'}

This Equivelent to the following Python code:

data = {}
data['hardcoded'] = 'text'
data['link'] = d("a").attr("href")
data['txt'] = d("a").text()
print data

While the python code is much more concise, pyqueryextract specs can be stored wherever JSON can be stored.

Explaination

At the moment there is only operation defined which is $query. The value it is set to is what will be searched for in the document. If $op is also defined, it will call the function named in index zero. For instance, attr. The remaining items in the $op list will be applied to the function name.

Something went wrong with that request. Please try again.