fizx / parsley

Parsley is a simple language for extracting structured data from web pages. Parsley consists of an powerful selector language wrapped with a JSON structure that can represent page-wide formatting.

This URL has Read+Write access

parsley / PAPER
100644 36 lines (28 sloc) 1.276 kb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
Abstract
================================================================
A common programming task is data extraction from xml and html documents. I introduce parsley, an embedded language (ala SQL, regular expressions) that improves the usability and/or speed of current extraction techniques.
 
Introduction
================================================================
 
Today, developers use a couple toolsets to do data extraction. Many developers use libraries like Hpricot for Ruby and Beautiful Soup for Python. These libraries allow extraction of xml subtrees via XPath or CSS selectors. These subtrees are futher refined using the scripting language, often with the help of regular expressions.
 
Other developers use XSLT. While fast, mature, and conceptually elegant, XSLT
 
- current techniques
- benefits of standardization
- best of current
 
Features
================================================================
- integrated grammars
  - with some expression examples
- multiple elements, one pass / context switching
- exslt / standard library
- json
- language integration
- pruning
- structural parsing
 
Examples
- Ruby/python/json
- structural parse
-
 
Benchmarks
- size comparision with XSLT
- speed comparision with nokogiri, hpricot
 
Conclusion