Parsoid JSAPI

Usage of the JavaScript API

This file describes usage of Parsoid as a standalone wikitext parsing package, in the spirit of mwparserfromhell. This is not the typical use case for Parsoid; it is more often used as a network service. See the HTTP API guide or Parsoid service on the wiki for more details.

These examples will use the prfun library and ES6 generators in order to fluently express asynchronous operations.

Since many methods in the API return Promises, we've also provided a Promise-aware REPL, that will wait for a promise to be resolved before printing its value. This can be started from the shell using:

node -e 'require("parsoid-jsapi").repl()'

Use "./" instead of "parsoid" if you are running this from a checked-out repository. Code examples below which contain lines starting with > show sessions using this REPL. (You may also wish to look in tests/mocha/jsapi.js for examples using a more traditional promise-chaining style.)

Use of Parsoid as a wikitext parser is straightforward (where text is wikitext input):

#/usr/bin/node --harmony-generators
var Promise = require('prfun');
var Parsoid = require('parsoid-jsapi');

var main = Promise.async(function*() {
	var text = "I love wikitext!";
	var pdoc = yield Parsoid.parse(text, { pdoc: true });
	console.log(pdoc.document.outerHTML);
});

// start me up!
main().done();

As you can see, there is a little bit of boilerplate needed to get the asynchronous machinery started. The body of the main() method can be replaced with your code.

The pdoc variable above holds a PDoc object, which has helpful methods to filter and manipulate the document. If you want to access the raw Parsoid DOM, however, it is easily accessible via the document property, as shown above, and all normal DOM manipulation functions can be used on it (Parsoid uses domino to implement these methods). Be sure to call update() after any direct DOM manipulation. PDoc is a subclass of PNodeList, which provides a number of useful access and mutation methods -- and if you use these you won't need to manually call update(). These provided methods can be quite useful. For example:

> let text = "I has a template!\n{{foo|bar|baz|eggs=spam}}\nSee it?\n";
> let pdoc = yield Parsoid.parse(text, { pdoc: true });
> console.log(yield pdoc.toWikitext());
I has a template!
{{foo|bar|baz|eggs=spam}}
See it?
    
> let templates = pdoc.filterTemplates();
> console.log(yield Promise.map(templates, Parsoid.toWikitext));
[ '{{foo|bar|baz|eggs=spam}}' ]
> let template = templates[0];
> console.log(template.name);
foo
> template.name = 'notfoo';
> console.log(yield template.toWikitext());
{{notfoo|bar|baz|eggs=spam}}
> console.log(template.params.map(function(p) { return p.name; }));
[ '1', '2', 'eggs' ]
> console.log(yield template.get(1).value.toWikitext());
bar
> console.log(yield template.get("eggs").value.toWikitext());
spam

Getting nested templates is trivial:

> let text = "{{foo|bar={{baz|{{spam}}}}}}";
> let pdoc = yield Parsoid.parse(text, { pdoc: true });
> console.log(yield Promise.map(pdoc.filterTemplates(), Parsoid.toWikitext));
[ '{{foo|bar={{baz|{{spam}}}}}}',
  '{{baz|{{spam}}}}',
  '{{spam}}' ]

You can also pass { recursive: false } to filterTemplates() and explore templates manually. This is possible because the get method on a PTemplate object returns an object containing further PNodeLists:

> let text = "{{foo|this {{includes a|template}}}}";
> let pdoc = yield Parsoid.parse(text, { pdoc: true });
> let templates = pdoc.filterTemplates({ recursive: false });
> console.log(yield Promise.map(templates, Parsoid.toWikitext));
[ '{{foo|this {{includes a|template}}}}' ]
> let foo = templates[0];
> console.log(yield foo.get(1).value.toWikitext());
this {{includes a|template}}
> let more = foo.get(1).value.filterTemplates();
> console.log(yield Promise.map(more, Parsoid.toWikitext));
[ '{{includes a|template}}' ]
> console.log(yield more[0].get(1).value.toWikitext());
template

Templates can be easily modified to add, remove, or alter params. Templates also have a nameMatches() method for comparing template names, which takes care of capitalization and white space:

> let text = "{{cleanup}} '''Foo''' is a [[bar]]. {{uncategorized}}";
> let pdoc = yield Parsoid.parse(text, { pdoc: true });
> pdoc.filterTemplates().forEach(function(template) {
...    if (template.nameMatches('Cleanup') && !template.has('date')) {
...        template.add('date', 'July 2012');
...    }
...    if (template.nameMatches('uncategorized')) {
...        template.name = 'bar-stub';
...    }
... });
> console.log(yield pdoc.toWikitext());
{{cleanup|date=July 2012}} '''Foo''' is a [[bar]]. {{bar-stub}}

At any time you can convert the pdoc into HTML conforming to the MediaWiki DOM spec (by referencing the document property) or into wikitext (by invoking toWikitext(), which returns a Promise for the wikitext string). This allows you to save the page using either standard API methods or the RESTBase API (once T101501 is resolved).

License

Copyright (c) 2011-2018 Wikimedia Foundation and others.

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.