Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add engine specific delimiters #75

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 23 additions & 4 deletions .verb.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ Into an object like this:

**Why did we create gray-matter in the first place?**

We created gray-matter after trying out other libraries that failed to meet our standards and requirements.
We created gray-matter after trying out other libraries that failed to meet our standards and requirements.

Some libraries met most of the requirements, but _none met all of them_.

Expand All @@ -71,7 +71,7 @@ Some libraries met most of the requirements, but _none met all of them_.
* Have no problem reading YAML files directly
* Have no problem with complex content, including **non-front-matter** fenced code blocks that contain examples of YAML front matter. Other parsers fail on this.
* Support stringifying back to front-matter. This is useful for linting, updating properties, etc.
* Allow custom delimiters, when it's necessary for avoiding delimiter collision.
* Allow custom delimiters, when it's necessary for avoiding delimiter collision.
* Should return an object with at least these three properties:
- `data`: the parsed YAML front matter, as a JSON object
- `content`: the contents as a string, without the front matter
Expand Down Expand Up @@ -137,6 +137,7 @@ In addition, the following non-enumberable properties are added to the object to
- `file.language` **{String}**: the front-matter language that was parsed. `yaml` is the default
- `file.matter` **{String}**: the _raw_, un-parsed front-matter string
- `file.stringify` **{Function}**: [stringify](#stringify) the file by converting `file.data` to a string in the given language, wrapping it in delimiters and prepending it to `file.content`.
- `file.delimiters` **{Array}**: The delimiters which surrounded the front-matter, stored as `[<open>, <close>]`. Will be `null` if there was no front-matter.


## Run the examples
Expand Down Expand Up @@ -275,7 +276,7 @@ Define custom engines for parsing and/or stringifying front-matter.

**Engine format**

Engines may either be an object with `parse` and (optionally) `stringify` methods, or a function that will be used for parsing only.
Engines may either be an object with `parse` and (optionally) `stringify` method and `delimiters`, or a function that will be used for parsing only.

**Examples**

Expand All @@ -300,7 +301,7 @@ const file = matter(str, {
engines: {
toml: {
parse: toml.parse.bind(toml),

delimiters: '+++',
// example of throwing an error to let users know stringifying is
// not supported (a TOML stringifier might exist, this is just an example)
stringify: function() {
Expand Down Expand Up @@ -363,6 +364,24 @@ categories = "front matter toml"
This is content
```

If you provide the `delimiters` property to a custom engine, then gray-matter will use that to detect the language. For example using `+++` to detect TOML.

```js
const str = `+++
title = "My post"
tags = ["random"]
+++
Content goes here.
`;
const file = matter(str, {
engines: {
toml: {
parse: toml.parse.bind(toml),
delimiters: '+++',
}
}
});
```

### options.delimiters

Expand Down
26 changes: 26 additions & 0 deletions examples/toml-custom.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
const matter = require('..');
const toml = require('toml');

/**
* Parse TOML front-matter
*/

const str = [
'+++',
'title = "TOML"',
'description = "Front matter"',
'categories = ["front", "matter", "toml"]',
'+++',
'This is content'
].join('\n');

const file = matter(str, {
engines: {
toml: {
parse: toml.parse.bind(toml),
delimiters: '+++'
}
}
});

console.log(file);
5 changes: 3 additions & 2 deletions gray-matter.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -42,10 +42,11 @@ declare namespace matter {
excerpt?: string
orig: Buffer | I
language: string
delimiters: [string, string]
matter: string
stringify(lang: string): string
}

/**
* Stringify an object to YAML or the specified language, and
* append it to the given string. By default, only YAML and JSON
Expand Down Expand Up @@ -108,7 +109,7 @@ declare namespace matter {
export function language<O extends matter.GrayMatterOption<string, O>>(
str: string,
options?: GrayMatterOption<string, O>
): { name: string; raw: string }
): { name: string; raw: string; delimiters: null | [string, string] }
}

export = matter
61 changes: 35 additions & 26 deletions index.js
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
const fs = require('fs');
const sections = require('section-matter');
const defaults = require('./lib/defaults');
const delimiters = require('./lib/delimiters');
const stringify = require('./lib/stringify');
const excerpt = require('./lib/excerpt');
const engines = require('./lib/engines');
Expand Down Expand Up @@ -56,39 +57,30 @@ function matter(input, options) {

function parseMatter(file, options) {
const opts = defaults(options);
const open = opts.delimiters[0];
const close = '\n' + opts.delimiters[1];
let str = file.content;

if (opts.language) {
file.language = opts.language;
}

// get the length of the opening delimiter
const openLen = open.length;
if (!utils.startsWith(str, open, openLen)) {
excerpt(file, opts);
return file;
const language = matter.language(str, opts);
if (language.name) {
file.language = language.name;
}

// if the next character after the opening delimiter is
// a character from the delimiter, then it's not a front-
// matter delimiter
if (str.charAt(openLen) === open.slice(-1)) {
if (!language.delimiters) {
excerpt(file, opts);
return file;
}

file.delimiters = language.delimiters;

const close = '\n' + file.delimiters[1];

// strip the opening delimiter
str = str.slice(openLen);
str = str.slice(language.raw.length);
const len = str.length;

// use the language defined after first delimiter, if it exists
const language = matter.language(str, opts);
if (language.name) {
file.language = language.name;
str = str.slice(language.raw.length);
}

// get the index of the closing delimiter
let closeIndex = str.indexOf(close);
if (closeIndex === -1) {
Expand Down Expand Up @@ -183,10 +175,10 @@ matter.read = function(filepath, options) {
};

/**
* Returns true if the given `string` has front matter.
* Returns true if the given `string` has default front matter.
* @param {String} `string`
* @param {Object} `options`
* @return {Boolean} True if front matter exists.
* @return {Boolean} True if default front matter exists.
* @api public
*/

Expand All @@ -205,15 +197,32 @@ matter.test = function(str, options) {
matter.language = function(str, options) {
const opts = defaults(options);
const open = opts.delimiters[0];
let raw, name, delims;

if (matter.test(str)) {
str = str.slice(open.length);
if (!matter.test(str, options)) {
return delimiters(str, options);
}
// if the next character after the opening delimiter is
// a character from the delimiter, then it's not a front-
// matter delimiter
if (str.charAt(open.length) === open.slice(-1)) {
return {
raw: '',
name: '',
delimiters: null
};
}
str = str.slice(open.length);
name = str.slice(0, str.search(/\r?\n/));
raw = open + name;
name = name.trim();
delims = opts.delimiters;
delims[0] = open + name;

const language = str.slice(0, str.search(/\r?\n/));
return {
raw: language,
name: language ? language.trim() : ''
raw,
name,
delimiters: delims
};
};

Expand Down
49 changes: 49 additions & 0 deletions lib/delimiters.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
const defaults = require('./defaults');
const utils = require('./utils');

module.exports = function(str, options) {
const opts = defaults(options);
const dlen = str.search(/\r?\n/);
let raw, name, delimiters;
if (dlen > 0) {
const customDelims = mapCustomDelimiters(opts);
raw = str.substr(0, dlen);
const firstLine = raw.trim();
if (customDelims[firstLine]) {
name = customDelims[firstLine];
delimiters = utils.arrayify(opts.engines[name].delimiters);
if (delimiters.length === 1) {
delimiters.push(delimiters[0]);
}
return {
raw: raw || '',
name: name || '',
delimiters
};
}
}

// No frontmatter
return {
raw: '',
name: '',
delimiters: null
};
};

function mapCustomDelimiters(opts) {
const customDelims = {};
for (const engine in opts.engines) {
if (opts.engines[engine].delimiters) {
const delims = utils.arrayify(opts.engines[engine].delimiters);
if (customDelims[delims[0]]) {
throw new Error('Another engine has already used delimiter: ' + delims[0]);
}
if (delims[0] === opts.delimiters[0]) {
throw new Error('Engine specific delimiters cannot match the default delimiters: ' + opts.delimiters[0]);
}
customDelims[delims[0]] = engine;
}
}
return customDelims;
}
5 changes: 3 additions & 2 deletions lib/stringify.js
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,9 @@ module.exports = function(file, data, options) {
}

data = Object.assign({}, file.data, data);
const open = opts.delimiters[0];
const close = opts.delimiters[1];
file.delimiters = file.delimiters || [];
const open = file.delimiters[0] || opts.delimiters[0];
const close = file.delimiters[1] || opts.delimiters[1];
const matter = engine.stringify(data, options).trim();
let buf = '';

Expand Down
1 change: 1 addition & 0 deletions lib/to-file.js
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ module.exports = function(file) {
// set non-enumerable properties on the file object
utils.define(file, 'orig', utils.toBuffer(file.content));
utils.define(file, 'language', file.language || '');
utils.define(file, 'delimiters', file.delimiters || ['', '']);
utils.define(file, 'matter', file.matter || '');
utils.define(file, 'stringify', function(data, options) {
if (options && options.language) {
Expand Down
30 changes: 18 additions & 12 deletions test/matter.language.js
Original file line number Diff line number Diff line change
Expand Up @@ -13,31 +13,37 @@ var matter = require('..');
describe('.language', function() {
it('should detect the name of the language to parse', function() {
assert.deepEqual(matter.language('---\nfoo: bar\n---'), {
raw: '',
name: ''
raw: '---',
name: '',
delimiters: ['---', '---']
});
assert.deepEqual(matter.language('---js\nfoo: bar\n---'), {
raw: 'js',
name: 'js'
raw: '---js',
name: 'js',
delimiters: ['---js', '---']
});
assert.deepEqual(matter.language('---coffee\nfoo: bar\n---'), {
raw: 'coffee',
name: 'coffee'
raw: '---coffee',
name: 'coffee',
delimiters: ['---coffee', '---']
});
});

it('should work around whitespace', function() {
assert.deepEqual(matter.language('--- \nfoo: bar\n---'), {
raw: ' ',
name: ''
raw: '--- ',
name: '',
delimiters: ['---', '---']
});
assert.deepEqual(matter.language('--- js \nfoo: bar\n---'), {
raw: ' js ',
name: 'js'
raw: '--- js ',
name: 'js',
delimiters: ['---js', '---']
});
assert.deepEqual(matter.language('--- coffee \nfoo: bar\n---'), {
raw: ' coffee ',
name: 'coffee'
raw: '--- coffee ',
name: 'coffee',
delimiters: ['---coffee', '---']
});
});
});
15 changes: 15 additions & 0 deletions test/parse-toml.js
Original file line number Diff line number Diff line change
Expand Up @@ -45,4 +45,19 @@ describe('parse TOML:', function() {
matter('---toml\n[props\nuser = "jonschlinkert"\n---\nContent\n');
});
});

it('should auto-detect TOML with custom delimiters.', function() {
var actual = parse('+++\ntitle = "autodetect-TOML-custom-delims"\n[props]\nuser = "jonschlinkert"\n+++\nContent\n', {
engines: {
toml: {
parse: toml.parse.bind(toml),
delimiters: '+++'
}
}
});
assert.equal(actual.data.title, 'autodetect-TOML-custom-delims');
assert(actual.hasOwnProperty('data'));
assert(actual.hasOwnProperty('content'));
assert(actual.hasOwnProperty('orig'));
});
});
22 changes: 22 additions & 0 deletions test/stringify.js
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,28 @@ describe('.stringify', function() {
].join('\n'));
});

it('should stringify a file object with custom delimiters', function() {
var file = { content: 'Name: {{name}}', data: {name: 'gray-matter'}, delimiters: ['+++', '+++'] };
var actual = matter.stringify(file);
assert.equal(actual, [
'+++',
'name: gray-matter',
'+++',
'Name: {{name}}\n'
].join('\n'));
});

it('should stringify a file object with extra language info', function() {
var file = { content: 'Name: {{name}}', data: {name: 'gray-matter'}, delimiters: ['---toml', '---'] };
var actual = matter.stringify(file);
assert.equal(actual, [
'---toml',
'name: gray-matter',
'---',
'Name: {{name}}\n'
].join('\n'));
});

it('should stringify an excerpt', function() {
var file = { content: 'Name: {{name}}', data: {name: 'gray-matter'} };
file.excerpt = 'This is an excerpt.';
Expand Down