Metalsmith plugin to load data from files directly into your metadata.
Switch branches/tags
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
lib
spec
.editorconfig
.eslintignore
.eslintrc.yaml
.gitignore
.travis.yml
CHANGELOG.md
CONTRIBUTING.md
LICENSE.md
README.md
README.post
README.pre
package-lock.json
package.json

README.md

metalsmith-data-loader

Metalsmith plugin to add extra metadata from external files. Very similar to metalsmith-models with the difference that this allows you to load information from either a folder outside of the source or files within the source tree. Also, this one lets you use JSON or YAML files.

npm version Build Status Dependencies Dev Dependencies codecov.io

The "dependencies" badge says there's a potential problem with js-yaml and suggests that packages use .safeLoad(), which is exactly what this plugin does.

What It Does

When working with templates, you sometimes want to generate something like a table in Markdown from external data. You can do this with metalsmith-hbt-md but you need to get the extra data in your metadata. That can be accomplished by metalsmith-models as long as you are willing to separate your source data from the template that needs it.

This plugin differs from that approach, allowing you to have your Markdown files adjacent to the data that is inserted into the table. This works better with an example.

table.md:

---
title: Just a test file to illustrate why the module is useful.
data: contacts.yaml
---

Here's a great list of contacts:

| First | Last | Email |
|-------|------|-------|
{{#data}}| {{first}} | {{last}} | {{email}} |
{{/data}

contacts.yaml (right next to the markdown):

-
    first: Tyler
    last: Akins
    email: fidian@rumkin.com
-
    first: Jane
    last: Doe
    email: j.doe@example.com

This isn't limited to table generation. You could load metadata specific to a collection of pages. Maybe you have a site where different authors maintain different pages and you could point to a single source for the author's information.

If you prefer to work with code, there is an example repository set up that illustrates how the plugin functions.

Installation

npm can do this for you.

npm install --save metalsmith-data-loader

Usage

Include this like you would include any other plugin. First, a CLI example that shows the default options. You don't need to specify any options unless you want to override their values.

{
    "plugins": {
        "metalsmith-data-loader": {
            "dataProperty": "data",
            "directory": "models/",
            "match": "**/*",
            "matchOptions": {},
            "removeSource": false
        }
    }
}

And here is the JavaScript example. This includes brief descriptions of each of the options.

// Load this, just like other plugins.
var dataLoader = require("metalsmith-data-loader");

// Then in your list of plugins you use it.
.use(dataLoader())

// Alternately, you can specify options.  The values shown here are
// the defaults.
.use(dataLoader({
    // Property name to scan for files to include.
    dataProperty: "data",

    // Directory containing models.  This is relative to the working
    // directory.  It does not need to exist unless you start loading
    // files from here.
    directory: "models/",

    // Pattern of files to match in case you want to limit processing
    // to specific files.
    match: "**/*",

    // Options for matching files.  See metalsmith-plugin-kit for
    // more information.
    matchOptions: {},

    // Flag indicating that the loaded data object should be removed
    // if it is found in the source.  Use this so your built project
    // doesn't include the metadata in the rendered output and another
    // copy that was consumed during the build.
    removeSource: false
})

This uses metalsmith-plugin-kit to match files. The .matchOptions object can be filled with options control how the matching behaves.

From here, you now need to specify the files to include. These examples all assume you didn't change dataProperty.

---
title: A single string loads one file
data: my-model.json
---

When loaded this way, the data metadata is replaced with the parsed contents of my-model.json.

---
title: An object holding a map of filenames.
data:
    users: users.json
    services: services.yaml
---

This lets you load multiple files and assign them to properties on the data object. One would access the information by using {{data.users}} or {{data.services}} in Mustache templates.

---
title: An array of files.  The filenames are replaced with their contents.
data:
    - file1.yaml
    - file2.yaml
---

The two files are loaded asynchronously and will replace the file1.yaml and file2.yaml. You would access these in Mustache by using {{data.0}} and {{data.1}} or you can iterate over {{#data}}.

Resolving Files

The combination of the directory configuration option and the data metadata property dictate which files are loaded. This table can help illustrate the relationship. In all of the examples, the directory configuration option is set to "models/" and the source file is always "src/path/file.md".

Metadata Path File to Load Description
file.yaml src/path/file.yaml Relative to source file.
../file.yaml src/file.yaml Relative to the source file.
/other-path/file.yaml src/other-path/file.yaml Resolved from root of source folder.
/../file.yaml file.yaml Can load things outside the source folder.
!file.yaml models/file.yaml Resolved from the directory, not source folder. See the note!
!../file.yaml file.yaml Can load items from outside the models directory. See the note!

Note: When the metadata path starts with an exclamation point (!), you must use quotes in your YAML frontmatter. Here's a sample.

---
title: Sample file that loads a file from a models directory
data: "!file-from-models.json"
---

You can play with the example repository to get a better handle on how the plugin works.

API

metalsmith-data-loader

Params

  • destination Object - Metalsmith Data Loader

Loads files and places their data structures in as file metadata.

Example

var dataLoader = require("metalsmith-data-loader");

// Create your Metalsmith instance and add this like other middleware.
metalsmith.use(dataLoader({
    // configuration goes here
}));

module.exports(options) ⇒ function

Factory to build middleware for Metalsmith.

Kind: Exported function
Params

module.exports~removeSource(sourceFile, modelFile, files, options)

Removes a file from the list of files to process. This only happens when it is included by another file through this data loader and when the option is enabled.

Kind: inner method of module.exports
Params

module.exports~resolveFile(metalsmith, sourceFile, modelFile, options) ⇒ string

Resolves a file path and checks to make sure it exists.

Paths can look like any of these:

model.json -> Relative to file ./model.json -> Relative to file dir/model.yaml -> Relative to file ../model.json -> Relative to file /model.yaml -> From root of source /dir/model.json -> From root of source !model.yaml -> From models folder (not in source) !dir/model.json -> From models folder (not in source)

Kind: inner method of module.exports
Params

  • metalsmith module:metalsmith - Metalsmith instance.
  • sourceFile string - Filename being processed.
  • modelFile string - Reference inside the file's metadata.
  • options options

module.exports~loadModel(dataFile) ⇒ Promise.<*>

Loads a file asynchronously and places metadata on the destination object.

Kind: inner method of module.exports
Returns: Promise.<*> - The loaded data.
Params

  • dataFile string - Resolved filename to load.

module.exports~metalsmithFile : Object

Metalsmith file object.

Kind: inner typedef of module.exports
Properties

  • contents Buffer
  • mode string

module.exportsmetalsmithFileCollection : Object.<string, module:metalsmith-data-loader--module.exportsmetalsmithFile>

Metalsmith collection of file objects.

Kind: inner typedef of module.exports

module.exports~options : Object

Options for the middleware factory.

Kind: inner typedef of module.exports
See: https://github.com/fidian/metalsmith-plugin-kit
Properties

  • dataProperty string - Name of property to use for loading models.
  • directory string - Path for storing external modules.
  • match module:metalsmith-plugin-kit~matchList - Files to match. Defaults to all files.
  • matchOptions module:metalsmith-plugin-kit~matchOptions - Options controlling how to match files.
  • removeSource boolean - When truthy, remove the model from the build result.

Development

This uses Jasmine, Istanbul and ESLint for tests.

# Install all of the dependencies
npm install

# Run the tests
npm run test

This plugin is licensed under the MIT License with an additional non-advertising clause. See the full license text for information.