Examples of using gulp-data with json files and watch

[wjthomas9]

Hi,

Do you have any examples of how to ensure templates get fresh data when saving a json file? It seems I have to stop and start gulp every time I make a change to a json file.

[SkeLLLa]

Hi, @wjthomas9.
Here's an example:

var getJsonAsync = function (p, cb) {
    p = path.join(path.dirname(p), path.basename(p, '.twig') + '.json');
    fs.stat(p, function (err) {
        if (err) {
            cb(undefined, {});
        } else {
            fs.readFile(p, 'utf8', function (errRead, data) {
                if (errRead) {
                    cb(undefined, {});
                } else {
                    var jsData = JSON.parse(data);
                    cb(undefined, jsData);
                }
            })
        }
    })
};

config.html.watch = function (target, isPart) {
    return isPart ? [
        path.join(dd, target, '**', '_*.twig')
    ] : [
        path.join(dd, target, '**', '*.twig'),
        path.join(dd, target, '**', '*.json'),
        '!' + path.join(dd, target, '**', '_*.twig')
    ];
}

var twig = function (env) {
    env.mode = config.env.modes.DEVELOPMENT;
    var dst = config.html.dst(env.target);

    return $.watch(config.html.watch(env.target, env.partials), {name: 'twig'})
        .pipe($.tap(function (f) {
            var src = /\.json$/.test(f.path) ? f.path.replace(/\.json$/, '.twig') : f.path;
            if (env.partials) {
                src = config.html.src(env.target);
            }
            return gulp.src(src)
                .pipe($.data(function (file, cb) {
                    return getJsonAsync(file.path, cb);
                }))
                .pipe($.twig(config.html[env.target].actions.twig.options))
                .pipe(gulp.dest(dst))
                .pipe($.debug({title: 'Reloading:'}))
                .pipe(browserSync.reload({stream: true}));
        }));
}

It's taken from my big gulpfile, so some variables will be undefined, if you just copy-paste it into your file, but the main idea should be understandable.

[vnaking]

@SkeLLLa : I don't understand your script, could you make it more understandable?

[xanisu]

Can someone bring here a bit of light?

I'm stucked trying to use gulp-data on a gulp-watch, but get nothing but errors.

Please, need some help.

This example is quite complex, I've tried to resume it but still no luck at all. :-(

[SkeLLLa]

@xanisu

First of all to read json files you'll need function like getJsonAsync in my example. It receives path of template file (in my case it was .twig file and tries to read a .json file with data that is placed near template file.

Then in watch task I add watcher (gulp-watch) on .twig and json files. So if some of files will change it will trigger the update.
Later in code to detect what should be changed I use gulp-tap plugin. If changed file is json, the we need to compile twig template again and then I create src variable that refers to correct template file. Then we use gulp-data as usual with our helper function getJsonAsync and then we call template compiler (in my case it's gulp-twig).

Also the case with template partials is handled. For example if partial template is changed (e.g. header template that is common for several pages) then I set variable env.partials and recompile all templates.

[xanisu]

thanks @SkeLLLa .

I'm going to give it a try with 'swig' templates, perhaps I've been no luck with .jade.

I think I must to change my approach to my problem. If I have no success with swig, will try 'twig' as your example.

Thanks!

[SkeLLLa]

@xanisu I think that it should work with both swig and twig. If you have no luck with twig also, then maybe post some of your gulp file code here, then may be I'll be able to help you.

[xanisu]

thanks @SkeLLLa Will let the results here once I've tested it. I appreciate your help.

[xanisu]

Finally I could get it working just with jade.

Heres my working example:

var gulp = require('gulp'),
  data = require('gulp-data'),
  browserSync = require("browser-sync").create(),
  fs = require('fs'),
  swig = require('gulp-swig'),
  jade = require('gulp-jade');

gulp.task('translate-It', function ()
{
    var dataFile = './language/language.json';
    return gulp.src('./templates/**/*.jade')
      .pipe(data(function(file) {
        return JSON.parse(fs.readFileSync(dataFile));
      }))
      .pipe(jade({ pretty: true }))
      .pipe(gulp.dest('./'));
});

gulp.task('watch-translate', function(){

  browserSync.init({
    server: {
      baseDir: "./"
    }
  });

  gulp.watch('./templates/**/*.jade', ['translate-It']);
})

Every time I change a jade template, I get an error message, but It still works, don't need to restart my gulp watch process.

[SkeLLLa]

@xanisu What kind of error? Can you please tell what that error says?

[xanisu]

@SkeLLLa Here it is

Cannot read property 'title' of undefined
    at eval (eval at <anonymous> (C:\Users\jflopez\AppData\Roaming\npm\node_modules\jade\lib\index.js:218:8), <anonymous>:188:53)
    at eval (eval at <anonymous> (C:\Users\jflopez\AppData\Roaming\npm\node_modules\jade\lib\index.js:218:8), <anonymous>:12594:22)
    at res (C:\Users\jflopez\AppData\Roaming\npm\node_modules\jade\lib\index.js:219:38)
    at renderFile (C:\Users\jflopez\AppData\Roaming\npm\node_modules\jade\bin\jade.js:270:40)
    at C:\Users\jflopez\AppData\Roaming\npm\node_modules\jade\bin\jade.js:136:5
    at Array.forEach (native)
    at Object.<anonymous> (C:\Users\jflopez\AppData\Roaming\npm\node_modules\jade\bin\jade.js:135:9)
    at Module._compile (module.js:413:34)
    at Object.Module._extensions..js (module.js:422:10)
    at Module.load (module.js:357:32)

Process finished with exit code 1

But the watch process is still working so, everytime a modify my .jade file, I get this error on output window (running gulp command on webStorm), but changes are well compiled by jade, and the output html result is fine.

[SkeLLLa]

@xanisu May be in your jade file you're referencing some fields that are not defined in your json. To make it clear you can try to experiment on simple jade template that has only one variable and a json file that also has one property.

[jitendravyas]

Can JSON data file have html tags with text?

[wjthomas9]

@jitendravyas Yep you can. But your front-end template parser needs to have a filter for parsing the HTML. For example, if you use something like Twig.js as your template parser, you can use the "escape" filter.

{{ someVariable | escape }}

http://twig.sensiolabs.org/doc/filters/escape.html

What template parser are you using?

[polarathene]

Might be nice to update the example for JSON use. It's been a while since I used Node and did web dev, I was confused why my template wasn't updating and assumed the template was caching, but it turned out to be the require call cached my file, I had to replace the require with fs and JSON.parse() like in the shared code above.

Another thing that was not clear(partly gulp-data docs as well as gulp-pug's) was gulp-data returns an object with the content in a data property/field, I think gulp-data just referred to it as a data object with no examples of it's output. gulp-pug mentioned the merging of a data field but not that it needed to be piped through(likely my fault for jumping back into gulp after forgetting how it works).