Filter block truncating content

[ElChupacabra26]

Filter blocks with template info are getting chopped off at TOKEN_VARIABLE_START. What I'm trying to do is pass a template string to my filter, so I can do some processing before calling renderString() on the block contents. Something like this:

{% filter foo %}
    <div class="singleVal">
        Your val is: {{someVal}}
    </div>
{% endfilter %}

syncFilters.foo = function foo (fooContent) {
    var output = '';
    var vals = require('config').someVals || ['default'];

    for (var i = 0; i < vals.length; i++) {
        output += nunjucks.renderString(fooContent, {someVal: vals[i]});
    }

    return output;
};

Is this the expected behavior and if so, is there another way of passing template data into a filter?

[carljm]

I would expect the content within the {% filter %} tag to be parsed and processed normally before being passed to the filter, so {{ someVal }} would be replaced by its value.

If you didn't want that, you'd wrap the contents in a {% raw %} tag.

If it's really just being cut off entirely, that seems like a bug that should be fixed.

[fabien]

I actually have the following test-case fail after updating nunjucks today, so I suspect there's something wrong indeed.

it('should handle tag: filter', function(next) {
    var tmpl = '{% filter replace("Bar", "TEST") %}\n';
    tmpl += 'Filtered: {{foo}}\n'; 
    tmpl += '{% endfilter %}'; 
    application.renderString(tmpl, { foo: 'FooBar' }, function(err, res) {
        should.not.exist(err);
        res.should.containEql('Filtered: FooTEST'); // failed output: 'Filtered: '
        next();
    });
});

[ElChupacabra26]

@carljm I actually tried wrapping my content in the raw tag as well which ended up giving me an empty string. It seems it is being truncated entirely. For example, Your val is: {{someVal}}... more text passes the content up to the {{ token and chops the rest off.

@fabien Under parseFilterStatement, nunjucks is passing body.children[0]... to the nodelist init, but I'm not sure if there's a reason it's being done that way... Didn't really try to dig any deeper honestly.

[carljm]

@fabien So you're saying this used to work, but broke at some point? Any chance you could bisect the commit that broke it?

[fabien]

@carljm turns out I had implemented my own filter tag, and now that it became a native nunjucks tag, it wasn't used anymore. My own tag implementation did work as expected (interpolating the content of the block before filtering).

[carljm]

@fabien Ah! Well then, any chance you'd be willing to share your implementation with the community by submitting it as PR to fix the broken built-in version? :-)

[fabien]

I'll look into it - IIRC I had my own filter for it. Then again, it might have been a fork of this one:
https://github.com/SamyPesse/nunjucks-filter

[vkbansal]

I too am facing similar issue. My code is as follows

// Not working. Returns empty string
{% filter replace('\n', '') %}
    {% block content %}{% endblock %}
{% endfilter %}

// Working
{% block content %}{% endblock %}

[jrehwaldt]

Any hints how this can be fixed? I added test cases for @vkbansal's (also mine) as well as @fabien's usecase in #647. Any ideas I had on how to fix this failed so far due to my incomplete knowledge of the parser/compiler.

[carljm]

@jrehwaldt That's an awesome contribution, thanks! It definitely looks like you've narrowed things down to the problematic assumption in the code. I think it's likely that with some study I (or someone else familiar with the internals) could give some useful guidance on how to approach the fix. Unfortunately I don't have time for that focused study today, or probably this week, but I'll put this at the top of my nunjucks list to get to when I can. Maybe someone else will be able to provide some useful feedback in the meantime.

Thanks again!

[ElChupacabra26]

@carljm I remember diving into the internals a few months back when I first ran into this problem and found a possible culprit, but wasn't familiar enough with the Nunj internals to dive any deeper. Not sure if it's the actual culprit, but see my second comment:

Under parseFilterStatement, nunjucks is passing body.children[0]... to the nodelist init

Maybe that'll give you something to go off of when you find some time to look into this issue.

[carljm]

@ElChupacabra26 Yeah, thanks for reminding me of that comment. That's actually the same observation @jrehwaldt makes in the linked tests PR.

[ElChupacabra26]

@carljm Ah yes, I see that now. Thanks

[carljm]

Fixed! Thanks @jrehwaldt for the failing test, and everyone for the investigation and pointers.

[jrehwaldt]

Thank you. Works perfectly.

[ElChupacabra26]

Thanks for the fix @carljm!