feat: Add support of inner expressions enclosed by parentheses

[skynetigor]

Adds support for parenthesized expressions as operands in tags, gated behind a new groupedExpressions option (default false). Closes #833.

const liquid = new Liquid({ groupedExpressions: true })

This is a non-standard extension to Liquid syntax — Ruby Liquid does not support this. The feature is opt-in to preserve strict compatibility by default and can be enabled by default in a future major version.

Backward compatibility: Even with groupedExpressions: true, all existing Liquid templates continue to work identically. The change is purely additive — it only gives meaning to syntax that previously threw a parse error ((expr | filter) in operand position). No valid Liquid template changes behavior. Range syntax (1..5) is unaffected. Templates written for standard Liquid are fully compatible regardless of the flag setting.

When enabled, parentheses that don't contain range syntax (..) are parsed as grouped sub-expressions, allowing filter chains and logical grouping to be used directly as operands:

{% if (username | size) >= 3 %}...{% endif %}
{% if (a | upcase) == (b | upcase) %}...{% endif %}
{% if (name | size) > 0 and (email | downcase) contains "@" %}...{% endif %}
{% unless (content | strip_html | size) == 0 %}...{% endunless %}
{% case (status | downcase) %}{% when "active" %}...{% endcase %}
{% for i in (1..(items | size)) %}...{% endfor %}

Existing range syntax (1..5) is unaffected — a lookahead scan checks for .. at depth 1 before deciding which parser to invoke.

Approach

• A looksLikeRange() lookahead in readValue() routes ( to either readRange() or the new readGroupedExpression()
• readGroupedExpression() temporarily limits the tokenizer boundary to the matching ) and delegates to the existing readFilteredValue() parser
• A new GroupedExpressionToken stores the parsed expression and filters; a resolveGroupedExpressions() utility recursively resolves them into Value instances when the Liquid instance is available
• Tags that parse values outside of Value (for, case/when) call the resolver in their constructors

Test plan

• Tokenizer unit tests: parsing grouped expressions, nested parens, range fallback, flag-off behavior
• Expression unit tests: existing range error behavior preserved when flag is off
• Integration tests for if, unless, for, case/when with groupedExpressions: true
• evalValueSync API tests for direct expression evaluation

[harttle]

The parsing is no longer near O(N) complexity with findMatchingParen and findMatchingParen introduced.

[harttle]

Though we know JS is single threaded, we need avoid this hack for simplicity. For example when readFilteredValue throw an error, we need this.N be correct in error handler.

[harttle]

Let's avoid lookslike, the language should be deterministic and non ambiguous.

[harttle]

Thanks for your effort on this long waited feature.

It's expected to break something thus I put it in the v11 version planning. But if it's totally compatible, l'd like to merge it in current major version with a feature bump.

As @jg-rp mentioned Shopify/liquid is also working on this, I think we can wait until then to keep aligned.

[skynetigor]

Many thanks for the review, I'll address all the comments 👍

[jg-rp]

You could replace readRange, readGroupedExpression, findMatchingParen and looksLikeRange with

  readGroupOrRange (): RangeToken | GroupedExpressionToken | undefined {
    this.skipBlank()
    const begin = this.p
    if (this.peek() !== '(') return
    ++this.p
    const lhs = this.readValueOrThrow()
    this.skipBlank()

    if (this.peek() === '.' && this.peek(1) === '.') {
      this.p += 2
      const rhs = this.readValueOrThrow()
      this.skipBlank()
      this.assert(this.read() === ')', 'invalid range syntax')
      return new RangeToken(this.input, begin, this.p, lhs, rhs, this.file)
    }

    if (this.groupedExpressions) {
      const expression = new Expression(toIterableIterator(lhs))
      this.skipBlank()
      const filters = this.readFilters()
      this.skipBlank()
      this.assert(this.read() === ')', 'unbalanced parentheses')
      this.skipBlank()
      return new GroupedExpressionToken(expression, filters, this.input, begin, this.p, this.file)
    }

    throw this.error('invalid range syntax')
  }

Along with some changes to readFilter and a toIterableIterator utility.

But it still all feels like a bit of a work around.

As soon as filters become context free expressions rather than a special post processing step, tokenizing/parsing really needs access to the current Liquid object so filter names can be resolved without resorting to resolveGroupedExpressions and resolvedValue.

In an ideal world I'd like to drop the concept of Value and ValueToken, and see Expression redefined so that "everything is an expression".

That being said - if you're keen to get this feature released without a wider refactoring - the approach with readGroupOrRange() should address @harttle's comments and work without any breaking changes.

(Developers with custom tags might still need to make changes when enabling the groupedExpressions option.)

[skynetigor]

Hi guys!
@jg-rp thanks for the suggestion, it's easier and cleaner implementation though I had to fine-tune it a bit. I've just pushed the changes with it.

@harttle I think it's safe change for the current major version. The flag groupedExpressions is false by default, so the previous syntax will work by default. The groupedExpressions flag just enables this extended syntax and the tests prove it. Please let me know if I need to add more tests 🙏

That would be a nice feature improving our customers experience 🤞

[harttle]

Try to create something like evalGroupedExpressionToken(), instead of pre populate resolvedValue, which can be scattered in multiple places.

[harttle]

Suggested change

	collection: ValueToken | GroupedExpressionToken

[harttle]

this can be removed now.

[harttle]

create a extractGroupedExpressionTokenVariables

[harttle]

I guess this should be moved to evalGroupedExpressionToken(), and if some structures can be created in parse time, need to be done in GroupedExpressionToken#constructor and consumed during evalGroupedExpressionToken()

[harttle]

templates (include Value) depends on tokens, tokens don't depends on templates

[rosomri]

Hi @harttle, Omri from @skynetigor's team - thanks for the detailed review! I'd like to confirm the approach before implementing:

Proposed Changes

1. Update token structure (fixes layering violation):

export class GroupedExpressionToken extends Token {
  public resolvedFilters?: Filter[]  // Change from resolvedValue to resolvedFilters
}

Tokens will store Filter[] (mid-level) instead of Value (template-layer), maintaining proper dependency direction.

2. Evaluate at render time (fixes eager resolution):

function * evalGroupedExpressionToken(token, ctx, lenient) {
  assert(token.resolvedFilters, 'grouped expression filters not resolved')
  
  let val = yield token.initial.evaluate(ctx, lenient)  // Lazy evaluation
  for (const filter of token.resolvedFilters!) {
    val = yield filter.render(val, ctx)  // Apply filters lazily
  }
  return val
}

This follows the generator pattern used throughout liquidjs for async/sync duality.

3. Build Filter instances during Value construction:

export function resolveGroupedExpressionFilters(token: Token, liquid: Liquid) {
  if (isGroupedExpressionToken(token)) {
    for (const t of token.initial.postfix) {
      resolveGroupedExpressionFilters(t, liquid)  // Recursive
    }
    token.resolvedFilters = token.filters.map(filterToken =>
      new Filter(filterToken, getFilter(liquid, filterToken.name), liquid)
    )
  }
  // ... handle Range/PropertyAccess recursively
}

Called from Value constructor, this builds Filter instances (which carry the liquid reference) at parse time for use at render time.

4. Resolution calls in tag constructors (regarding this):

Tags that store raw ValueToken and evaluate them directly with evalToken() still need explicit resolveGroupedExpressionFilters() calls:

• for.ts - collection property
• case.ts - when values
• tablerow.ts - collection property

Tags that wrap values with new Value() (like if, unless, assign, echo) don't need explicit calls since Value constructor handles resolution recursively via initial.postfix.

Why This Works

• Maintains proper layering: tokens → render → templates
• Evaluation happens lazily at render time using generators (consistent with Value.value())
• Filter instances created at parse time carry liquid reference for render-time filter resolution
• Minimal parse-time work: only build Filter instances, defer actual evaluation to render time

Attaching a draft PR with the proposed implementation - https://github.com/harttle/liquid
Does this approach align with your architectural vision?

[jg-rp]

I was imagining that we could change the Tokenizer constructor to accept a liquid argument.

type TokenizerCtor = new (
  input: string,
  liquid: Liquid, // Get options directly from the Liquid object.
  file?: string,
  range?: [number, number]
) => Tokenizer;

And add FilteredValueToken to ValueToken.

type ValueToken =
    RangeToken
    | LiteralToken
    | QuotedToken
    | PropertyAccessToken
    | NumberToken
    | FilteredValueToken // This is new.

We'd then need to change FilteredValueToken to hold instances of Filter instead of FilterToken, and Tokenizer.readFilter would need to return Filter instead of FilterToken, which is now possible because it has access to Tokenizer.liquid.

(Notice that the existing definition of FilteredValueToken is almost identical to GroupedExpressionToken.)

To continue to support runtime tokenization in some filters, we'd also need to change Context to store a reference to the active Liquid object, allowing filters to create new tokenizers via this.context.liquid.

This approach moves responsibility for Filter construction from Value to Tokenizer, and avoids anything other than Expression knowing that sub expressions exist.

[harttle]

Can we reuse FilteredValueToken and let readGroupOrRange() return FilteredValueToken | RangeToken? And FilteredValueToken can also be used to new Value(filteredValueToken) so it can be used in tag implementions with minor change.

I admit ValueToken and Expression are not well designed which hopefully can be addressed in v11. For now I don't think we can include filtered value into ValueToken as ValueToken is also used for readProperty() which should not accept a filtered value as the object of the property.

[harttle]

@rosomri thank you for the effort. Comments to your proposals:

1. Token shouldn't depend on templates. Thus GroupedExpressionToken shouldn't have property of type Filter
2. evalGroupedExpressionToken() is good by itself. Need investigate can we reuse FilteredValueToken so this procedure can be reused from Value#value().

3.,4. Same, let's see whether we can reuse.

[jg-rp]

Can we reuse FilteredValueToken and let readGroupOrRange() return FilteredValueToken | RangeToken? And FilteredValueToken can also be used to new Value(filteredValueToken) so it can be used in tag implementions with minor change.

Sounds good 👍 . We could even add an ExpressionToken (or similar) type alias:

export type ValueToken =
    RangeToken
    | LiteralToken
    | QuotedToken
    | PropertyAccessToken
    | NumberToken

export type ExpressionToken = ValueToken | FilteredValueToken;

There are quite a few places where ValueToken is used and would need to include FilteredValueToken (or similar), like HashToken, RangeToken, FilterArg. They should all probably allow arbitrary nesting of parenthesized expressions.

[rosomri]

@harttle and @jg-rp, thank you for the detailed architectural feedback again! I've implemented the suggested improvements in a draft PR to demonstrate the approach: #878

Changes in #878

1. Reused FilteredValueToken instead of GroupedExpressionToken (as suggested here)

• Simplified readGroupOrRange() to return FilteredValueToken | RangeToken
• Eliminated code duplication

2. Fixed layering violation (as noted here)

• Removed resolvedFilters?: Filter[] from tokens (tokens no longer depend on template layer)
• Added liquid reference to Context for runtime filter resolution (following @jg-rp's suggestion here)

3. Lazy filter resolution at render time (as noted here)

• Build Filter instances in evalFilteredValueToken() during rendering
• Removed all resolveGroupedExpressionFilters() calls from parse time

Results

✅ All 1537 tests pass
✅ Proper architectural layering maintained: tokens → render → templates
✅ Code simplified: -48 lines, removed duplicate class

Question

The implementation adds liquid to Context for filter resolution. Does this approach align with your architectural vision? Or would you prefer passing Liquid to the Tokenizer constructor instead (which would require changes to filter runtime tokenization support)?

I'd appreciate your feedback on #878 to ensure the architectural direction is correct before finalizing this feature. Thanks!

[jg-rp]

Added liquid reference to Context for runtime filter resolution

👍 Render time filter resolution is a valid approach. This is what Shopify/liquid does.

I've drafted #879 as a proof of concept for maintaining parse-time filter resolution. I'm not yet convinced it is the best approach, just an option to consider.

[harttle]

It’ll be good to avoid passing liquid instance to lower layers, if we can.
As previously FilteredValueToken also doesn’t check filter existence.