Template problem with nested curly braces and chameleon expressions.

[mauritsvanrees]

This is with Zope 4.5.1.
See community.plone.org where Marcel Liebischer reports this. I can confirm it, and see the same in the current core Plone 5.2 development buildout, which I think has some newer versions for template related packages.

Let me copy the original report here:

---

I ran into another problem with page templates and Plone 5.2.2.

It seems there are problems with nested curly braces and chameleon expressions. This could easily happen if you try to build a data-attribute with some JSON data in a page template.

Steps to reproduce:

<span data-test='{"foo": "${view/somemethod}"}'/>

-> does not work. Throws "zope.location.interfaces.LocationError .. (view object, 'somemethod)'"

<span data-test='"foo": "${view/somemethod}"'/>

-> works fine (but it's not valid JSON)

<span data-test='{"foo": "${python: view.somemethod()}"'/>

-> works fine

The issues does not exist in Plone 5.2.0.

[dataflake]

It would help if someone who knows Plone could help get past all those parts that have to do with Plone and present a test case with Zope only.

[mauritsvanrees]

Simply create a Zope instance, start it up, in the ZMI add a Page Template (I chose id test). Edit to have this in the body:

1. <span data-test='{"foo": "${template/id}"}'/>: fails with: zExceptions.NotFound: id}" - Expression: "template/id}"" - Filename: /test - Location: (line 17: col 28) - Arguments: template: here: context: container: nothing: None options: {'args': ()} root: request: modules: traverse_subpath: [] user: default: repeat: loop: {} target_language: None translate: .translate at 0x1085b3940> attrs: {'data-test': '{"foo": "${template/id}"}'}

2. <span data-test='"foo": "${template/id}"'/>: works (but is not valid JSON, which is what the original poster wanted). Result: <span data-test='"foo": "test"'/>.

3. <span data-test='{"foo": "${python: template.id}"}'/>: works fine. Result: <span data-test='{"foo": "test"}'/>.

The first one shows that path substitution in a string with extra curly braces fails.

The second one shows that path substitution in a normal string without curly braces works.

The third one shows that python substitution in a sting with curly braces works.

[dataflake]

Do you have a link to a file that tells me what the versions for Plone 5.2.0 and 5.2.2 are so I can find the actual differences?

[d-maurer]

Maurits van Rees wrote at 2020-10-30 08:52 -0700:

This is with Zope 4.5.1. See [community.plone.org](https://community.plone.org/t/plone-5-2-1-5-2-2-page-template-issues/12793/16?u=mauritsvanrees) where Marcel Liebischer reports this. I can confirm it, and see the same in the current core Plone 5.2 development buildout, which I think has some newer versions for template related packages. Let me copy the original report here: ---------- I ran into another problem with page templates and Plone 5.2.2. It seems there are problems with nested curly braces and chameleon expressions. This could easily happen if you try to build a data-attribute with some JSON data in a page template. Steps to reproduce: <span data-test='{"foo": "${view/somemethod}"}'/> -> does not work. Throws "zope.location.interfaces.LocationError .. (view object, 'somemethod)'"

`chameleon` uses a vulnerable heuristic to process interpolations: when it comes to something that looks like a potential interpolation, it starts with the longest following string which might be the interpolation and checks whether its parsing results in an error; without parsing error, it accepts this interpolation, otherwise, it tries the next shorter string. In the case above, `${view/somemethod}"}` does not cause an error during parsing but raises a location error at runtime. In my opinion, the basic problem comes from `chameleon`. However, the problem is much aggravated since Zope's page templates use `zope.tales` as default TALES expression engine. While the `chameleon` path expression has a very tight syntax (no special characters allowed in path components), that of `zope.tales` is very loose (almost everything is allowed in path components). As a consequence, the interpolation above will work with `chameleon.tales` but not with `zope.tales`. The loose path expression syntax of `zope.tales` allows to use path expressions for most general dict lookups (as most special characters are allowed). Tightening its syntax would restrict dict lookups to simple keys (keys without disallowed characters) -- and thereby break applications at other places.

[d-maurer]

Jens Vagelpohl wrote at 2020-10-30 09:01 -0700:

It would help if someone who knows Plone could help get past all those parts that have to do with Plone and present a test case with Zope only.

The provided example is not Plone specific. An interpolation of the form `${path}...}` where `...` can be almost everything (including `...` literally) will behave as described -- in a stock Zope template. I explained the behavior in my previous reply. A reliable solution would require a `chameleon` change. Special cases could be handled by somewhat tightening the syntax allowed for `zope.tales` path expressions. For example, the case above could be tackled by forbidding "}" as part of components in `zope.tales` path expressions. I already did something like this for `$` -- to allow for interpolations of the form `${<path1>} ... ${<path2>}`.

[mauritsvanrees]

Do you have a link to a file that tells me what the versions for Plone 5.2.0 and 5.2.2 are so I can find the actual differences?

Plone 5.2.0 and 5.2.1 will behave the same here, I expect, so let's take that one as the baseline.

• 5.2.1: https://dist.plone.org/release/5.2.1/versions.cfg
• 5.2.2: https://dist.plone.org/release/5.2.2/versions.cfg

The relevant difference is easy (but big):

• 5.2.1 uses Zope 4.1.3
• 5.2.2 uses Zope 4.5.1, including the template engine changes from 4.4 and the related fixes from 4.5.

[d-maurer]

Dieter Maurer wrote at 2020-10-30 19:36 +0100:

... A reliable solution would require a `chameleon` change. Special cases could be handled by somewhat tightening the syntax allowed for `zope.tales` path expressions. For example, the case above could be tackled by forbidding "}" as part of components in `zope.tales` path expressions. I already did something like this for `$` -- to allow for interpolations of the form `${<path1>} ... ${<path2>}`.

`$` is handled by the last `if` in `Products.PageTemplates.engine.MappedExpr.__init__`. It could be generalized to further characters (such as e.g. `}`) - effectively excluding those characters from being useable in path expressions.

[d-maurer]

I have implemented the approach sketched in the previous comment in #927

[mauritsvanrees]

Yes, the merged PR works. The previously failing code from my first example is now properly handled without error.
Thank you!