The "\>" delimeter is not rendering correctly

[thetamage]

I'm using the c# port of stringtemplate 4.0.6.9004.

Per the cheatsheet, stringtemplate4 is supposed to recognize \< and \> as escaped characters < and >. Escaping \< works properly but \> does not.

This template worked correctly in stringtemplate3 but in upgrading to stringtemplate4, we see that the string "\<\>" rendering yields "<\>".

[sharwell]

The C# port (which is actually in the repository https://github.com/antlr/antlrcs) appears to behave the same way as the reference implementation. The only two characters escaped are the expression start delimiter (< in your usage) and the character }.

stringtemplate4/src/org/stringtemplate/v4/compiler/STLexer.java

Lines 428 to 429 in 9cc59e1

	if ( input.LA(2)==delimiterStartChar ||
	input.LA(2)=='}' )

[peteruhnak]

The behavior is in fact regression between 4.0.4 and 4.0.7 (I imagine due to tweaks here https://github.com/antlr/stringtemplate4/blob/master/CHANGES.txt#L27 )

Can this be fixed so it is consistent?

I imagine that changing the escaping would break templates using 4.0.7+ (although people using 4.0.4 could finally upgrade), so perhaps it can be put behind some global configuration flag? (setAutoEscapeClosingDelimiter(false)).

Note that in 4.0.4 it was broken for ST, but not for STGFile nor STGString.

4.0.4:

test_escape_st
Expected :a BB <c> <d>
Actual   :a BB <c> <d\>

4.0.7:

test_escape_st
Expected :a BB <c> <d>
Actual   :a BB <c> <d\>
 

test_escape_stg_string
Expected :a BB <c> <d>
Actual   :a BB <c> <d\>

test_escape_stg_file
Expected :a BB <c> <d>
Actual   :a BB <c> <d\>

test code:

STGTest.java

package sandbox;

import org.junit.Test;
import org.stringtemplate.v4.ST;
import org.stringtemplate.v4.STGroupFile;
import org.stringtemplate.v4.STGroupString;

import static org.junit.Assert.assertEquals;

public class STGTest {

  @Test
  public void test_escape_st() {
    String s = "a <b> \\<c> \\<d\\>";

    ST template = new ST(s);
    String result = template.add("b", "BB").render();
    assertEquals("a BB <c> <d>", result);
  }

  @Test
  public void test_escape_stg_string() {
    String s = "base(b) ::= <<a <b> \\<c> \\<d\\>>>";

    STGroupString group = new STGroupString(s);
    ST template = group.getInstanceOf("base");
    String result = template.add("b", "BB").render();
    assertEquals("a BB <c> <d>", result);
  }

  @Test
  public void test_escape_stg_file() {
    STGroupFile group = new STGroupFile("template.stg");
    ST template = group.getInstanceOf("base");
    String result = template.add("b", "BB").render();
    assertEquals("a BB <c> <d>", result);
  }
}

template.stg

base(b) ::= <<
a <b> \<c> \<d\>
>>

[parrt]

Thanks for the detailed analysis. We'll look at this for next release.

[seanabraham]

Just to add to this, this issue is particularly devious in .stg files when the templates are wrapped in << >> because with this behavior you have still have to sometimes escape the > to prevent the occasional >> which would terminate the template early.

For example with the current behavior this happens:

Given value == "Foo",

• If you always escape >: \<<value>\> is correctly rendered as <Foo> but \<Bar\> incorrectly renders as <Bar\>.
• If you never escape >: \<Bar> correctly renders as <Bar> but \<<value>> terminates the template early and subsequent templates in the file can't be found.

This leaves only the option of conditionally escaping > only when necessary to prevent >>. This is pretty confusing and has has made upgrading ST4 pretty difficult since it required manually removing some of the \s from existing .stg files.

[akond]

Using [org.antlr/ST4 "4.3.4"] I get the same error:

(.render (ST. "a \\<newline\\> b"))
=> "a <newline\\> b"