[21] Processed string templates falsely contain backslash characters #1719

kriegaex · 2023-12-10T05:23:15Z

Tested with ecj-3.37.0-20231209.011330-8.jar, i.e. probably with a build based on commit a476dd1.

This works as expected:

/**
 * Examples taken from <a href="https://openjdk.org/jeps/430">JEP 430</a>
 */
public class StringPatternsPreview1 {
  public static void main(String[] args) {
    // Embedded expressions can be strings
    String firstName = "Bill", lastName = "Duck";
    System.out.println(STR."\{firstName} \{lastName}");

    // Embedded expressions can be used in multi-line strings
    String title = "My Web Page";
    String text = "Hello, world";
/*
    String html = STR."""
      <html>
        <head>
          <title>\{title}</title>
        </head>
        <body>
          <p>\{text}</p>
        </body>
      </html>
      """;
    System.out.println(html);
*/
  }
}

$ java -jar 'c:\Users\Alexa\.m2\repository\org\eclipse\jdt\ecj\3.37.0-SNAPSHOT\ecj-3.37.0-SNAPSHOT.jar' --enable-preview --release 21 src/main/java/StringPatternsPreview1.java
(...)

$ java --enable-preview -cp src/main/java StringPatternsPreview1
Bill Duck

Now activate the commented-out section and try again:

$ java -jar 'c:\Users\Alexa\.m2\repository\org\eclipse\jdt\ecj\3.37.0-SNAPSHOT\ecj-3.37.0-SNAPSHOT.jar' --enable-preview --release 21 src/main/java/StringPatternsPreview1.java
(...)

$ java --enable-preview -cp src/main/java StringPatternsPreview1
\Bill \Duck
<html>
  <head>
    <title>My Web Page</title>
  </head>
  <body>
    <p>Hello, world</p>
  </body>
</html>

See? Suddenly, there is a side effect of the second string template interpolation on the result of the first! Instead of Bill Duck it unexpectedly prints \Bill \Duck. I noticed this by chance when preparing a smoke test for the upcoming AspectJ version and immediately tested with upstream ECJ.

The text was updated successfully, but these errors were encountered:

TODO: Due to eclipse-jdt/eclipse.jdt.core#1719, one line per test is currently rendered incorrectly. After the upstream fix, change "\Bill \Duck" to "Bill Duck". Signed-off-by: Alexander Kriegisch <Alexander@Kriegisch.name>

kriegaex · 2023-12-10T07:06:55Z

BTW, if you are interested in a longer version of my test, it is here. Maybe you want a similar smoke test, just in case you do not have anything like that in place already.

The declarative version of my assertions on the expected output can be found in this file. There is also a TODO comment for the one line rendered incorrectly, pointing to this issue.

TODO: Due to eclipse-jdt/eclipse.jdt.core#1719, one line per test is currently rendered incorrectly. After the upstream fix, change "\Bill \Duck" to "Bill Duck". Signed-off-by: Alexander Kriegisch <Alexander@Kriegisch.name>

srikanth-sankaran · 2023-12-11T03:36:27Z

Thanks for the report. Reproduced. Bizarre to say the least. Investigating

srikanth-sankaran · 2023-12-11T04:18:01Z

Smallest test case that shows the problem:

public class X {
  public static void main(String[] args) {
    String name = "Bill";
    System.out.println(STR."\{name}");
    
    String html = STR."""
         \{name}
      """;
   
  }
}

Scanner seems to return wrong text fragments. Jay please take a look. TIA

srikanth-sankaran · 2023-12-11T07:15:12Z

As I mentioned here: #1517 (comment) : , it is a good practice to grab all the code snippets in the JEP document into a regression test. So we know there is basic coverage. Not late even now.

eclipse-jdt#1719

jarthana · 2023-12-12T04:36:31Z

The problem is the stale Scanner#textBlockOffset left by the previous calls to scanForTextBlock() via jumpOverBody(). Perhaps, we can avoid lot of things that we do for templates and embedded expressions if scanner#diet is true. But for this issue (and #1722), just resetting just as we enter scanForStringLiteral() seems sufficient.

As I mentioned here: #1517 (comment) : , it is a good practice to grab all the code snippets in the JEP document into a regression test. So we know there is basic coverage. Not late even now.

Most of the current tests in StringTemplateTest were taken from the JEP. I will do another pass if something is missing. In this case, it was the combination of two tests that caused the issue.

kriegaex · 2023-12-12T06:57:44Z

As I mentioned here: #1517 (comment) : , it is a good practice to grab all the code snippets in the JEP document into a regression test. So we know there is basic coverage.

I agree with you. I mentioned that several times before, e.g. in a comment on a JDK 19 issue more than a year ago , see #450 (comment):

Another potentially helpful idea is to always try to incorporate code examples from the corresponding JEPs into the test bed and compare the behaviour to Javac. Of course, that does not in any way replace implementing the JEP based on the JVM spec, but having a few obvious test cases might help to drive design and implementation and gain additional confidence before merging a beta branch into main.

Most of the current tests in StringTemplateTest were taken from the JEP. I will do another pass if something is missing. In this case, it was the combination of two tests that caused the issue.

That is true. It only happens when I accumulate several use cases in one test, in this case first a template processor with a normal string and then one with a multi-line string. So, it was not trivial to catch.

eclipse-jdt#1719

…1730) #1719

…clipse-jdt#1730) eclipse-jdt#1719

…1730) #1719

@SuppressWarnings

* configure default output.. = bin/ * JavaSearchScope: improve encloses() performance #474 * [test] remove outdated latestBREE project * fix some ecj markers Especially after moving files to compiler.batch - which has no resource warnings - the @SuppressWarnings("resource") is not used - leading to a marker * Single async "Synchronizing projects" Job #419 Scheduling multiple times "while the job is running, the job will still only be rescheduled once" (javadoc) so there will be always only a single job running. Using a Set prevents touching projects multiple times. #419 Manually tested by changing Compiler Building options "Circular dependencies", which triggers a new build. * [test] fix AbstractJavaModelTests #1333 avoid asynchronous refresh. Implementation taken from org.eclipse.core.tests.resources.refresh.RefreshProviderTest.joinAutoRefreshJobs() #1333 * TestVerifier: never wait 100ms improve test time, accurate timeout * [performance] ClassFileReader: use already open Zip File instead of creating new ZipFile instance. also: * use Files.readAllBytes * removed unused code * faster toUri avoiding isDirectory() check for the JARs which are known to be no directory improves performance of read() on windows by factor 2 tested with java reference search to java.lang.Object * Stop skipping compare-with-baseline for jdt.annotation v2 One less thing that has to be manually tracked and done. * Version bump for jdt.annotation Pointed by https://download.eclipse.org/eclipse/downloads/drops4/I20231130-0020/buildlogs/reporeports/reports/versionChecks.html * Fix github urls in NOTICE file * Use try-with-resource and enable warning if not org.eclipse.jdt.core.compiler.problem.explicitlyClosedAutoCloseable=warning StringWriter does not need flush() or close() and can not throw IOException. Tests excluded. To get rid of boiler plate code. * version bumps * [21] JEP 430 String Templates (#1513) Grammar, compiler AST, resolution and code generation changes * Javadoc: fix unclosed <code> * break from loop within labeled block causes loss of nullness info (#1660) fixes #1659 * Fixes incorrect Javadoc after StringBuffer to StringBuilder change * NPE in ASTRewriteFlattener as return value of GuardedPattern.getPattern() is null (#1647) NPE in ASTRewriteFlattener as return value of GuardedPattern.getPattern() is null * Adding pomless build to JDT core This enables pomless builds for JDT coreand removes the simple pom files. Future commits can reduce the usage of pom files further. This might require enhancements in pomless builds to specify the test classes and suites eclipse-tycho/tycho#3105 * Using Simplify lambda expression and method reference syntax cleanup on core Using the JDT UI "Simplify lambda expression and method reference syntax" clean-up on jdt.core. * Internal compiler error: ArrayIndexOutOfBoundsException in latest i build (#1664) fixes #1661 * Using Simplify lambda expression and method reference syntax cleanup on all plug-ins except core Using the JDT UI "Simplify lambda expression and method reference syntax" clean-up on all plug-ins except jdt.core. * Using short-circuit in IncrementalImageBuilder * Re-normalize line-endings in git of all files to Linux style ("\n") Some files were checked-in into git having windows style line endings (\r\n). This is in general not wanted because it can cause modified files without any difference in git-staging on Windows if auto-crlf is enabled. To re-normalize line endings of all files use the following command (including dot): git add --renormalize . * Bump bundle dependencies to trigger a rebuild / fix SDK build error The change in bytecode of FullSourceWorkspaceBuildTests is due the constant value change of o.e.jdt.internal.compiler.parser.TerminalTokens.TokenNameEOF coming from #1513. Fixes eclipse-platform/eclipse.platform.releng.aggregator#1617 * False positive "Dead code" compiler error reported on org.eclipse.pde.internal.core.util.PDEJavaHelper.getExternalPackageFragment(String, String) (#1671) fixes #1667 * Use diamond operator in jdt.core repo Using the JDT UI clean-up, this removes the redundant type information. Also activating org.eclipse.jdt.core.compiler.problem.redundantSpecificationOfTypeArguments=warning The rest of the changes in the prefs file were done by the tooling, not by selecting anther value. * Improve constructor completions inside method invocations Fix #1587 (#1588) * Try-with-resource clean-up in JDT core Using the JDT UI try-with-resource clean-up on core. Also manually inlining the declaration of multiple places into the try() clause. * Make inner classes static where possible in JDT core Running the JDT performnace clean-up "Make inner classes static where possible" in JDT core * Use Use lazy logical operator (&& and ||) Running the performance clean-up "Use lazy logical operator (&& and ||)" on JDT core * Using Integer.toString directly in Disassembler It is also a JDT UI performance clean-up but this clean-up found only one occurrence. * use Path.of() to avoid 'Potential resource leak' warnings * @NonNullByDefault does not work for type arguments of a local type (#1694) fixes #1693 * ClassCastException during code completion on Annotation (#1696) fixes #1440 Also fixes noise from #1662 * CompilationUnitResolver: Name the CU that causes Exception (#1690) for example during Cleanup eclipse-jdt/eclipse.jdt.ui#950 Co-authored-by: Jörg Kubitz <jkubitz-eclipse@gmx.de> * Code selection support for String template expressions (#1699) Fixes #1685 * Report error if string template is used without preview option enabled (#1697) Improves the fix for #544 * ECJ crashes when an embedded expression contains broken code (#1702) Set haltOnSyntaxError when parsing for embedded expressions. Also using the correct delimiters for text blocks in printExpression() methods. * Selection model tests for string templates (#1704) * tests: enable discouragedReference=warning, declare x-friends to get rid of warnings during build (does not respect the jdt preferences) * Javadoc format fixes Contributes to eclipse-platform/eclipse.platform.releng.aggregator#1531 * Performance: Add public API for Batch Reads in UI - closes #1614 During Batches: * cache Zip Files * enable JavaModelManager.temporaryCache Also: * uses Batch Reads during some core actions that benefit from it. * adds trace logging when caching is missing. #1614 * Javadoc format fixes (part 2) Contributes to eclipse-platform/eclipse.platform.releng.aggregator#1531. It's a pity that this takes multiple cycles but fixing one thing from the log uncovers the next. * ECJ 3.36.0 regression: The type 'E extends java.lang.Exception' is not a valid substitute for the type parameter 'E extends java.lang.@nonnull Exception' (#1708) fixes #1691 + also slightly updates NullAnnotationTests18 as NullAnnotationTests21 * Implement support for code completion inside embedded expression of (only) string templates (not text block templates) (#1712) * Implement support for code completion inside embedded expression of (only) string templates (not text block templates) * Fixes #1711 * Fixes #1641 (#1713) * Add new testcase #1701 (#1705) * Content assist does not propose overrides in records (#1718) * Fixes #1095 * JavaModelManager: lazy initialize TouchJob #1720 #1720 * Run the JavaCoreStandaloneTest in the build #1720 * Bogus error about return expression involving pattern matching (#1731) Fixes #1726 * Compiler fails to recognize an exhaustive switch (#1733) * Fixes #1725 * [21] Processed string templates falsely contain backslash characters (#1730) #1719 * 1641_enum_further_fixes (#1739) * Fix one too many pops arising from string concat invoke dynamic (#1740) * Fix one too many pops arising from string concat invoke dynamic * Fixes #1394 * Wrong placement of exception range closure results in AIOOB (#1744) Fixes #1686 * Improper warning - Potential null pointer access (#1469) Fixes #1461 Sets the flow info reach mode to FlowInfo.UNREACHABLE_BY_NULLANALYSIS after 'Object o = null;if (Objects.isNull(o)) return;', 'Object o = "";if (Objects.nonNull(o)) return; Signed-off-by: Snjezana Peco <snjezana.peco@redhat.com> * Touch bundles affected by the changed ecj version See #1394 See eclipse-platform/eclipse.platform.releng.aggregator#1659 * Upload eventfile and unit test results * 2023-06->2023-09 Seems to have broke dependency graph management in our project (#1698) * Add test for reproducing #1654 * Fix to make that test pass --------- Co-authored-by: Stephan Herrmann <stephan.herrmann@berlin.de> * [memory] SoftReference for ResourceCompilationUnit.contents #1743 Ability to reduce memory during searches that find many files #1743 * deduplicate "eclipse" #1743 CharDeduplication was not designed to deduplicate tokens with length 7+ which could lead to high memory consumption. With this change tokens of all sizes can be deduplicated. #1743 A benchmark implemented in CharDeduplicationTest.main(String[]) shows the new deduplication is performed at similar speed (.21s instead of .16s) but deduplicates much more tokens (99% instead of 36%). * 1703.constant definitions (#1756) * Fixes [21] AIOOB at switchStatement TNode.addPattern (#1757) org.eclipse.jdt.internal.compiler.ast.SwitchStatement $TNode.addPattern --------- Signed-off-by: Snjezana Peco <snjezana.peco@redhat.com> Co-authored-by: Jörg Kubitz <jkubitz-eclipse@gmx.de> Co-authored-by: Eric Milles <eric.milles@thomsonreuters.com> Co-authored-by: Александър Куртаков <akurtakov@gmail.com> Co-authored-by: Jay Arthanareeswaran <jarthana@in.ibm.com> Co-authored-by: Stephan Herrmann <stephan.herrmann@berlin.de> Co-authored-by: Lars Vogel <Lars.Vogel@vogella.com> Co-authored-by: Suby S Surendran <suby.surendran@ibm.com> Co-authored-by: Hannes Wellmann <wellmann.hannes1@gmx.net> Co-authored-by: Andrey Loskutov <loskutov@gmx.de> Co-authored-by: Gayan Perera <gayanper@gmail.com> Co-authored-by: Jörg Kubitz <51790620+jukzi@users.noreply.github.com> Co-authored-by: Srikanth Sankaran <131454720+srikanth-sankaran@users.noreply.github.com> Co-authored-by: Ed Merks <ed.merks@gmail.com> Co-authored-by: Snjeza <snjezana.peco@redhat.com> Co-authored-by: Christoph Läubrich <laeubi@laeubi-soft.de> Co-authored-by: Michael Haubenwallner <michael.haubenwallner@ssi-schaefer.com>

…clipse-jdt#1730) eclipse-jdt#1719

kriegaex · 2024-02-12T05:53:51Z

I just merged JDT Core upstream into the AspectJ fork. Now, it works as expected. Thank you, @jarthana.

eclipse-jdt/eclipse.jdt.core#1719 was fixed. Change test to expect the correct strings for processed string pattern. Signed-off-by: Alexander Kriegisch <Alexander@Kriegisch.name>

srikanth-sankaran mentioned this issue Dec 11, 2023

[String templates] Internal compiler error: java.lang.NegativeArraySizeException: -3 at Scanner.getCurrentTokenInRange(Scanner.java:525) #1722

Closed

srikanth-sankaran assigned jarthana Dec 11, 2023

srikanth-sankaran added this to the 4.31 M1 milestone Dec 11, 2023

jarthana added a commit to jarthana/eclipse.jdt.core that referenced this issue Dec 12, 2023

[21] Processed string templates falsely contain backslash characters

0d4afb0

eclipse-jdt#1719

jarthana mentioned this issue Dec 12, 2023

[21] Processed string templates falsely contain backslash characters #1730

Merged

3 tasks

jarthana added a commit to jarthana/eclipse.jdt.core that referenced this issue Dec 12, 2023

[21] Processed string templates falsely contain backslash characters

037039e

eclipse-jdt#1719

jarthana closed this as completed in #1730 Dec 12, 2023

jarthana added a commit that referenced this issue Dec 12, 2023

[21] Processed string templates falsely contain backslash characters (#…

2f4180a

…1730) #1719

mpalat pushed a commit to mpalat/eclipse.jdt.core that referenced this issue Dec 13, 2023

[21] Processed string templates falsely contain backslash characters (e…

b65ec7e

…clipse-jdt#1730) eclipse-jdt#1719

jarthana added a commit to jarthana/eclipse.jdt.core that referenced this issue Dec 13, 2023

[21] Processed string templates falsely contain backslash characters (e…

4638609

…clipse-jdt#1730) eclipse-jdt#1719

noopur2507 pushed a commit that referenced this issue Dec 15, 2023

[21] Processed string templates falsely contain backslash characters (#…

1a5c645

…1730) #1719

rgrunber pushed a commit to eclipse-jdtls/eclipse-jdt-core-incubator that referenced this issue Jan 9, 2024

[21] Processed string templates falsely contain backslash characters (e…

2db5335

…clipse-jdt#1730) eclipse-jdt#1719

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[21] Processed string templates falsely contain backslash characters #1719

[21] Processed string templates falsely contain backslash characters #1719

kriegaex commented Dec 10, 2023

kriegaex commented Dec 10, 2023 •

edited

Loading

srikanth-sankaran commented Dec 11, 2023

srikanth-sankaran commented Dec 11, 2023

srikanth-sankaran commented Dec 11, 2023

jarthana commented Dec 12, 2023

kriegaex commented Dec 12, 2023 •

edited

Loading

kriegaex commented Feb 12, 2024

[21] Processed string templates falsely contain backslash characters #1719

[21] Processed string templates falsely contain backslash characters #1719

Comments

kriegaex commented Dec 10, 2023

kriegaex commented Dec 10, 2023 • edited Loading

srikanth-sankaran commented Dec 11, 2023

srikanth-sankaran commented Dec 11, 2023

srikanth-sankaran commented Dec 11, 2023

jarthana commented Dec 12, 2023

kriegaex commented Dec 12, 2023 • edited Loading

kriegaex commented Feb 12, 2024

kriegaex commented Dec 10, 2023 •

edited

Loading

kriegaex commented Dec 12, 2023 •

edited

Loading