ECJ out of sync with JLS 9.6.4.1 #1097

srikanth-sankaran · 2023-05-26T10:06:24Z

What it does

Reverse the changes made for https://bugs.eclipse.org/bugs/show_bug.cgi?id=552082 as a first step
To sync up with JLS 9.6.4.1 by a subsequent commit to fix #1096

How to test

Author checklist

I have thoroughly tested my changes
The change is following the coding conventions
I have signed the Eclipse Contributor Agreement (ECA)

https://bugs.eclipse.org/bugs/show_bug.cgi?id=552082

srikanth-sankaran · 2023-05-30T10:32:12Z

@stephan-herrmann, this backs out your code and also advances it forward - May I request you to glance through the changes ? TIA.

See comments beginning with #1092 (comment) for background although it turned out #1092 is completely orthogonal

stephan-herrmann · 2023-05-30T14:41:46Z

@stephan-herrmann, this backs out your code and also advances it forward - May I request you to glance through the changes ? TIA.

See comments beginning with #1092 (comment) for background although it turned out #1092 is completely orthogonal

I was a bit confused by the network of JDK bugs, and about how to figure out which solution replaces which previous solutions, so I collected the changes directly from JLS:

JLS 8: "... T is applicable in all declaration contexts except type parameter declarations, and in no type contexts. "
JLS 14: "... T is applicable in all nine declaration contexts and in all 16 type contexts."
JLS 17+: "... A is applicable in all declaration contexts and in no type contexts."

Going from 8 to 17+ would have been easy. With the detour via 14 (which javac skipped?) more effort is needed, so from an outside look, backing out changes done for 14 sounds like the natural thing to do.

My next question would be: should ecj follow any of this back and forth based on compliance settings, or should we assume the 17+ variant is simply the best and should be supported for all compliance levels?

I'll take an inside look, too.

stephan-herrmann

No objections, just some questions for further clarification, opportunities for (future) cleanup etc.

org.eclipse.jdt.core.compiler.batch/src/org/eclipse/jdt/internal/compiler/ast/Annotation.java

stephan-herrmann · 2023-05-30T16:12:58Z

org.eclipse.jdt.core.compiler.batch/src/org/eclipse/jdt/internal/compiler/ast/Annotation.java

-		if ((metaTagBits & (TagBits.AnnotationTargetMASK)) != 0) {
-			if ((metaTagBits & (TagBits.AnnotationForTypeParameter | TagBits.AnnotationForTypeUse)) == 0) {
+		if ((metaTagBits & (TagBits.AnnotationTargetMASK)) == 0) { // In the absence of explicit target, applicable only to declaration sites
+			if (targetType != AnnotationTargetTypeConstants.CLASS_TYPE_PARAMETER && targetType != AnnotationTargetTypeConstants.METHOD_TYPE_PARAMETER)


In my naive understanding, methods isRuntimeTypeVisible and isRuntimeTypeInvisible would only cover codegen for type annotations (declaration scenarii being covered by isRuntimeVisible/isRuntimeInvisible).
If that were true then parameter targetType would be unnecessary, as in absence of a @Target we can directly answer false, can't we?

The only test that disagrees is the new testGH1096(), challenging a no-target annotation on a type parameter.

Does this mean, annotations on type parameters are handled through the TYPE_USE-related code, rather than like "normal" declaration annotations?

Should we schedule a follow-up issue, to reconsider this?

In my naive understanding, methods isRuntimeTypeVisible and isRuntimeTypeInvisible would only cover codegen for type annotations (declaration scenarii being covered by isRuntimeVisible/isRuntimeInvisible). If that were true then parameter targetType would be unnecessary, as in absence of a @Target we can directly answer false, can't we?

Consider this program:

import java.lang.annotation.ElementType; import java.lang.annotation.Target; @Target(ElementType.TYPE_USE) @interface TA { } @TA public class X { }

which reaches

SingleMemberAnnotation(Annotation).isRuntimeTypeInvisible(int) line: 777 ClassFile.generateRuntimeAnnotations(Annotation[], long) line: 4581 ClassFile.addAttributes() line: 423 TypeDeclaration.generateCode(ClassFile) line: 772 TypeDeclaration.generateCode(CompilationUnitScope) line: 831

and org.eclipse.jdt.core.tests.compiler.regression.TypeAnnotationTest.testGH1096()

which reaches the call stack:

MarkerAnnotation(Annotation).isRuntimeTypeInvisible(int) line: 777 TypeReference$AnnotationCollector.internalVisit(Annotation) line: 197 TypeReference$AnnotationCollector.visit(MarkerAnnotation, BlockScope) line: 244 MarkerAnnotation.traverse(ASTVisitor, BlockScope) line: 43 TypeParameter.getAllAnnotationContexts(int, int, List<AnnotationContext>) line: 72 ClassFile.completeMethodInfo(MethodBinding, int, int) line: 2455 MethodDeclaration(AbstractMethodDeclaration).generateCode(ClassFile) line: 377 MethodDeclaration(AbstractMethodDeclaration).generateCode(ClassScope, ClassFile) line: 292 TypeDeclaration.generateCode(ClassFile) line: 761

In both scenarios the method is supposed to return true.

What is a suitable value for the parameter targetType of the method
org.eclipse.jdt.internal.compiler.ast.Annotation.isRuntimeTypeInvisible(int) for the first call (stack) ??

(Despite it not being relevant - the parameter gets "touched" only when no target is specified) I simply invented a value that is outside of all possible type annotation target locations enumerated by table 4.7.20 of the JVMS.

There are only three meaningful values that influence the method's behavior:
targetType == AnnotationTargetTypeConstants.CLASS_TYPE_PARAMETER
targetType == AnnotationTargetTypeConstants.METHOD_TYPE_PARAMETER and
targetType being anything else - For that -1 given the symbolic name NonJVMS4_7_20_TargetLocation was my choice.

Better ideas welcome.

The only test that disagrees is the new testGH1096(), challenging a no-target annotation on a type parameter.

Does this mean, annotations on type parameters are handled through the TYPE_USE-related code, rather than like "normal" declaration annotations?

Should we schedule a follow-up issue, to reconsider this?

....compiler/src/org/eclipse/jdt/core/tests/compiler/regression/RecordsRestrictedClassTest.java

stephan-herrmann · 2023-05-30T16:28:34Z

...ts.compiler/src/org/eclipse/jdt/core/tests/compiler/regression/RepeatableAnnotationTest.java

@@ -1472,12 +1464,15 @@ public void testDeprecation2() { // verify that deprecation warning does not sho
 				"@interface T {\n" +
 				"  public int value() default -1;\n" +
 				"}\n" +
-				"interface I<@T(1) @T(2) K> {\n" +
+				"interface I extends @T(1) Runnable {\n" +


Why is the test input changed?

Just to retain it as a negative test. Otherwise the code would compile fine with the new behavior implemented

stephan-herrmann · 2023-05-30T16:29:57Z

...piler.batch/src/org/eclipse/jdt/internal/compiler/codegen/AnnotationTargetTypeConstants.java

@@ -19,6 +19,8 @@

 public interface AnnotationTargetTypeConstants {

+	int NonJVMS4_7_20_TargetLocation         = -1;


I was a bit at loss, where in JVMS exactly I should look for an explanation...

I was a bit at loss, where in JVMS exactly I should look for an explanation...

Per above, I need a sentinel value for the targetType parameter of org.eclipse.jdt.internal.compiler.ast.Annotation.isRuntimeTypeInvisible(int) method that is outside the legal range of values for type annotation target locations enumerated by JVMS Table 4.7.20 -

Better suggestions welcome

Rather than a lot of hand waving (where I couldn't be sure whether or not I was talking bullshit), I experimented with disentangling how we handle type annotations and declaration annotations as much as possible. Please find my result in PR #1117 - which is created on top of this PR but pushed separately as not to interfere with your work here.

The key connection to this constant is: the invocations where the new constant was used could be removed altogether :)

WDYT?

I was a bit at loss, where in JVMS exactly I should look for an explanation...

I have in the latest additional push, gotten rid of this head-scratch producing enumeration.

Rather than a lot of hand waving (where I couldn't be sure whether or not I was talking bullshit), I experimented with disentangling how we handle type annotations and declaration annotations as much as possible. Please find my result in PR #1117 - which is created on top of this PR but pushed separately as not to interfere with your work here.

The key connection to this constant is: the invocations where the new constant was used could be removed altogether :)

WDYT?

I will review this - I suggest let us do this in a follow up ticket - (I'll raise one)

srikanth-sankaran · 2023-05-31T04:26:37Z

I was a bit confused by the network of JDK bugs, and about how to figure out which solution replaces which previous solutions, so I collected the changes directly from JLS:

JLS 8: "... T is applicable in all declaration contexts except type parameter declarations, and in no type contexts. "

JLS 14: "... T is applicable in all nine declaration contexts and in all 16 type contexts."

JLS 17+: "... A is applicable in all declaration contexts and in no type contexts."

Going from 8 to 17+ would have been easy. With the detour via 14 (which javac skipped?) more effort is needed, so from an outside look, backing out changes done for 14 sounds like the natural thing to do.

Yes, javac never resolved https://bugs.openjdk.org/browse/JDK-8231436 which is still open that corresponds to the spec bug https://bugs.openjdk.org/browse/JDK-8231435 in JDK14 (which was superseded subsequently in JDK17)

My next question would be: should ecj follow any of this back and forth based on compliance settings, or should we assume the 17+ variant is simply the best and should be supported for all compliance levels?

IMO, it is best to skip supporting past behavior - it is not the case that a valid program ceases to be valid or changes meaning in some subtle manner - which would call for preserving old behavior with a compliance check.

eclipse-jdt#1097 (comment)

https://bugs.eclipse.org/bugs/show_bug.cgi?id=568240

https://bugs.eclipse.org/bugs/show_bug.cgi?id=566803

stephan-herrmann · 2023-06-01T15:25:00Z

My next question would be: should ecj follow any of this back and forth based on compliance settings, or should we assume the 17+ variant is simply the best and should be supported for all compliance levels?

IMO, it is best to skip supporting past behavior - it is not the case that a valid program ceases to be valid or changes meaning in some subtle manner - which would call for preserving old behavior with a compliance check.

sounds good to me.

NonJVMS4_7_20_TargetLocation.

@t

clean-up on top of PR #1097 + reduce mixed handling of type/declaration annotations - when generating decl annos don't care about visibility as type anno + remove redundant filtering of decl annos + remove interim NonJVMS4_7_20_TargetLocation + replace SE7AnnotationTargetMASK with AnnotationForDeclarationMASK + clarify that AnnotationCollector does not traverse type parameter + a few clarifying code comments + update two tests in RepeatableAnnotationTest: - test026 extend the error message - testUnspecifiedTarget2 don't expect conflict, @t applies to all decls Co-authored-by: Srikanth Sankaran <srikanth.sankaran@advantest.com>

@t

clean-up on top of PR eclipse-jdt#1097 + reduce mixed handling of type/declaration annotations - when generating decl annos don't care about visibility as type anno + remove redundant filtering of decl annos + remove interim NonJVMS4_7_20_TargetLocation + replace SE7AnnotationTargetMASK with AnnotationForDeclarationMASK + clarify that AnnotationCollector does not traverse type parameter + a few clarifying code comments + update two tests in RepeatableAnnotationTest: - test026 extend the error message - testUnspecifiedTarget2 don't expect conflict, @t applies to all decls Co-authored-by: Srikanth Sankaran <srikanth.sankaran@advantest.com>

* Reverse the changes made for https://bugs.eclipse.org/bugs/show_bug.cgi?id=552082 * Tweak for test failures in Jenkins * Implement the recent changes to JLS 9.6.4.1 * Incorporate review comment eclipse-jdt#1097 (comment) * Additional test case for https://bugs.eclipse.org/bugs/show_bug.cgi?id=568240 * Additional regression test for https://bugs.eclipse.org/bugs/show_bug.cgi?id=566803 * Review follow up: Get rid of the head-scratch producing enumerator NonJVMS4_7_20_TargetLocation.

@t

clean-up on top of PR eclipse-jdt#1097 + reduce mixed handling of type/declaration annotations - when generating decl annos don't care about visibility as type anno + remove redundant filtering of decl annos + remove interim NonJVMS4_7_20_TargetLocation + replace SE7AnnotationTargetMASK with AnnotationForDeclarationMASK + clarify that AnnotationCollector does not traverse type parameter + a few clarifying code comments + update two tests in RepeatableAnnotationTest: - test026 extend the error message - testUnspecifiedTarget2 don't expect conflict, @t applies to all decls Co-authored-by: Srikanth Sankaran <srikanth.sankaran@advantest.com>

srikanth-sankaran added 3 commits May 26, 2023 15:33

Reverse the changes made for

49ecc60

https://bugs.eclipse.org/bugs/show_bug.cgi?id=552082

Tweak for test failures in Jenkins

8e7cc9e

Implement the recent changes to JLS 9.6.4.1

9828d8a

srikanth-sankaran requested a review from stephan-herrmann May 30, 2023 10:29

stephan-herrmann approved these changes May 30, 2023

View reviewed changes

Incorporate review comment

3f7146d

eclipse-jdt#1097 (comment)

srikanth-sankaran added this to the 4.29 M1 milestone Jun 1, 2023

srikanth-sankaran added 2 commits June 1, 2023 12:29

Additional test case for

079389a

https://bugs.eclipse.org/bugs/show_bug.cgi?id=568240

Additional regression test for

1ab9b23

https://bugs.eclipse.org/bugs/show_bug.cgi?id=566803

stephan-herrmann mentioned this pull request Jun 1, 2023

ECJ out of sync with JLS 9.6.4.1 #1117

Merged

Review follow up: Get rid of the head-scratch producing enumerator

f2c0d9a

NonJVMS4_7_20_TargetLocation.

srikanth-sankaran merged commit 03658fd into eclipse-jdt:master Jun 8, 2023
9 checks passed

srikanth-sankaran deleted the GH1096 branch June 8, 2023 06:24

ECJ out of sync with JLS 9.6.4.1 #1097

ECJ out of sync with JLS 9.6.4.1 #1097

srikanth-sankaran commented May 26, 2023 •

edited

srikanth-sankaran commented May 30, 2023

stephan-herrmann commented May 30, 2023

stephan-herrmann left a comment

stephan-herrmann May 30, 2023

srikanth-sankaran May 31, 2023

stephan-herrmann May 30, 2023

srikanth-sankaran May 31, 2023

stephan-herrmann May 30, 2023

srikanth-sankaran May 31, 2023

stephan-herrmann Jun 1, 2023

srikanth-sankaran Jun 5, 2023

srikanth-sankaran Jun 5, 2023

srikanth-sankaran commented May 31, 2023

stephan-herrmann commented Jun 1, 2023

		@@ -19,6 +19,8 @@

		public interface AnnotationTargetTypeConstants {

		int NonJVMS4_7_20_TargetLocation = -1;

ECJ out of sync with JLS 9.6.4.1 #1097

ECJ out of sync with JLS 9.6.4.1 #1097

Conversation

srikanth-sankaran commented May 26, 2023 • edited

What it does

How to test

Author checklist

srikanth-sankaran commented May 30, 2023

stephan-herrmann commented May 30, 2023

stephan-herrmann left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

srikanth-sankaran commented May 31, 2023

stephan-herrmann commented Jun 1, 2023

srikanth-sankaran commented May 26, 2023 •

edited