Stubunit #3

Bohdankm22 · 2017-07-27T17:26:06Z

No description provided.

… declarations before the package declaration.

…s in implemented in the CompilationUnit - Implemented VoidVisitorAdapter#visit method for StubUnit to visit all the compilation units StubUnit has.

…n a stub file.

…n exception with a message indicating that it is not implemented.

… the VoidVisitor of the javaparser.

…ub files as well as to the class. This change is require because stub files could contain additional package declarations and it is valid to have annotations on package in Java.

…gning new empty list of Annotations.

- Added looking ahead while starting parsing a compilation unit because it is possible that CompilationUnit consists only with package declaration without any classes/interfaces/enums. - Removed redundant code that no more needed after adding LOOKAHEAD(PackageDeclaration())

…mpilationUnit contains only package declaration; - Added support for top-level static interfaces in the stub files; - Adjusted javaparser-test to not give an error for top-level interfaces.

…nto stubunit

- Changed LOOKAHEAD expecting PackageDeclaration

…3 tokens ahead.

- Added ability to pass receiver annotations to the method;

…ly pass receiver annotations.

…o stubunit

…ame.

…ge is needed for stub files without package declarations.

…o stubunit

mernst

This looks good overall. Here are a few comments, with a couple things to fix before it is merged.

mernst · 2017-07-27T18:49:02Z

javaparser-core/src/main/java/com/github/javaparser/ast/body/MethodDeclaration.java

@@ -158,7 +161,15 @@ public MethodDeclaration setName(final SimpleName name) {
    }

    @Override
-    public MethodDeclaration setParameters(final NodeList<Parameter> parameters) {
+    public MethodDeclaration setParameters(NodeList<Parameter> parameters) {
+        if (parameters != null && parameters.size() > 0 && parameters.get(0).getNameAsString().equals("this")) {


Would it be possible to put these changes in CallableDeclaration.setParameters? Then there would be no need to make the change to the constructor, above on lines 93 and 97.

mernst · 2017-07-27T18:51:41Z

javaparser-core/src/main/java/com/github/javaparser/ast/body/MethodDeclaration.java

+            // Removing receiver parameter as it should only be added to pass the annotations for receiver
+            parameters.remove(0);
+        }
+        if (parameters == null) {


I don't see why this test is necessary. Your code does not set parameters to null, and neither does the existing JavaParser. Remove it if it cannot be executed.

mernst · 2017-07-27T18:53:02Z

javaparser-core/src/main/java/com/github/javaparser/ast/validator/chunks/ModifierValidator.java

@@ -49,7 +49,7 @@ public void visit(ClassOrInterfaceDeclaration n, ProblemReporter reporter) {

    private void validateClassModifiers(ClassOrInterfaceDeclaration n, ProblemReporter reporter) {
        if (n.isTopLevelType()) {
-            validateModifiers(n, reporter, PUBLIC, ABSTRACT, FINAL, STRICTFP);
+            validateModifiers(n, reporter, PUBLIC, ABSTRACT, STATIC, FINAL, STRICTFP);


Why is static permitted on a top-level type?
If the old Stub Parser permitted that, it was probably to handle inner classes, which the new Stub Parser should handle correctly.
You might be able to drop the special handling of static here and elsewhere.

The new stubparser does not treat inner classes that declared on the top-level (e.g Runtime$One).
Possible resolutions could be to treat all classes with "$" in a name as inner.

I was hoping that we could change the stub file format to be the same as a Java file. That is, instead of a stub file containing

public class Outer { ... } public class Outer$Inner { ... }

the stub file would contain

public class Outer { ... public class Inner { ... } }

Is that possible?

It is definitely not possible in java.jj. But it is possible to add a code that will go through the StubUnit after it is parsed and look for classes that contain "$" in names and try put it inside another if found the match for outer class. But this will happen only after the stub file is already parsed into the StubUnit.
Should I work on it?

Are you saying that java.jj does not handle nested/inner classes? Or that your changes to java.jj break java.jj's handling of nested/inner classes?

We would like every Java file to be a legal stub file. That goal is not possible if stub files prohibit nested/inner classes.

It does handle nested/inner classes if they written inside the outer classes.
In the case when the class is written in the top-level, but with name Outer$Inner it is treated as a top-level class.

Here is an example of this in the Checker Framework.
https://github.com/Bohdankm22/checker-framework/blob/master/checker/jtreg/nullness/issue824/First.astub
Inner interfaces are written at the top level. Same way static nested classes could be declared.
Here are two ways of the issues resolution:

Handle static classes/interfaces at the top-level (already implemented)

After parsing a stub file look for the inner/nested classes/interfaces at the top-level and try to move it inside outer classes.

Example of the nested static class written in top-level in the Checker Framework
https://github.com/Bohdankm22/checker-framework/blob/master/framework/src/org/checkerframework/common/basetype/flow.astub#L103

mernst · 2017-07-27T18:55:05Z

javaparser-core/src/main/java/com/github/javaparser/ast/visitor/SimpleVoidVisitor.java

@@ -0,0 +1,337 @@
+package com.github.javaparser.ast.visitor;


I don't see this new class used anywhere by the Stub Parser. Can you delete the file?

This is used in https://github.com/Bohdankm22/checker-framework/blob/ec72c7ca8a3ccc2178df46a5478d4e7b867aa36f/framework/src/org/checkerframework/framework/stub/StubUtil.java#L9

mernst · 2017-07-27T18:58:22Z

...rc/main/java/com/github/javaparser/printer/lexicalpreservation/LexicalPreservingPrinter.java

@@ -443,7 +443,7 @@ private static ObservableProperty findNodeListName(NodeList nodeList) {
            if (m.getParameterCount() == 0 && m.getReturnType().getCanonicalName().equals(NodeList.class.getCanonicalName())) {
                try {
                    Object raw = m.invoke(parent);
-                    if (!(raw instanceof NodeList)) {
+                    if (raw != null && !(raw instanceof NodeList)) {


If this could never be null in the JavaParser, write a short comment about why it can be null in the Stub Parser.

mernst · 2017-07-27T21:29:01Z

javaparser-core/src/main/javacc/java.jj

@@ -711,36 +711,46 @@ StubUnit StubUnit():
    ModifierHolder modifier;
    TypeDeclaration tn = null;
    ModuleDeclaration module = null;
+    NodeList<AnnotationExpr> annotations = new NodeList<AnnotationExpr>();


It would be helpful to write a brief comment saying what each of these two variables represents.

mernst · 2017-07-27T21:40:08Z

javaparser-core/src/main/javacc/java.jj

 }
 {
    ( LOOKAHEAD(2) ";" )*
-    [ LOOKAHEAD(PackageDeclaration()) pakage = PackageDeclaration() ]
+    /*
+     * Reading leading imports to support the old stub files that contain import declarations


Is this still a documented, supported feature of the stub file format? I think it is, in which case this comment should be updated.
If it's not a documented feature of the stub file format, then remove the feature from the code.

mernst · 2017-07-27T21:47:26Z

javaparser-core/src/main/javacc/java.jj

+    (LOOKAHEAD(3, PackageDeclaration())
+        (
+             {
+                cu = new CompilationUnit(range(token_source.getHomeToken(), token()), pakage, imports, types, module);


Write a comment that this is the previous compilation unit, which has been completely read already.

mernst · 2017-07-27T21:48:52Z

javaparser-core/src/main/javacc/java.jj

+                module = null;
+             }
+             pakage = PackageDeclaration()
+             (in = ImportDeclaration() { imports = add(imports, in); })*


What happens if two different compilation units within a single stub file have different, conflicting import statements? That is, one imports a.b.C and the other imports d.e.C.

All the imports found in a stub file are collected into a single list of imports. Each of the compilations units has the reference to the list.

I understand that. I'm asking what happens if the combined list contains conflicts that none of the individual lists would have.
(Maybe nothing bad happens? Or maybe it does, and there should be a global list for imports before the first package and there should be a local list for each compilation unit, and a compilation unit gets the concatenation of those two rather than the concatenation of every import that precedes it in the file.)

According to the Checker Framework manual, all imports declaration should be at the beginning of a stub file.
https://checkerframework.org/manual/#stub
Other imports readings were added because some of the legacy stub files have imports declarations between the compilation units.
I am not sure how does it affect the conflict.
However, in the old stubparser, it also collects all the imports into a single list.
It could be left like that or I can remove redundant imports readings and change all the legacy stub files to declare imports only at the beginning of files.

Since it is easy to do, let's continue permitting import statements throughout the stub file. Please update the spec.
Leave the implementation as is, and if there is a problem in the future, we'll deal with it then.

…odDeclaration.

… list of receiver annotations to the variable. - Changed the condition in the LexicalPreservingPrinter#findNodeListName

- Adjusted the comments on the parser StubUnit parse.

- Added back tests to avoid top-level static classes/interfaces.

…in multilevel

mernst

Looks good. Please address my comments, then merge.

mernst · 2017-08-02T12:50:45Z

javaparser-core/src/main/java/com/github/javaparser/ast/body/MethodDeclaration.java

@@ -158,11 +158,6 @@ public MethodDeclaration setName(final SimpleName name) {
    }

    @Override
-    public MethodDeclaration setParameters(final NodeList<Parameter> parameters) {


Why is this code deleted?

This code was added before to handle receiver annotations. Then it was moved to the super class setParameters method.

mernst · 2017-08-02T13:02:49Z

javaparser-core/src/main/java/com/github/javaparser/ast/type/ArrayType.java

+        // parts of the array rather than two sources of information about the same parts of the
+        // array.
+        if (arrayBracketPairLists.length > 0) {
+            type.setArrayAnnotations(arrayBracketPairLists[0]);


Why does this ignore everything but the outermost set of array brackets?
Does this pass tests that you have written that contain multi-dimensional arrays with annotations on various levels?

mernst · 2017-08-02T13:04:54Z

javaparser-core/src/main/java/com/github/javaparser/ast/type/Type.java

@@ -146,4 +152,27 @@ public boolean replace(Node node, Node replacementNode) {
        }
        return super.replace(node, replacementNode);
    }
+
+    public List<AnnotationExpr> getAnnotationsAtLevel(int level) {


Every new method needs Javadoc, even if not all of JavaParser is properly documented. For example, what are the legal arguments? (Why is -1 legal for level?) What are the possible return values (it might return null).
Should the method throw an exception if level is greater than the number of array dimensions?

mernst · 2017-08-02T13:10:14Z

javaparser-core/src/main/javacc/java.jj

+    /*
+     * Stub file format described in the Checker Framework Manual:
+     * https://checkerframework.org/manual/#stub-format .
+     * All imports must be at the beginning of the file.


Delete the next three lines; it's adequate to refer to the manual.
Should we relax the restriction that all imports must be at the beginning of the file?
I think you have already done this, in which case you can delete the restriction from the Checker Framework manual.

mernst · 2017-08-02T13:14:22Z

javaparser-core/src/main/javacc/java.jj

+        |
+        (
+            {
+                modifier = null;


Why is this necessary, given that 5 lines later modifier is also set?

Matozoid retry modifier node

Fix #3 Add Test unit

Update with all changes in javaparser master

Bohdankm22 added 23 commits June 25, 2017 13:03

Added support for the legacy stub files that contains leading imports…

a91de3a

… declarations before the package declaration.

- Added implementation to the accept methods in StubUnit in the way a…

d4b2a75

…s in implemented in the CompilationUnit - Implemented VoidVisitorAdapter#visit method for StubUnit to visit all the compilation units StubUnit has.

- Added support of multiple static class and interface declarations i…

6b3a405

…n a stub file.

Implemented PrettyPringVisitor for StubUnit.

3693d79

Implemented visit method in CloneVisitor for StubUnit.

5b05fd1

Changed GenericListVisitorAdapter visit method on StubUnit to throw a…

ea8a216

…n exception with a message indicating that it is not implemented.

- Transplanted SimpleVoidVisitor from the stubparser and adjusted for…

8ef4562

… the VoidVisitor of the javaparser.

- adjusted parsing annotations that could belong to the package in st…

765aa12

…ub files as well as to the class. This change is require because stub files could contain additional package declarations and it is valid to have annotations on package in Java.

- Fixed the problem with annotations list assigned null. Instead assi…

17bbc8b

…gning new empty list of Annotations.

- Redesigned the way of reading StubUnits. Now it is possible that Co…

47fa716

…mpilationUnit contains only package declaration; - Added support for top-level static interfaces in the stub files; - Adjusted javaparser-test to not give an error for top-level interfaces.

- Added required for the Checker Framework visit method on ReferenceType

5249bd7

Merge branch 'stubunit' of https://github.com/Bohdankm22/javaparser i…

93f1cd3

…nto stubunit

- Remove redundant code.

484f7af

- Changed LOOKAHEAD expecting PackageDeclaration

- Fixed issue with LOOKAHEAD on PackageDeclaration. Changed to check …

76a553c

…3 tokens ahead.

- Added receiver annotations to the MethodDeclaration;

57c9f84

- Added ability to pass receiver annotations to the method;

- Removing receiver parameter from method declaration as it should on…

a2b09f3

…ly pass receiver annotations.

Merge branch 'master' of https://github.com/javaparser/javaparser int…

4ab8b8f

…o stubunit

- Fixed the problem with NullPointerException in method findNodeListN…

978ba7d

…ame.

- Changed to make a first package declaration not required. This chan…

20d0c83

…ge is needed for stub files without package declarations.

- Added toString to the NodeList class.

c4eb808

Merge branch 'master' of https://github.com/javaparser/javaparser int…

be3c7aa

…o stubunit

Merge branch 'master' into stubunit

6442670

Bohdankm22 closed this Jul 27, 2017

Bohdankm22 reopened this Jul 27, 2017

- Added merge fixes.

632f551

mernst requested changes Jul 27, 2017

View reviewed changes

Bohdankm22 added 3 commits July 27, 2017 18:37

- Moved receiverAnnotations list to the CallableDeclaration from Meth…

5b58958

…odDeclaration.

- In case of absence of receiver annotations changed to set the empty…

d448f61

… list of receiver annotations to the variable. - Changed the condition in the LexicalPreservingPrinter#findNodeListName

- Removed redundant variable declaration;

79b328b

- Adjusted the comments on the parser StubUnit parse.

Bohdankm22 and others added 6 commits July 27, 2017 21:00

- Reformatted a java.jj file

e80acd2

- Removed ability to have top-level static classes/interfaces.

fb40bd0

- Added back tests to avoid top-level static classes/interfaces.

- Added support of getting annotations for particular level of array …

6667c9a

…in multilevel

Tweak comments

936b8ca

Don't refer to section number, which might change.

257bf40

Tweak comment

279dcd6

mernst approved these changes Aug 2, 2017

View reviewed changes

- Addressed requested changes in the pull request #3.

8451c98

Bohdankm22 merged commit 4758265 into master Aug 3, 2017

Bohdankm22 deleted the stubunit branch August 3, 2017 00:10

mernst referenced this pull request in mernst/stubparser Jan 7, 2019

Merge pull request #3 from ftomassetti/matozoid-retry_modifier_node

d58e0f9

Matozoid retry modifier node

smillst referenced this pull request in smillst/stubparser Apr 12, 2020

Merge pull request #4 from myhelmisaari/mathieu

90898d3

Fix #3 Add Test unit

mernst referenced this pull request in mernst/stubparser Nov 11, 2020

Merge pull request #3 from javaparser/master

34f04eb

Update with all changes in javaparser master

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Stubunit #3

Stubunit #3

Bohdankm22 commented Jul 27, 2017

mernst left a comment

mernst Jul 27, 2017

mernst Jul 27, 2017

mernst Jul 27, 2017

Bohdankm22 Jul 28, 2017

mernst Jul 28, 2017

Bohdankm22 Jul 28, 2017

mernst Jul 28, 2017

Bohdankm22 Jul 28, 2017

Bohdankm22 Jul 28, 2017

Bohdankm22 Jul 28, 2017

mernst Jul 27, 2017

Bohdankm22 Jul 27, 2017

mernst Jul 27, 2017

mernst Jul 27, 2017

mernst Jul 27, 2017

mernst Jul 27, 2017

mernst Jul 27, 2017

Bohdankm22 Jul 28, 2017

mernst Jul 28, 2017

Bohdankm22 Jul 28, 2017

mernst Jul 28, 2017

mernst left a comment

mernst Aug 2, 2017

Bohdankm22 Aug 2, 2017

mernst Aug 2, 2017

mernst Aug 2, 2017

mernst Aug 2, 2017

mernst Aug 2, 2017

Stubunit #3

Stubunit #3

Conversation

Bohdankm22 commented Jul 27, 2017

mernst left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mernst left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment