[MCOMPILER-333] Clean generatedSourcesDirectory along with outputDirectory #23

marquiswang · 2023-04-20T22:22:34Z

The generatedSourcesDirectory (by default
target/generated-sources/annotations) contains source code generated by annotation processors. These generated sources are also outputs of the compiler, and thus should be cleaned along with the class files in the outputDirectory, in order to correctly do incremental compilations.

If this isn't done, then javac may not correctly compile the generated code, since it was there before. See
https://docs.oracle.com/en/java/javase/17/docs/specs/man/javac.html#annotation-processing

See https://github.com/marquiswang/incremental-compile-dagger-test for a reproducing example.

slawekjaranowski

As it is separate component we need separate jira issue 😄 to correctly build a release notes.

I know that unit tests are missing here ... can you try add first one for your change?

elharo · 2023-04-21T11:23:43Z

src/main/java/org/apache/maven/shared/incremental/IncrementalBuildHelper.java

    /**
     * Once the {@link #beforeRebuildExecution(org.apache.maven.shared.incremental.IncrementalBuildHelperRequest)} got
     * called, this will contain the list of files in the build directory.
     */
    private String[] filesBeforeAction = new String[0];

+    /**
+     * Once the {@link #beforeRebuildExecution(org.apache.maven.shared.incremental.IncrementalBuildHelperRequest)} got


After the {@link #beforeRebuildExecution(org.apache.maven.shared.incremental.IncrementalBuildHelperRequest)} is
* called, this contains the list of files in the build directory.

elharo · 2023-04-21T11:24:45Z

src/main/java/org/apache/maven/shared/incremental/IncrementalBuildHelper.java

+    }
+
+    /**
+     * Set the DirectoryScanner which shall get used by this build helper.


which shall get used by this build helper --> that scans the target/generated sources directory

Oops yeah just copy/pasted the javadocs. I will update them.

elharo · 2023-04-21T11:26:22Z

src/main/java/org/apache/maven/shared/incremental/IncrementalBuildHelper.java

+     * or create new a DirectoryScanner if none is yet set.
+     * The DirectoryScanner is used for detecting changes in a directory
+     */
+    public DirectoryScanner getGeneratedSourcesDirectoryScanner()


Do these methods need to be public? It looks like they can be package protected/default access

These are copy-pasted from the getDirectoryScanner(). It isn't clear to me why that one is public. I made this one public to be consistent.

elharo · 2023-04-21T11:27:23Z

src/main/java/org/apache/maven/shared/incremental/IncrementalBuildHelper.java

+     * Set the DirectoryScanner which shall get used by this build helper.
+     * @param generatedSourcesDirectoryScanner
+     */
+    public void setGeneratedSourcesDirectoryScanner( DirectoryScanner generatedSourcesDirectoryScanner )


I'm not sure why these methods (or this class for that matter) is needed. It seems a directory scanner could be easily created without it.

But I agree entirely with you that this class seems pointless. But I have no idea who uses it so it doesn't seem safe to change? Or would you prefer that I delete/inline this class.

Though this class is literally the only thing in maven-shared-incremental.

Don't change existing public methods in this PR, but also don't compound problems by introducing new ones we don't need.

elharo · 2023-04-21T11:28:53Z

src/main/java/org/apache/maven/shared/incremental/IncrementalBuildHelper.java

-            throw new MojoExecutionException( "Error while storing the mojo status", e );
-        }
+        writeChangedFiles(
+                getDirectoryScanner(),


These getter methods don't seem to help. They're an unnecessary layer of indirection. Just use the constructor directly.

So yeah I was just going for consistency with the existing API!

That only makes sense if the existing API is correct. Consistency with bad code is not a goal.

elharo · 2023-04-21T11:29:28Z

src/main/java/org/apache/maven/shared/incremental/IncrementalBuildHelper.java


        // in case of clean compile the file is not created so next compile won't see it
        // we mus create it here
+        File mojoConfigFile;


declare on same line as initialize

elharo · 2023-04-21T11:30:02Z

src/main/java/org/apache/maven/shared/incremental/IncrementalBuildHelper.java

+            DirectoryScanner directoryScanner,
+            String[] filesBeforeAction,
+            File mojoConfigBase,
+            String createdFilesLstFilename ) throws MojoExecutionException


Lst --> List

elharo · 2023-04-21T11:30:17Z

src/main/java/org/apache/maven/shared/incremental/IncrementalBuildHelper.java

+    private static void writeChangedFiles(
+            DirectoryScanResult outputDirectoryScanResult,
+            File mojoConfigBase,
+            String createdFilesLstFilename )


Lst --> List

elharo · 2023-04-21T11:31:45Z

src/main/java/org/apache/maven/shared/incremental/IncrementalBuildHelper.java

+            throws IOException
+    {
+        File createdFiles = new File( mojoConfigBase, createdFilesLstFilename );
+        FileUtils.fileWriteArray( createdFiles, outputDirectoryScanResult.getFilesAdded() );


This method is deprecated. Use

java.nio.files.Files.write(file.toPath(), data.getBytes(encoding), StandardOpenOption.CREATE)

and specify UTF-8 encoding

elharo · 2023-04-21T11:33:02Z

src/main/java/org/apache/maven/shared/incremental/IncrementalBuildHelper.java

+
+    private static String[] deleteFiles( File fileNameIndex, File parent ) throws IOException
+    {
+        String[] oldFiles = FileUtils.fileReadArray( fileNameIndex );


deprecated. use java.nio.files.Files.readAllLines() instead and specify UTF-8

marquiswang · 2023-04-21T15:05:15Z

As it is separate component we need separate jira issue 😄 to correctly build a release notes.

Should that still be MCOMPILER?

I know that unit tests are missing here ... can you try add first one for your change?

Uhhhhhhhh... I'm not sure it is practical for me to write the unit tests for this file. If you are able to whip together a quick one I can add one for this change?

…ctory The generatedSourcesDirectory (by default target/generated-sources/annotations) contains source code generated by annotation processors. These generated sources are also outputs of the compiler, and thus should be cleaned along with the class files in the outputDirectory, in order to correctly do incremental compilations. If this isn't done, then javac may not correctly compile the generated code, since it was there before. See https://docs.oracle.com/en/java/javase/17/docs/specs/man/javac.html#annotation-processing See https://github.com/marquiswang/incremental-compile-dagger-test for a reproducing example.

slachiewicz · 2023-06-06T06:30:18Z

@elharo please review again

elharo · 2023-06-06T11:39:00Z

src/main/java/org/apache/maven/shared/incremental/IncrementalBuildHelper.java

@@ -61,17 +69,28 @@
    private MavenProject mavenProject;

    /**
-     * Used for detecting changes between the Mojo execution.
+     * Used for detecting changes in the output directory between the Mojo execution.


something's off with "between the Mojo execution". possibly should be "between Mojo executions" or perhaps something else is meant?

elharo · 2023-06-06T11:39:28Z

src/main/java/org/apache/maven/shared/incremental/IncrementalBuildHelper.java

+    private DirectoryScanner directoryScanner = new DirectoryScanner();
+
+    /**
+     * Used for detecting changes in the generated sources directory between the Mojo execution.


elharo · 2023-06-06T11:39:52Z

src/main/java/org/apache/maven/shared/incremental/IncrementalBuildHelper.java

    /**
     * Once the {@link #beforeRebuildExecution(org.apache.maven.shared.incremental.IncrementalBuildHelperRequest)} got
     * called, this will contain the list of files in the build directory.
     */
    private String[] filesBeforeAction = new String[0];

+    /**
+     * Once the {@link #beforeRebuildExecution(org.apache.maven.shared.incremental.IncrementalBuildHelperRequest)} got


elharo · 2023-06-06T11:41:35Z

src/main/java/org/apache/maven/shared/incremental/IncrementalBuildHelper.java

-            throw new MojoExecutionException( "Error while storing the mojo status", e );
-        }
+        writeChangedFiles(
+                getDirectoryScanner(),


no need to call a getter method in the same class, just use the directoryScanner field directly

elharo · 2023-06-06T11:42:40Z

src/main/java/org/apache/maven/shared/incremental/IncrementalBuildHelper.java

+            DirectoryScanner directoryScanner,
+            String[] filesBeforeAction,
+            File mojoConfigBase,
+            String createdFilesListFileName ) throws MojoExecutionException


This is a constant in the same class so no need to pass it as an argument

elharo · 2023-06-06T11:52:08Z

src/main/java/org/apache/maven/shared/incremental/IncrementalBuildHelperRequest.java

+
+    public void setGeneratedSourcesDirectory( File generatedSourcesDirectory )
+    {
+        this.generatedSourcesDirectory = generatedSourcesDirectory;


Is it worth checking here that the file exists and is a directory? Throw an IllegalArgumentException if not.

elharo · 2023-06-06T11:52:17Z

src/main/java/org/apache/maven/shared/incremental/IncrementalBuildHelperRequest.java

+
+    public IncrementalBuildHelperRequest generatedSourcesDirectory( File generatedSourcesDirectory )
+    {
+        this.generatedSourcesDirectory = generatedSourcesDirectory;


Is it worth checking here that the file exists and is a directory?

elharo · 2023-06-06T11:53:08Z

src/main/java/org/apache/maven/shared/incremental/IncrementalBuildHelperRequest.java

@@ -73,4 +75,20 @@ public IncrementalBuildHelperRequest outputDirectory( File outputDirectory )
        this.outputDirectory = outputDirectory;
        return this;
    }
+
+    public File getGeneratedSourcesDirectory()


needs Javadoc that clearly explains what this directory is.

elharo · 2023-06-06T11:53:15Z

src/main/java/org/apache/maven/shared/incremental/IncrementalBuildHelperRequest.java

+        return generatedSourcesDirectory;
+    }
+
+    public void setGeneratedSourcesDirectory( File generatedSourcesDirectory )


needs Javadoc

elharo · 2023-06-06T11:53:21Z

src/main/java/org/apache/maven/shared/incremental/IncrementalBuildHelperRequest.java

+        this.generatedSourcesDirectory = generatedSourcesDirectory;
+    }
+
+    public IncrementalBuildHelperRequest generatedSourcesDirectory( File generatedSourcesDirectory )


needs Javadoc that clearly explains what this directory is.

marquiswang force-pushed the master branch 3 times, most recently from 22d7211 to 3eef81a Compare April 20, 2023 23:19

slawekjaranowski requested changes Apr 21, 2023

View reviewed changes

elharo requested changes Apr 21, 2023

View reviewed changes

marquiswang force-pushed the master branch from 3eef81a to 975c3ac Compare April 21, 2023 15:39

elharo requested changes Jun 6, 2023

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[MCOMPILER-333] Clean generatedSourcesDirectory along with outputDirectory #23

[MCOMPILER-333] Clean generatedSourcesDirectory along with outputDirectory #23

marquiswang commented Apr 20, 2023

slawekjaranowski left a comment

elharo Apr 21, 2023

elharo Jun 6, 2023

elharo Apr 21, 2023

marquiswang Apr 21, 2023

elharo Apr 21, 2023

marquiswang Apr 21, 2023

elharo Apr 21, 2023

marquiswang Apr 21, 2023

marquiswang Apr 21, 2023

elharo Apr 21, 2023

elharo Apr 21, 2023

marquiswang Apr 21, 2023

elharo Apr 21, 2023

elharo Apr 21, 2023

elharo Apr 21, 2023

marquiswang Apr 21, 2023

elharo Apr 21, 2023

marquiswang Apr 21, 2023

elharo Apr 21, 2023

elharo Apr 21, 2023

marquiswang commented Apr 21, 2023

slachiewicz commented Jun 6, 2023

elharo Jun 6, 2023

elharo Jun 6, 2023

elharo Jun 6, 2023

elharo Jun 6, 2023

elharo Jun 6, 2023

elharo Jun 6, 2023

elharo Jun 6, 2023

elharo Jun 6, 2023

elharo Jun 6, 2023

elharo Jun 6, 2023

[MCOMPILER-333] Clean generatedSourcesDirectory along with outputDirectory #23

Are you sure you want to change the base?

[MCOMPILER-333] Clean generatedSourcesDirectory along with outputDirectory #23

Conversation

marquiswang commented Apr 20, 2023

slawekjaranowski left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

marquiswang commented Apr 21, 2023

slachiewicz commented Jun 6, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment