Add an option to exclude paths from being analyzed #599

kageiit · 2017-04-11T04:41:41Z

This adds an option to suppress some paths entirely from being analyzed by the error prone analyzer. This is particularly useful for generated code or third party code which consumers may not want to analyze.

Fixes #523

The input patterns are treated as regular expressions for path matching. For example:
-XepExcludedPaths:".*/build/generated/.*\.java" will exclude all java files inside any build/generated/ directories

kageiit · 2017-04-11T04:41:51Z

cc @msridhar

vanniktech · 2017-04-16T11:27:06Z

This would also fix #511

kageiit · 2017-06-07T20:04:46Z

@cushon ping on this

nick-someone · 2017-06-22T22:57:13Z

Hello! We discussed this a bit:

We believe that the combination of -XepDisableWarningsInGeneratedCode and separating the compilation of generated code into its own compiler invocation (perhaps without Error Prone enabled, or with Error Prone configured to turn off a select set of checks) may be a better long term idea.

The flag allows error prone to skip generating warnings for code annotated with @Generated, but will still break the build if the generated code violates the error-level patterns (i.e.: the stuff that would break your build if you wrote it by hand). We've selected the list of error-level checks based on our impression that these represent real issues that should be corrected.

If you find that the generated code is consistently violating these error-level checks, we may want to consider reclassifying the checks, or fixing the code generator if the code it's generation contains the buggy code.

kageiit · 2017-06-22T23:04:38Z

Hey @nglorioso This is for more than just generated code. There are cases where we have vendor/third party code that needs to be part of the same sourceset. Also, not all code generators add the @Generated annotation, so just the existing -XepDisableWarningsInGeneratedCode options will not work for those cases.

I think having an option like this would make it very flexible and let consumers suppress generated code issues etc. and also any other potential future use cases like during a migration when two modules are merged etc, to turn off EP on a particular path so it can be done incrementally etc. when dealing with large projects at scale

ZacSweers · 2017-09-11T03:27:42Z

Ping on this?

vladgurovich · 2017-10-12T01:07:37Z

Furthermore, this seems like something that would enable a gradual rollout of this tool on a legacy code that would otherwise generate hundreds of errors or warnings:
I can enable error-prone on a smaller subset of files/classes and gradually increase that subset as i fix the bugs that error-prone uncovers

cushon · 2017-11-01T22:44:35Z

check_api/src/main/java/com/google/errorprone/ErrorProneAnalyzer.java

+   */
+  private boolean isExcludedPath(URI uri) {
+    Pattern excludedPattern = errorProneOptions.getExcludedPattern();
+    return (excludedPattern != null) && excludedPattern.matcher(uri.toString()).matches();


Can you use ASTHelpers.getFileNameFromUri instead of URI.toString? The former has special handling for jar entries, which I think it's possible to see here.

kageiit · 2017-11-01T23:37:43Z

@cushon updated as per your suggestion

cushon · 2017-11-01T23:39:24Z

LGTM, but can you add test coverage?

This was leftover code that shouldn't have made it here

googlebot · 2017-11-03T02:59:50Z

So there's good news and bad news.

👍 The good news is that everyone that needs to sign a CLA (the pull request submitter and all commit authors) have done so. Everything is all good there.

😕 The bad news is that it appears that one or more commits were authored by someone other than the pull request submitter. We need to confirm that they're okay with their commits being contributed to this project. Please have them confirm that here in the pull request.

Note to project maintainer: This is a terminal state, meaning the cla/google commit status will not change from this State. It's up to you to confirm consent of the commit author(s) and merge this pull request when appropriate.

msridhar · 2017-11-03T03:00:27Z

I'm ok with my commits being included

cushon · 2017-11-03T03:13:44Z

check_api/src/main/java/com/google/errorprone/ErrorProneAnalyzer.java

      } else if (finishedCompilation(path.getCompilationUnit())) {
        // Otherwise this TaskEvent is for a ClassTree, and we can scan the whole
        // CompilationUnitTree once we've seen all the enclosed classes.
-        transformer.get().apply(new TreePath(compilation), subContext, descriptionListener);
+        if (!isExcludedPath(compilation.getSourceFile().toUri())) {


It would be nice to de-duplicate this logic.

cushon · 2017-11-03T03:14:46Z

check_api/src/main/java/com/google/errorprone/ErrorProneOptions.java

@@ -412,4 +435,12 @@ public static ErrorProneOptions processArgs(String[] args) {
    Preconditions.checkNotNull(args);
    return processArgs(Arrays.asList(args));
  }
+
+  private static Pattern getExcludedPattern(Iterable<String> excludeSubStrings) {


WDYT about making the flag take a regex instead of a comma-separated list that gets processed into a regex? Making it an arbitrary regex avoids the need to define what the syntax is.

Makes sense; will change

cushon · 2017-11-03T03:15:12Z

check_api/src/test/java/com/google/errorprone/ErrorProneOptionsTest.java

@@ -164,6 +166,20 @@ public void recognizesCompilingTestOnlyCode() {
  }

  @Test
+  public void recognizesExcludedPaths() {


Can you add an integration test that verifies the option is used to filter the compilation units that get analyzed?

Will look into it; is there an existing test whose structure I can copy?

There are test cases in ErrorProneJavaCompilerTest that use flags to enable/disable checks and change severities. Maybe something like that?

Cool, thanks! I added a test case there.

cushon · 2017-11-08T20:22:06Z

Thanks! I'm merging this change internally and it will show up on github in the next day or so.

Can you also create a PR to add documentation to http://errorprone.info/docs/flags?

msridhar · 2017-11-08T20:43:58Z

Awesome! Will create a documentation PR.

Fixes #599 RELNOTES: Add an option to exclude paths from being analyzed ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=175049602

vorburger · 2017-11-30T00:23:58Z

FTR: Not release as of right now, but #821 says about to...

googlebot added the cla: yes label Apr 11, 2017

tbroyer mentioned this pull request Jun 2, 2017

How to ignore Path '/build/generated/source/apt/debug/...' tbroyer/gradle-errorprone-plugin-v0.0.x#42

Closed

kageiit force-pushed the suppress_paths branch from e256757 to 35c5b26 Compare June 7, 2017 20:04

nick-someone mentioned this pull request Jun 23, 2017

Collecting ideas for a new set of configuration options #648

Open

cushon reviewed Nov 1, 2017

View reviewed changes

Gautam Korlam added 2 commits November 1, 2017 16:27

Add an option to exclude paths from being analyzed

5818c2c

Use ASTHelpers.getFileNameFromUri

51c422b

kageiit force-pushed the suppress_paths branch from 35c5b26 to 51c422b Compare November 1, 2017 23:37

msridhar added 2 commits November 2, 2017 19:58

Don't substitute for '.' in paths

e4a5dff

This was leftover code that shouldn't have made it here

add test

4019abf

googlebot added cla: no and removed cla: yes labels Nov 3, 2017

cushon reviewed Nov 3, 2017

View reviewed changes

msridhar added 2 commits November 2, 2017 20:54

address comments

28d7ede

remove extraneous blank lines in import blocks

3d5cc3c

msridhar mentioned this pull request Nov 9, 2017

add docs for -XepExcludedPaths #818

Merged

cushon closed this in 92254ee Nov 9, 2017

ronshapiro pushed a commit that referenced this pull request Nov 9, 2017

Add an option to exclude paths from being analyzed

0fbb326

Fixes #599 RELNOTES: Add an option to exclude paths from being analyzed ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=175049602

msridhar mentioned this pull request Nov 9, 2017

DisableErrorsInGeneratedCode #808

Open

ronshapiro pushed a commit that referenced this pull request Nov 9, 2017

Add an option to exclude paths from being analyzed

afbda33

Fixes #599 RELNOTES: Add an option to exclude paths from being analyzed ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=175049602

ronshapiro pushed a commit that referenced this pull request Nov 13, 2017

Add an option to exclude paths from being analyzed

8971f7e

Fixes #599 RELNOTES: Add an option to exclude paths from being analyzed ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=175049602

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add an option to exclude paths from being analyzed #599

Add an option to exclude paths from being analyzed #599

kageiit commented Apr 11, 2017

kageiit commented Apr 11, 2017

vanniktech commented Apr 16, 2017

kageiit commented Jun 7, 2017

nick-someone commented Jun 22, 2017

kageiit commented Jun 22, 2017

ZacSweers commented Sep 11, 2017

vladgurovich commented Oct 12, 2017 •

edited

Loading

cushon Nov 1, 2017

kageiit commented Nov 1, 2017

cushon commented Nov 1, 2017

googlebot commented Nov 3, 2017

msridhar commented Nov 3, 2017

cushon Nov 3, 2017

cushon Nov 3, 2017 •

edited

Loading

msridhar Nov 3, 2017

cushon Nov 3, 2017 •

edited

Loading

msridhar Nov 3, 2017

cushon Nov 3, 2017

msridhar Nov 3, 2017

cushon commented Nov 8, 2017

msridhar commented Nov 8, 2017

vorburger commented Nov 30, 2017

Add an option to exclude paths from being analyzed #599

Add an option to exclude paths from being analyzed #599

Conversation

kageiit commented Apr 11, 2017

kageiit commented Apr 11, 2017

vanniktech commented Apr 16, 2017

kageiit commented Jun 7, 2017

nick-someone commented Jun 22, 2017

kageiit commented Jun 22, 2017

ZacSweers commented Sep 11, 2017

vladgurovich commented Oct 12, 2017 • edited Loading

cushon Nov 1, 2017

Choose a reason for hiding this comment

kageiit commented Nov 1, 2017

cushon commented Nov 1, 2017

googlebot commented Nov 3, 2017

msridhar commented Nov 3, 2017

cushon Nov 3, 2017

Choose a reason for hiding this comment

cushon Nov 3, 2017 • edited Loading

Choose a reason for hiding this comment

msridhar Nov 3, 2017

Choose a reason for hiding this comment

cushon Nov 3, 2017 • edited Loading

Choose a reason for hiding this comment

msridhar Nov 3, 2017

Choose a reason for hiding this comment

cushon Nov 3, 2017

Choose a reason for hiding this comment

msridhar Nov 3, 2017

Choose a reason for hiding this comment

cushon commented Nov 8, 2017

msridhar commented Nov 8, 2017

vorburger commented Nov 30, 2017

vladgurovich commented Oct 12, 2017 •

edited

Loading

cushon Nov 3, 2017 •

edited

Loading

cushon Nov 3, 2017 •

edited

Loading