New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Issue #3573: Resolve design problem of FileContentsHolder (step 1) #4468
Conversation
@timurt
Please, read http://checkstyle.sourceforge.net/idea.html#Inspections and run the inspections locally.
@romani @rnveach |
import java.util.SortedSet; | ||
|
||
/** | ||
* Represents the file results. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
file processing results
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
@@ -41,7 +39,7 @@ | |||
/** | |||
* Notify all listeners about the errors in a file. | |||
* @param fileName the audited file | |||
* @param errors the audit errors from the file | |||
* @param fileProcessingResult the audit errors from the file |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please, change javadoc
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
@@ -308,12 +309,14 @@ private void processFiles(List<File> files) throws CheckstyleException { | |||
* @throws CheckstyleException if error condition within Checkstyle occurs. | |||
* @noinspection ProhibitedExceptionThrown | |||
*/ | |||
private SortedSet<LocalizedMessage> processFile(File file) throws CheckstyleException { | |||
private FileProcessingResult processFile(File file) throws CheckstyleException { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
method javadoc should be corrected
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
@@ -374,13 +377,13 @@ public void fireFileStarted(String fileName) { | |||
* Notify all listeners about the errors in a file. | |||
* | |||
* @param fileName the audited file | |||
* @param errors the audit errors from the file | |||
* @param fileProcessingResult the audit errors from the file |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
javadoc is not correct. Should be fixed.
* Returns the contents of the file. | ||
* @return an object containing the full text of the file | ||
*/ | ||
public FileText getText() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The name of the method is cryptic. What does 'text' mean?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
renamed to fileText
cb726f6
to
bc0c0c3
Compare
Codecov Report
@@ Coverage Diff @@
## master #4468 +/- ##
======================================
Coverage 100% 100%
======================================
Files 287 288 +1
Lines 15395 15410 +15
Branches 3502 3503 +1
======================================
+ Hits 15395 15410 +15
Continue to review full report at Codecov.
|
@MEZk
|
config/checkstyle_checks.xml
Outdated
@@ -350,7 +350,7 @@ | |||
DetailsAST, CheckstyleException, UnsupportedEncodingException, BuildException, ConversionException, FileNotFoundException, TestException"/> | |||
</module> | |||
<module name="ClassFanOutComplexity"> | |||
<property name="max" value="25"/> | |||
<property name="max" value="26"/> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No, it is not allowed to change the config. There should be good reasons to change the config for all project. Please, revert.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
but otherwise my PR won't pass CI
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done, got error
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@timurt
Please, create an issue. Explain, what your problem is there. Make a suppression, put a link to the issue into suppression file.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@MEZk
Done
@Test | ||
public void testNonNullFileText() throws IOException { | ||
final String charsetName = "ISO-8859-1"; | ||
final FileText fileText = new FileText(new File("src/test/resources/com/puppycrawl/tools/" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The test should have its own input file.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
final String stripped = CommonUtils.relativizeAndNormalizePath(basedir, fileName); | ||
boolean hasNonFilteredViolations = false; | ||
for (final LocalizedMessage element : errors) { | ||
for (final LocalizedMessage element : fileProcessingResult.getMessages()) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please, extract fileProcessingResult.getMessages() to the local variable
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
@timurt CI should be green for us to merge. You must suppress this error for the class. |
@rnveach |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- Please, fix new points.
- Rename commit message to
Issue #3573: resolve design problem of FileContentsHolder (step 1)
to clearly indicate that this commit and PR are only the first step.
final String stripped = CommonUtils.relativizeAndNormalizePath(basedir, fileName); | ||
boolean hasNonFilteredViolations = false; | ||
for (final LocalizedMessage element : errors) { | ||
final SortedSet<LocalizedMessage> messages = fileProcessingResult.getMessages(); | ||
for (final LocalizedMessage element : messages) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- remove final modifier
- rename element to something more appropriate
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
'auditErrors' name is ok?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
remove final modifier
is it about final SortedSet<LocalizedMessage> messages = fileProcessingResult.getMessages();
?
when I remove it, verification fails with Variable 'messages' should be declared final
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@timurt
This line: for (final LocalizedMessage element : messages) {
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
public void testNonNullFileText() throws IOException { | ||
final String charsetName = "ISO-8859-1"; | ||
final FileText fileText = new FileText(new File("src/test/resources/com/puppycrawl/tools/" | ||
+ "checkstyle/api/InputFileProcessingResult.java"), charsetName); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You do not assert the content of the file directly in the test. Do we really need a separate file? Can we use temp folder and create the empty file there it it satisfied coverage threshold?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
@@ -174,6 +174,6 @@ protected final void fireErrors(String fileName) { | |||
final SortedSet<LocalizedMessage> errors = messageCollector | |||
.getMessages(); | |||
messageCollector.reset(); | |||
getMessageDispatcher().fireErrors(fileName, errors); | |||
getMessageDispatcher().fireErrors(fileName, new FileProcessingResult(null, errors)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
as we need to pass null as the constructor parameter it means that this is a problem in the design. Please, create one more constructor which receives one parameter and sets fileText to null. The constructor can be reused in the FileProcessingResult itself.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
@timurt |
* | ||
* @param messages the sorted set of messages | ||
*/ | ||
public FileProcessingResult(SortedSet<LocalizedMessage> messages) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reuse this constructor in the previous one
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not previous constructor in this one?
Like this or not?
public FileProcessingResult(SortedSet<LocalizedMessage> messages) {
this(null, messages);
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
final String stripped = CommonUtils.relativizeAndNormalizePath(basedir, fileName); | ||
boolean hasNonFilteredViolations = false; | ||
for (final LocalizedMessage element : errors) { | ||
final SortedSet<LocalizedMessage> auditErrors = fileProcessingResult.getMessages(); | ||
for (LocalizedMessage element : auditErrors) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Rename element to errorMessage, errors to errorMessages. We loop over messages, not audit events
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
|
||
public class FileProcessingResultTest { | ||
@Rule | ||
public TemporaryFolder temporaryFolder = new TemporaryFolder(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is it public?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
http://junit.org/junit4/javadoc/4.12/org/junit/Rule.html
Annotates fields that reference rules or methods that return a rule. A field must be public, not static, and a subtype of TestRule (preferred) or MethodRule. A method must be public, not static, and must return a subtype of TestRule (preferred) or MethodRule.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The filed should have been final. Now, I'm OK with the change.
@@ -0,0 +1,5 @@ | |||
package com.puppycrawl.tools.checkstyle.api; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We do not need this file anymore, please remove
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
@timurt |
@MEZk done |
@rnveach |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
public FileText getFileText() { | ||
FileText result = null; | ||
if (fileText != null) { | ||
result = new FileText(fileText); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do we need a new instance? FileText
is almost immutable except for lineBreaks
, which is just populated on demand.
Can't we return the field as is?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@rnveach
Cobertura fails when I just return value, as the example I took FileContents#getText method
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@timurt Please explain how cobertura is failing with just return fileText;
.
There are no branches or such, and we know the method is already being executed.
Is coverage failing in another class?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@timurt
This point is not addressed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@MEZk
done
final AuditEvent event = new AuditEvent(this, stripped, element); | ||
final SortedSet<LocalizedMessage> errorMessages = fileProcessingResult.getMessages(); | ||
for (LocalizedMessage errorMessage : errorMessages) { | ||
final AuditEvent event = new AuditEvent(this, stripped, errorMessage); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Step 2
6. Add FileContents to audit event to make it available for filters. We will add AST root to FileContents later
@MEZk Shouldn't the event eventually accept FileProcessingResult
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@rnveach
Maybe, but this should not be done in this PR. I'll think about it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@rnveach
You are right. AuditEvent should receive FileProcessingResult. Changed the plan.
#3573 (comment)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok.
* | ||
* @param messages the sorted set of messages | ||
*/ | ||
public FileProcessingResult(SortedSet<LocalizedMessage> messages) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@MEZk I think we should eventually remove this constructor as it doesn't contain the file contents. We are always working with a file, so I can't see why it shouldn't be populated (aside from any exception on file).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@rnveach
Yes, you are right. We always can instantiate FileText based on the file name and pass error messages and FileText to the constructor with two parameters. I did not see this fact. Hovewer, it will require to catch IOException from FileText constructor.
@timurt
Please, remove the constructor with one parameter. You always have the file path (file name) to instantiate FileText object and pass it to the constructor. So, both FileText object and the sorted set of localized messages can be instantiated.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it will require to catch IOException from FileText constructor.
@MEZk I'm not sure where you think IOException
is coming from. Both constructors don't have an IOException being thrown right now and I am not talking about creating FileText
in the constructor.
FileText
is already created in TreeWalker https://github.com/checkstyle/checkstyle/pull/4468/files#diff-503cc093d3169abd5697cc8189f27f32R319 and passed around. We should be using that FileText
everywhere somehow.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Both constructors don't have an IOException being thrown right now
FileText constructor throws IOException.
We will need to create FileText here. The constructor of FileText which receives the File object throws IOException.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@MEZk We pass FileText
to all FileSets at https://github.com/checkstyle/checkstyle/pull/4468/files/8da878eaf14710f1d578ecadfa1caafe94512dc3#diff-503cc093d3169abd5697cc8189f27f32R321 . So TreeWalker
should be able to hand it over to AbstractCheck
without having to create a new one.
Since the class is still List<String>
outside Checker, it depends on #3034 being done first.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@rnveach
DO we need 'file' as the first parameter of the process method? We already have the reference to the file in FileText.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@MEZk No we don't need it, but to accomplish this we would have to send the direct class FileText
to all parties as they only receive List<String>
as described in #3034.
It also seems slightly weird we would go to FileText
for the file's name. we will soon also have FileProcessingResult
with a lot of file related stuff.
Should we create a single class like CheckstyleFile
. It will contain the file name, FileProcessingResult, FileText, and everything else related to the input and output of the process
method? We could move the unnecessary fields from FileText
into CheckstyleFile
.
If so, let's create another issue on this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The new issue was created #4527.
related to the input and output of the process method
Why do we need to include the output of the process method into CheckstyleFile? Can you explain in the issue ?
FileProcessingResult represents results (localized messages and information about the processed file). It can include CheckstyleFile which can be an argument of the process method.
*/ | ||
@Override | ||
public void fireErrors(String fileName, SortedSet<LocalizedMessage> errors) { | ||
public void fireErrors(String fileName, FileProcessingResult fileProcessingResult) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@MEZk Should fileName
be in FileProcessingResult
?
It seems like they are fully conntected. FileText
can also retrieve the File
object, which can get it's name too.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@MEZk @rnveach
I'm stuck with one thing. Inside Checker#processFile, there is try-catch for IOException
in a case specified file does not exist, it means message will be logged, but fileText
will be null. If FileText
object is null then we will not be able to extract fileName
, absolutePath
and etc. from it, for example inside Checker#fireErrors
it will throw NPE
.
Problem is fileName
is used in various places even if the file does not exist
I can suggest to add additional filePath
field to FileProcessingResult
or left PRed implementation?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If FileText object is null then we will not be able to extract fileName, absolutePath and etc.
fileName
will be a separate field inside FileProcessingResult
, imo. Then we won't have to worry when FileText
is null.
@MEZk |
@MEZk |
@timurt |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@timurt
Please, suppress the TC violation. CheckerTest is a test class, it should be allowed to have many dependencies in the class.
new Object[] {"arg"}, null, getClass(), null)); | ||
final FileProcessingResult processingResult = new FileProcessingResult( | ||
file.getAbsolutePath(), fileText, messages); | ||
assertEquals(charsetName, processingResult.getFileText().getCharset().name()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we use just assertNotNull(processingResult.getFileText()) ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I inserted this assert before assertEquals(charsetName, processingResult.getFileText().getCharset().name());
I see no reason to delete previous assert
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do we need
assertEquals(charsetName, processingResult.getFileText().getCharset().name());
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
no actual reason, just to assert charset didn't change
I can remove it
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@timurt
Please, remove. It is unnecessary. Judging by the name of the test the main idea is to check that FileText is not null when it is passed to FileProcessingResult.
@MEZk
|
e6a0c57
to
b082e33
Compare
please reference name of inspection at https://github.com/checkstyle/checkstyle/blob/master/config/intellij-idea-inspections.xml#L1996 |
799cf44
to
d8b5b75
Compare
config/intellij-idea-inspections.xml
Outdated
@@ -1997,6 +1997,29 @@ | |||
<inspection_tool class="SuperClassHasFrequentlyUsedInheritors" enabled="true" level="ERROR" enabled_by_default="true" /> | |||
<inspection_tool class="SuperTearDownInFinally" enabled="false" level="ERROR" enabled_by_default="false" /> | |||
<inspection_tool class="SuppressionAnnotation" enabled="true" level="ERROR" enabled_by_default="true"> | |||
<scope name="Tests" level="ERROR" enabled="true"> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why was the scope changed?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@MEZk
my previous tries were not successful so I tried to create another scope
I am just out of ideas
this is my previous configuration
<inspection_tool class="SuppressionAnnotation" enabled="true" level="ERROR" enabled_by_default="true">
<option name="myAllowedSuppressions">
<list>
<option value="deprecation" />
<option value="unchecked" />
<option value="rawtypes" />
<!-- There is no other way to deliver filename that was under processing.
See https://github.com/checkstyle/checkstyle/issues/2285-->
<option value="ProhibitedExceptionThrown" />
<option value="MismatchedQueryAndUpdateOfCollection" />
<!-- Till https://github.com/checkstyle/checkstyle/issues/3066 -->
<option value="ProhibitedExceptionCaught" />
<!-- No way to split apart huge if/else branches in test. -->
<option value="IfStatementWithTooManyBranches" />
<!-- we have to catch Exception in Checker and rethrow it -->
<option value="ProhibitedExceptionThrown" />
<!-- we have to use it when pass null argument in test purporses -->
<option value="NullArgumentToVariableArgMethod" />
<!-- we have to use it because number of dependencies of CheckerTest exceeds 33. -->
<option value="ClassWithTooManyDependencies" />
</list>
</option>
</inspection_tool>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can use scope for the certain option to make less changes to the config.
See https://github.com/timurt/checkstyle/blob/d8b5b759e859f66c8c5e932bce7eb5906e626912/config/intellij-idea-inspections.xml#L967
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@MEZk
Is it correct?
<inspection_tool class="ClassWithTooManyDependencies" enabled="true" level="ERROR" enabled_by_default="true">
<scope name="Tests" level="ERROR" enabled="true">
<!-- we have to use it because number of dependencies of CheckerTest exceeds 33. -->
<option name="limit" value="40" />
</scope>
<option name="limit" value="33" />
</inspection_tool>
@SuppressWarnings("ClassWithTooManyDependencies")
public class CheckerTest extends BaseCheckTestSupport {
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
scope Production
means src/main?
scope Tests
means src/test ?
562cedd
to
9f264b9
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry, that I did not find time to review your intentions far before.
Sorry, I am against this update:
<scope name="Tests" level="ERROR" enabled="true"> | ||
<!-- we have to use it because number of dependencies of CheckerTest exceeds 33. --> | ||
<option name="limit" value="40" /> | ||
</scope> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if there is problem in one file for whole rule, we do not change a rule, we do suppression in certain class.
please use javadoc tag suppression for CheckerTest.
*/ | ||
@Override | ||
public void fireErrors(String fileName, SortedSet<LocalizedMessage> errors) { | ||
public void fireErrors(FileProcessingResult fileProcessingResult) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fileProcessingResult.getFileText
is not used it is already sign smth is wrong in design.
please see my comment at #3573 (comment)
@timurt @rnveach @romani |
it is better to close this PR |
#3573