[cpd] Found match is not maximal

[jdupak]

Affects PMD Version: 6.55.0

void print_fence(short current_row, short fence_size) {
  short i = 0;
  if (current_row == 0 ||
      current_row == fence_size) { // checks for first and last row
    if (fence_size % 2 == 0) {     // checks if the fence is even
      for (; i < fence_size;
           i += 2) { // i += 2 because two characters are being printed
        printf("-|");
      }
    } else {
      printf("|");
      i++; // i++ because one character was printed
      for (; i < fence_size;
           i += 2) { // i += 2 because two characters are being printed
        printf("-|");
      }
    }
  } else {
    if (fence_size % 2 == 0) { // checks if the fence is even
      for (; i < fence_size;
           i += 2) { // i += 2 because two characters are being printed
        printf(" |");
      }
    } else {
      printf("|");
      i++; // i++ because one character was printed
      for (; i < fence_size;
           i += 2) { // i += 2 because two characters are being printed
        printf(" |");
      }
    }
  }
  printf("\n");
}

The for loop and if are also identical.

[jsotuyod]

This is actually not a false negative.

Even though the code is extremely similar, the literal strings " |" and "-|" are not identical, and therefore won't be part of the match.

If C/C++ supported --ignore-literals (which it currently doesn't), such differences could be optionally ignored to produce a larger match.

[jdupak]

Oh, I see it now. I got confused by my own visualization.

Anyway, could you give me some pointers, where in code the ignore literals is/should be handled. I could really use this so I will try to implement it myself.

[jsotuyod]

@jdupak sure! Any contributions are more than welcomed!

First of, you would need to declare support for it on

pmd/pmd-cpp/src/main/java/net/sourceforge/pmd/lang/cpp/CppLanguageModule.java

Lines 46 to 51 in c9926fb

	public LanguagePropertyBundle newPropertyBundle() {
	LanguagePropertyBundle bundle = super.newPropertyBundle();
	bundle.definePropertyDescriptor(CpdLanguageProperties.CPD_IGNORE_LITERAL_SEQUENCES);
	bundle.definePropertyDescriptor(CpdLanguageProperties.CPD_IGNORE_LITERAL_AND_IDENTIFIER_SEQUENCES);
	bundle.definePropertyDescriptor(CPD_SKIP_BLOCKS);
	return bundle;

By adding a line bundle.definePropertyDescriptor(CpdLanguageProperties.CPD_ANONYMIZE_LITERALS);

Then you would have to edit CppCpdLexer to firstly pick up the property value on the constructor, and then add a method override for processToken(TokenFactory tokenEntries, JavaccToken cppToken) to actually implement this.

Doing so basically requires first looking at cppToken.kind and determining if it is a literal of any kind (numbers, strings, etc.) and if so, overriding the image to a standard value for each literal kind (so all strings are seen as equals, all numbers are seen as equals, etc.).

Java implements exactly that here:

pmd/pmd-java/src/main/java/net/sourceforge/pmd/lang/java/cpd/JavaCpdLexer.java

Line 56 in c9926fb

protected void processToken(TokenFactory tokenEntries, JavaccToken javaToken) {

For ease of comprehension, these are the relevant bits of the Java implementation (as it also does other transformations):

    @Override
    protected void processToken(TokenFactory tokenEntries, JavaccToken javaToken) {
        String image = javaToken.getImage();

        if (ignoreLiterals && (javaToken.kind == JavaTokenKinds.STRING_LITERAL
                || javaToken.kind == JavaTokenKinds.CHARACTER_LITERAL
                || javaToken.kind == JavaTokenKinds.INTEGER_LITERAL
                || javaToken.kind == JavaTokenKinds.FLOATING_POINT_LITERAL)) {
            image = JavaTokenKinds.describe(javaToken.kind);
        }

        tokenEntries.recordToken(image, javaToken.getReportLocation());
    }

I believe this should be straight-forward given this context and detail, but let me know if you have any questions.