Add Java Properties language for .properties extension

[eager]

This pull adds a Java Properties language for .properties.

Description

I noticed that our some of our project’s .properties files had incorrect syntax highlighting after escaped #:

Because INI and Java Properties are very similiar—Codemirror’s properties mode appears to be a super-set of both—the disambiguation heuristic is somewhat complex. It replicates an outer-if block ([key]=[value]) that will match INI, unless the file also matches Java Properties’ comment markers (^[#!]), similar to the following code:

if /^[^#!;][^=]*=/.match(data)
  # Match all key=value
  if /^[;\[]/.match(data)
    # Match INI comment token or sections
    Language["INI"]
  elsif /^[#!]/.match(data)
    # Match Java comment tokens
    Language["Java Properties"]
  else
    # Assume key=value is INI, if no comment tokens
    Language["INI"]
  end
elsif /^[^#!][^:]*:/.match(data)
  # Match key:value
  Language["Java Properties"]
end

Checklist:

• I am adding a new language.

  • The extension of the new language is used in hundreds of repositories on GitHub.com.
    • Search results for each extension:
      • https://github.com/search?o=desc&q=extension%3Aproperties+size%3A%3E1000+and+NOT+jdk&s=indexed&type=Code
  • I have included a real-world usage sample for all extensions added in this PR:
    • Sample source(s):
      • INI/ms.properties: fnp/pylucene
      • INI/defaults.properties: phingofficial/phing
      • Java Properties/libraries.properties: dustin/java-memcached-client
      • Java Properties/sounds.properties: spullara/maptool
    • Sample license(s):
      • phingofficial/phing: LGPL-3.0
      • dustin/java-memcached-client: MIT
      • spullara/maptool: Apache 2.0
      • fnp/pylucene: Apache 2.0
  • I have included a syntax highlighting grammar.
  • I have included a change to the heuristics to distinguish my language from others using the same extension.

• I am associating a language with a new file extension.

  • The new extension is used in hundreds of repositories on GitHub.com
    • Search results for each extension:
      • https://github.com/search?utf8=%E2%9C%93&type=Code&ref=searchresults&q=extension%3AFOOBAR+KEYWORDS+NOT+nothack
  • I have included a real-world usage sample for all extensions added in this PR:
    • Sample source(s):
      • [URL to each sample source, if applicable]
    • Sample license(s):
  • I have included a change to the heuristics to distinguish my language from others using the same extension.

• I am fixing a misclassified language

  • I have included a new sample for the misclassified language:
    • Sample source(s):
      • [URL to each sample source, if applicable]
    • Sample license(s):
  • I have included a change to the heuristics to distinguish my language from others using the same extension.

• I am changing the source of a syntax highlighting grammar

  • Old: https://github-lightshow.herokuapp.com/
  • New: https://github-lightshow.herokuapp.com/

• I am adding new or changing current functionality

  • I have added or updated the tests for the new or changed functionality.

[pchaigno]

Thanks for submitting the pull request @eager!

Because .properties would remain unambiguous, I didn’t add samples.

We still require a sample file for each extension.

LGTM otherwise!

[pchaigno]

Using @Alhadis' Harvester script, I collected 2913 sample urls (from the 19M total) from the search extension:properties and downloaded the 615 corresponding repositories. According to Linguist, out of those 615 repositories with at least one .properties file, 226 are 100% Java, 457 have some Java (including the 226), and 158 don't contain any Java.

@eager Based on this analysis, I think we'll need to keep the .properties extension as an INI extension too, find a sample for INI, and add a heuristic rule to disambiguate the two.

[eager]

@pchaigno thanks for the review and pointer to Harvester. I had only been doing spot-checking of about a dozen results, so I’ll grab some more samples from there for disambiguation.

A few questions:

1. Are there project guidelines for how to handle ambiguous languages that surface during a samples search, but that aren‘t in the original scope?

I noticed in my initial spot-checking that there was some usage of .properties that were neither INI nor Java Properties, such as this. My best guess is they are for the OpenAstexViewer Scripting Language.

2. Any sense of which language we should default to when it’s impossible to disambiguate? I don’t think it will be possible to disambiguate the most basic sample:

key = value

3. Would it help to remove group: Java from the language definition, so that if we lean toward Java Properties, for its more robust syntax highlighting, it doesn’t add misleading repo statistics?

[pchaigno]

1. Are there project guidelines for how to handle ambiguous languages that surface during a samples search, but that aren‘t in the original scope?

Not really. When we do find such languages we consider them for language addition though. That means we then follow the appropriate steps from the contribution guidelines. Of course, we only add them if they meet the in-the-wild criteria.

I noticed in my initial spot-checking that there was some usage of .properties that were neither INI nor Java Properties, such as this. My best guess is they are for the OpenAstexViewer Scripting Language.

Then, we should first try to estimate their number on GitHub. Do you know keywords from that language we could use for the search? (From the documentation you linked to, it's not clear to me whether charge should be considered a keyword.)

2. Any sense of which language we should default to when it’s impossible to disambiguate? I don’t think it will be possible to disambiguate the most basic sample:

Usually, we don't assign any language in the Heuristic strategy if we can't recognize a known pattern or keyword. Not assigning a language, means we defer to the next strategy, the Classifier strategy. I'm expecting there are much more INI files than Java Properties files, so I'd be tempted to default to INI here, if we identify such a key=value construct.

@lildude @Alhadis What do you think (about defaulting to INI for key=value constructs vs. deferring to Classifier)?

3. Would it help to remove group: Java from the language definition, so that if we lean toward Java Properties, for its more robust syntax highlighting, it doesn’t add misleading repo statistics?

Yes, I think we should remove it from the Java group in any case. It feels a bit weird to have a data language being marked as a parent programming language (and I'm not sure how that would end up counting the files...).

[lildude]

@lildude @Alhadis What do you think (about defaulting to INI for key=value constructs vs. deferring to Classifier)?

As this is only for the .properties extension, I think we might be safe to do so. There are definitely a lot of uses of it as an INI-type file which clearly isn't related to Java, but I could be missing something.

[lildude]

Nudge.

[pchaigno]

Sorry for the delay @eager.

Could you add a test for the new heuristic rules in test_heuristics.rb? We should be good to go after that.

EDIT: Actually, there's a failure we need to handle. I'll have a look.

[pchaigno]

Should this be an elsif? It's either key=value or key:value, right? I think that will fix the test failure.

[pchaigno]

@eager Did you get a chance to work again on this lately?

[eager]

@pchaigno apologies for disappearing on this for a while. Updated, with the test failure fixed.