Expand C-family docstring resolution

[efritz]

Before:

After:

Summary:

Each supported language in these extensions have a set of properties attached to them (file extensions, what identifier patterns look like, what comments look like, etc). One of these properties is docstringIgnore, which will allow us to ignore the lines between where a ctags definition occurs and the comment that is meant to belong to it. Until now, all we've used this for is for Java-style annotations, e.g.

/** docstring **/
@MyBigAnnotation
class Something {
    // ...
}

This adds support for the C-family convention of breaking definitions up onto multiple lines:

/** docstring **/
RET_VALUE
ActualMethod(...) {
    // ...
}

This could have unintended side-effects where comments that do not belong to a ctags definition just happens to be close enough to it, but I think it's an acceptable risk tradeoff (while hand-testing I couldn't get it to do something worse than the previous behavior).

Fixes https://github.com/sourcegraph/sourcegraph/issues/17925.

[tjdevries]

Side question: How to add a test for something like this (is it simple?)

[efritz]

@tjdevries It's difficult to test changes to the language spec properties, so we default back to testing by hand. In order to run the extensions, you can side load it against a live SG instance (I used sourcegraph.com here as a user reported an actual line of code, which is always nice).

(Note that I had to disable the default sourcegraph/cpp instance in my user settings so I didn't have two extensions fighting over each other to provide slightly different data).

It would be nice to add integration tests here, but it could be a bit heavy-handed as it requires an instance (and search results from the backend). There are a good amount of tests in the machinery though, so I'm confident we didn't regress on things outside of the C-family heuristics that were changed here.

We have tests to ensure we are using the regex pattern correctly, we just don't test the explicit pattern on the language which is hopefully declarative and trivial.

[garobrik]

LGTM