Proposal: Custom extension fields

[pragmaware]

This is a proposal for adding custom extension fields.

It's not reasonable to add specific fields for all the possible extensions we might think of. Each parser will have its own subset instead. Each parser will be responsable of checking if the custom extension fields is enabled or not. Each parser will provide the label for the custom extension field (which is a static string anyway).

This pull also contains a sample usage of the custom extension fields: template information for classes.

Another commit in this pull enables the custom extension field with a parser specific option: --c++-enable-template-info. The advantage is that it is incredibly simple. No complicated code to add to handle parser specific parameters.

(Btw, the first two commits are an artefact of synchronizing with upstream: not sure how to get rid of them..maybe by deleting the fork and recreating it?)

[coveralls]

Coverage increased (+0.03%) to 82.44% when pulling 3f177ad on pragmaware:custom-fields into dac081e on universal-ctags:master.

[masatake]

Adding a filed is better than adding new option; T is still available.

Following commi is an example of adding a field.

bc7096c

[masatake]

I read your patch. Currently customLabel holds a string. It should hold fieldType of field.h.

[pragmaware]

A couple of things to consider:

• we might want to add many new fields. For c++ I have in mind at least "endline" (for end lines of scopes) and "templatespec" (for template specialisations).
• what I have called "template" is more properly called "generic" in java and C#

[masatake]

I thnk the idea behind custom fields is good. Ideally I would like to extend it. We can replace extensionFields field in tagEntryInfo with:

struct ExtensionField {
    fieldType ftype;
    void*     data;
};

struct ExtensionFieldsHolder {
    size_t count;       /* used */
    size_t size;        /* acutally allocated */
    struct ExtensionField *fields;
};

Currently many things about extensionField is still handled in entry.c. With the replacement, many things can be done field.c.

For attaching a field to a tag entry, you can use

void attachField (tagEntryInfo *tag, fieldType ftype, void *value);

A couple of things to consider:

we might want to add many new fields. For c++ I have in mind at least "endline" (for end lines of scopes) and "templatespec" (for template specialisations).
what I have called "template" is more properly called "generic" in java and C#

You can use e and g or T for them
Using letters for representing fields may be unrealistic. It will be nice if the long names can be used.

$ ctags --fields="+{endline}{generic}"

Anyway, I would like to merge your latest result about C++ parser. I will write about the step for merging.

[pragmaware]

Yep, having all "variable" extension fields is a nice idea. It probably requires some work in all the parsers though.

The template information in the C++ parser is there since quite a lot of time, it's only waiting for a suitable method of writing it to output. So it can still wait a bit while we think about it.

[masatake]

Yep, having all "variable" extension fields is a nice idea. It probably requires some work in all the parsers though.

o.k.
I will work on making the field "variable". I caught cold, so I cannot do it quickly.

[masatake]

I'm working on this item now...

I find a better API. It is nice if a parser can declare their own fields statically like

static fieldDesc cxxFields [] = {
    { 't', "template", "help message...",             ... },
    { 'e', "endline",  "help message...", ... }
    { 's', "templatespec", "help message...",         ... },

};

parserDefinition * CppParser (void)
{
    static const char * const extensions [] =
    {
        "c++", "cc", "cp", "cpp", "cxx",
        "h", "h++", "hh", "hp", "hpp", "hxx", "inl",
#ifndef CASE_INSENSITIVE_FILENAMES
        "C", "H", "CPP", "CXX",
#endif
        NULL
    };

    static selectLanguage selectors[] = { selectByObjectiveCKeywords, NULL };

    parserDefinition* def = parserNew("C++");

    def->fields     = cxxField;
    def->kinds      = cxxTagGetKindOptions();
    ...
    return def;
}

But I cannot implement all at once. I will provide small/simple one first as the initial shot.
I hope you can continue your C++ work based on it. I will improve incrementally.

[pragmaware]

Yep, this way also would work. I'm OK with whatever lets me add custom fields to the output.
Let me know when you think that the minimum usable implementation is ready and I'll make use of it :)

[masatake]

After implementing I recognized that fields in an entry cannot be allocated dynamically; there is no chance to make it free. I have to use statically allocated array for awhile.

[pragmaware]

No longer relevant since #872 has been merged.