Comment Support in JSON output

[fosterbrereton]

This PR takes comments parsed by clang and folds them into the processed json (the bulk of this work happens in the new ProcessComment routine in utilities.cpp). This PR also incorporates these comments in the Hyde emitters for YAML output. Note that this feature takes advantage of the functionality already implemented in Clang to parse C++ comments for inline documentation. Not all inline documentation will be accounted for. Specifically, comments not tied to an AST node will be processed. For example, top-level fields like @file will not show up in Hyde output, because they are not processed into a node in the Clang AST.

• This adds a new __INLINED__ value. If a documentation field (e.g., brief) is found documented in-line in the C++ sources, the field will change from __MISSING__ to inlined, and the inlined value will be used. Fields marked __INLINED__ can still be filled in if additional documentation external to the source file is desired.
  • Specifically for the owner field, this requires a special @hyde-owner in-source documentation, if such were to be used by a developer. Other fields will try to map intelligently from what may already be provided for other source documentation tools (e.g., Doxygen.)
• Better YAML front-matter detection.
• Adds a new -hyde-fixup-subfield mode, which will convert from Hyde v1 to Hyde v2 docs. This will move all fields except layout and title into a hyde subfield in the YAML. Note this mode is unique in that it takes pre-existing documentation as source(s), not a C++ source file.
• Fixes Set tested_by to __OPTIONAL__ by default for anything that’s compiler-generated. #67
• Fixes Denote compiler-provided routines #6
• Implicit, = default, and = delete routines will no longer require documentation, that is, all their __MISSING__ fields will be marked __OPTIONAL__.
• Attempts will be made to bubble-up brief and description fields from method/function overloads to the top-level page, reducing the amount of missing fields a developer must fill in.
• Additional Clang warnings are automatically enabled during runs of Hyde. For example, -Werror=documentation is now specified for the driver to ensure the documentation in the sources accurately reflect the C++ entities with which they are associated.
• Many examples were cleaned up and inline documentation added to them to flex the new capabilities of the tool.

Example

Given the following source file add_one.cpp:

// Compile via:
// 
//     clang++ -fparse-all-comments -Wdocumentation -fcomment-block-commands=hyde -Xclang -ast-dump -c add_one.cpp

/**
    @brief This is a brief comment for foo
*/
struct foo {
    /// stores the variable value `x`
    int _x;
};

/**
    This is some kind of high-level comment.

    @hyde-owner fbrereto
    @brief This is a brief comment for add_one.
           It has multiple lines so you have to pay attention.
    @param x the variable to increment.
    @return one more than the input value.

    This is another comment, not sure why.
*/
int add_one(int x) {
    return x + 1;
}

/***************************************************************************//**
 * A brief history of Doxygen-style banner comments.
 *
 * This is a Doxygen-style C-style "banner" comment. It starts with a "normal"
 * comment and is then converted to a "special" comment block near the end of
 * the first line. It is written this way to be more "visible" to developers
 * who are reading the source code.
 *
 * This style of commenting behaves poorly with clang-format.
 *
 * @param theory Even if there is only one possible unified theory. it is just a
 *               set of rules and equations.
 ******************************************************************************/
void doxygenBanner( int theory );

Here is the resulting JSON output with the related comments fields added.

{
  "classes": [
    {
      "comments": {
        "command": [
          {
            "name": "brief",
            "text": "This is a brief comment for foo"
          }
        ]
      },
      "ctor": "unspecified",
      "defined_in_file": "/Users/fbrereto/Desktop/add_one.cpp",
      "deprecated": false,
      "dtor": "unspecified",
      "fields": {
        "foo::_x": {
          "access": "public",
          "comments": {
            "paragraph": [
              {
                "text": "stores the variable value `x`"
              }
            ]
          },
          "defined_in_file": "/Users/fbrereto/Desktop/add_one.cpp",
          "deprecated": false,
          "name": "_x",
          "namespaces": [],
          "parents": [
            "foo"
          ],
          "qualified_name": "foo::_x",
          "type": "int"
        }
      },
      "kind": "struct",
      "methods": {},
      "name": "foo",
      "namespaces": [],
      "parents": [],
      "qualified_name": "foo"
    }
  ],
  "enums": [],
  "functions": {
    "add_one": [
      {
        "arguments": [
          {
            "name": "x",
            "type": "int"
          }
        ],
        "comments": {
          "command": [
            {
              "name": "hyde-owner",
              "text": "fbrereto"
            },
            {
              "name": "brief",
              "text": "This is a brief comment for add_one. It has multiple lines so you have to pay attention."
            },
            {
              "name": "return",
              "text": "one more than the input value."
            }
          ],
          "paragraph": [
            {
              "text": "This is some kind of high-level comment."
            },
            {
              "text": "This is another comment, not sure why."
            }
          ],
          "param": [
            {
              "direction": "in",
              "direction_explicit": false,
              "index_valid": true,
              "name": "x",
              "text": "the variable to increment.",
              "vararg_param": false
            }
          ]
        },
        "defined_in_file": "/Users/fbrereto/Desktop/add_one.cpp",
        "deprecated": false,
        "name": "int add_one(int)",
        "namespaces": [],
        "parents": [],
        "qualified_name": "int add_one(int)",
        "return_type": "int",
        "short_name": "add_one",
        "signature": "int add_one(int)",
        "signature_with_names": "int add_one(int x)",
        "visibility": "default",
        "visibility_explicit": "false"
      }
    ],
    "doxygenBanner": [
      {
        "arguments": [
          {
            "name": "theory",
            "type": "int"
          }
        ],
        "comments": {
          "paragraph": [
            {
              "text": "************************************************************************"
            },
            {
              "text": "A brief history of Doxygen-style banner comments."
            },
            {
              "text": "This is a Doxygen-style C-style \"banner\" comment. It starts with a \"normal\" comment and is then converted to a \"special\" comment block near the end of the first line. It is written this way to be more \"visible\" to developers who are reading the source code."
            },
            {
              "text": "This style of commenting behaves poorly with clang-format."
            }
          ],
          "param": [
            {
              "direction": "in",
              "direction_explicit": false,
              "index_valid": true,
              "name": "theory",
              "text": "Even if there is only one possible unified theory. it is just a set of rules and equations. ****************************************************************************",
              "vararg_param": false
            }
          ]
        },
        "defined_in_file": "/Users/fbrereto/Desktop/add_one.cpp",
        "deprecated": false,
        "name": "void doxygenBanner(int)",
        "namespaces": [],
        "parents": [],
        "qualified_name": "void doxygenBanner(int)",
        "return_type": "void",
        "short_name": "doxygenBanner",
        "signature": "void doxygenBanner(int)",
        "signature_with_names": "void doxygenBanner(int theory)",
        "visibility": "default",
        "visibility_explicit": "false"
      }
    ]
  },
  "namespaces": [],
  "paths": {
    "src_path": "/Users/fbrereto/Desktop/add_one.cpp",
    "src_root": ""
  },
  "typealiases": [],
  "typedefs": []
}