Generate JSON file with index of all modules and functions

[ericmj]

Currently there is no way find all modules and functions without scraping the generated docs, which is difficult to say the least and not backwards compatible. An index file would make it much easier for external tools that want to implement search on documentation.

JSON files with the docs and specs would also be good to have.

[milmazz]

@ericmj dist/sidebar_items.js includes a JS object that list all the exceptions, modules, protocols and additional documentation, its structure is something like this:

{
    "exceptions": [{
        "id": "EEx.SyntaxError",
        "title": "EEx.SyntaxError",
        "functions": [{
            "id": "exception/1",
            "anchor": "exception/1"
        }, {
            "id": "message/1",
            "anchor": "message/1"
        }]
    }],
    "extras": [{
        "id": "api-reference",
        "title": "API Reference",
        "headers": []
    }],
    "modules": [{
        "id": "EEx",
        "title": "EEx",
        "functions": [{
            "id": "compile_file/2",
            "anchor": "compile_file/2"
        }, {
            "id": "compile_string/2",
            "anchor": "compile_string/2"
        }, {
            "id": "eval_file/3",
            "anchor": "eval_file/3"
        }, {
            "id": "eval_string/3",
            "anchor": "eval_string/3"
        }],
        "macros": [{
            "id": "function_from_file/5",
            "anchor": "function_from_file/5"
        }, {
            "id": "function_from_string/5",
            "anchor": "function_from_string/5"
        }]
    }, {
        "id": "EEx.Engine",
        "title": "EEx.Engine",
        "callbacks": [{
            "id": "handle_body/1",
            "anchor": "c:handle_body/1"
        }, {
            "id": "handle_expr/3",
            "anchor": "c:handle_expr/3"
        }, {
            "id": "handle_text/2",
            "anchor": "c:handle_text/2"
        }],
        "functions": [{
            "id": "handle_assign/1",
            "anchor": "handle_assign/1"
        }, {
            "id": "handle_body/1",
            "anchor": "handle_body/1"
        }, {
            "id": "handle_expr/3",
            "anchor": "handle_expr/3"
        }, {
            "id": "handle_text/2",
            "anchor": "handle_text/2"
        }]
    }, {
        "id": "EEx.SmartEngine",
        "title": "EEx.SmartEngine",
        "functions": [{
            "id": "handle_body/1",
            "anchor": "handle_body/1"
        }, {
            "id": "handle_expr/3",
            "anchor": "handle_expr/3"
        }, {
            "id": "handle_text/2",
            "anchor": "handle_text/2"
        }]
    }],
    "protocols": []
}

[ericmj]

Any way we can get it as a pure json object without the surrounding javascript?

[milmazz]

I can review if it's possible tonight... can I ask what do you have in mind? :)

[ericmj]

We need to store an index of all functions and modules for external tools to be able to implement search. Right now all tools scrape the html that ex_doc generates which is worse than looking at an index for obvious reasons. I would also like a separate file that contains all documentation for each function and module so that tools can implement full text search.

EDIT: My long term plan is to use this for hexdocs so that we can search over multiple packages and have full text search.

[josevalim]

To sum up the discussion on #496, the proper way to solve this would be through a JSON formatter, which emits the whole documentation in a way to be consumed by a third party.

[josevalim]

Closing in favor of the PR.