WIP: JSON formatter

[milmazz]

This is an early version of the JSON formatter suggested in #495, the idea is to get some feedback from @josevalim, @ericmj and anyone who wants to join to the discussion.

Also, I have some questions:

1. In lib/ex_doc/formatter/json.ex you will find a little "spec" of the JSON Formatter, is this the right place to put such information?
2. Do we want to include information about assets? Right now the JSON formatter produce just one file called <project_name>.json. If we want to include information about the assets, should we include the files under the #{config.output}/assets folder?
3. I reuse ExDoc.{ModuleNode,FunctionNode,TypeNode} structs, is that OK or do we need to create new ones with lets say different/better names?

You can see an output example here

[sourcelevel-bot]

TODO found

[josevalim]

Thank you @milmazz. I think @ericmj scrapped the idea of using a JSON formatter, so let's hear from him. In any case, I don't think we should be looking for another formatter to maintain, so a separate project is probably the best way to go! :)

[whatyouhide]

A while ago I suggested something different: have ex_doc only output JSON and then have formatters read off JSON when formatting, so that things like epub could be in a different project. Not sure if related.

[josevalim]

Currently the contract with formatters uses Elixir data structures, I see no reason to "downgrade" it to JSON. And JSON won't work for HTML without breaking local use. -- *José Valimwww.plataformatec.com.br <http://www.plataformatec.com.br/>Founder and Director of R&D*

[whatyouhide]

I see no reason to "downgrade" it to JSON

Well there are a few reason, I would say mainly the ability to be able to use any tool (not just Elixir) to generate documentation and the increased decoupling. I realize decoupling could be achieved with Elixir data structures as well though (not as broadly but), so it may be thinking too far ahead.

[josevalim]

If we move to JSON, it will end up with the ePub formatter parsing the JSON and building Elixir data structures. If you really need to use other tools, just then it makes sense to opt-in to JSON. -- *José Valimwww.plataformatec.com.br <http://www.plataformatec.com.br/>Founder and Director of R&D*

[whatyouhide]

@josevalim yeah, that makes sense as well. Until such tools appear, we can hold this off IMO then.

[ericmj]

@josevalim To my knowledge a JSON formatter is still the best way to have external applications consume documentation metadata.

To elaborate on the basis of this; if hexdocs.pm would like add full text search or search over all projects there needs to be a way to read metadata about the docs. If we include a JSON document with this information when we publish to hexdocs anyone would be able to consume this.

[milmazz]

@josevalim Do you agree with @ericmj? Can I continue with this approach?

@ericmj Do you agree with the current JSON structure?

[ericmj]

Don't pipe into anonymous function calls created on the fly.

[ericmj]

Update Hex or avoid committing mix.lock.

[ericmj]

Do you agree with the current JSON structure?

It looks good, the only thing that isn't clear is where modules, functions, extras and attachments are located in the whole json structure. I don't think package metadata should be included because they are not really related to the docs imo and could be find other ways.

[milmazz]

@ericmj In the latest version I removed the package metadata as you suggested, also, there are some improvements in the JSON structure:

{
	"version": "0.16.1",
	"name": "ex_doc",
	"language": "en",
	"icon": null,
	"homepage_url": null,
	"extras": null,
	"description": "ExDoc is a documentation generation tool for Elixir",
	"attachments": null,
	"about": "ExDoc/version/1",
	"items": {
		"tasks": [],
		"protocols": [],
		"modules": [{
			"title": "ExDoc",
			"source_url": "https://github.com/elixir-lang/ex_doc/blob/master/lib/ex_doc.ex#L1",
			"source_path": "lib/ex_doc.ex",
			"module": "Elixir.ExDoc",
			"functions": [],
			"types": [],
			"callbacks": []
		}],
		"extras": [],
		"exceptions": [],
		"attachments": []
	}
}

[josevalim]

@ericmj @milmazz sorry but I am still not convinced this belongs in ExDoc itself. As soon as we publish it, it becomes part of our public contract, and it is unclear this will actually fit the purpose of whoever needs docs in JSON. It is like designing an API without a consumer then only to find out the API is lacking.

[milmazz]

I'll close this PR because it remained in the Open state without activity for more than 60 days.

Feel free to re-open this PR if you want or you think is still relevant for this project.