Add spans to doc.to_json

[thomashacker]

Description

This PR adds spans to doc.to_json.

It is saved as a dict with SpanGroup names as keys and list[Span] as values.
The stored spans contain: start, end, label, and kb_id

Types of change

New feature to existing function

Checklist

• I confirm that I have the right to submit this contribution under the project's MIT license.
• I ran the tests, and all new and existing tests passed.
• My changes don't require a change to the documentation, or if they do, I've added all required information.

[adrianeboyd]

This opens a can of worms related to whatever we're calling the "JSON format" and options for serializing docs to JSON.

What are the intended uses of Doc.to_json?

Doc.to_json claims:

RETURNS (dict): The data in spaCy's JSON format.

but it's not the same format that can be loaded by spacy.training.converters.json_to_doc (and therefore not the format documented at https://spacy.io/api/data-formats#json-input), and this may really get confusing, especially since there's also no parallel Doc.from_json.

Doc.to_json diverges a lot from spacy.training.gold_io.docs_to_json and I'm trying to remember what the reasons were for these to be implemented separately / differently...

[svlandeg]

While I agree with Adriane's concerns about "the JSON format", I think that's an issue we'll have to address more properly in a separate discussion/PR. Given that we currently do already have doc.to_json and that we won't be removing this in a next release, we might as well make it support spans.

Good work on also including an appropriate unit test, @thomashacker !

I had a few more small comments, cf below.