interface for adjusting tokenisation in graph

[jonorthwash]

Make a way to adjust the tokenisation of elements of a UD graph. This should allow you to divide existing tokens, combine existing tokens, reorder existing tokens, split a token into subtokens, insert tokens, and delete tokens, and renumbering should be automatic.

Keyboard shortcuts should be involved, but mouse actions too, suck as clicking and dragging across an element that should be split from another element before hitting the "split" key or something.

[ftyers]

so imagine you have

thisisatest.

you click on (t, 0) hold down click and drag to (s, 3) and let off click

this makes:

this isatest.

now you click on (i, 5) and hold down click and drag to (s, 6), which makes

this is atest.

etc. until all the list is properly surface-tokenised.

[jonorthwash]

But you'd have to enter "tokenise" mode, and then leave it when done. And you'd need a similar way to e.g., combine existing tokens, which has two subtypes: merge tokens, and merge and create subtokens.

[jonorthwash]

Basic interface:

• select content (e.g., across tokens), press key combo A to merge/split tokens
• select content (e.g., within a token), press key combo B to merge/split subtokens/"words"

Will need to include:

• logic for renumbering; one option:
  • Say nodes 8 and 9 are merged. Renumber all nodes above 9, subtracting one, and keep track in a hash of mapping from old to new. Then go through all targets and renumber according to hash.

[maryszmary]

At the moment the interface supports splitting a token into new ones. To split a token, click the right button of the mouse on the token node, then (using arrows) go to the place you want to split and insert a space, then press enter. All the attributes default to belong to the first part.

An illustration. Before:

After:

So far the positioning of the input element is not ideal, I'm going to improve it.

[jonorthwash]

Nice! Can tokens be merged too, or not yet?

[maryszmary]

Not yet.

[maryszmary]

Now they can. Here is how it can be made:
Left click on the main token (the token you want to take the markup from), then press m, then select with an arrow (right or left), which neighbor you want to append to the main token.

An example: left click on the token, then m

I press the right arrow which token i want to append

[jonorthwash]

Cool!

[jonorthwash]

This is the basic functionality for this issue. We'll open a new issue for subtoken-related functionality.