Explain LTM (longest token matching) in regexes

[smls]

The current documentation of | alternations is regrettably short and inaccurate.

This is causing users to get confused about what | actually does - see e.g. #1141 and the multiple Rakudo RT's that have popped up (and been rejected) about this.

This really needs to be explained much more accurately.

Resources:

• The design docs have a long wall of text about it S05#Longest-token_matching.
  (As well as some more relevant stuff in S05#Overview and the intro blurb above it.)
• In RT #132066, I tried to give a short explanation for the basic rule.

Essentially, what | does is this:

1. Select the branch whose declarative prefix matches the longest string.
2. If it's a tie, select the match with the highest specificity.
3. If it's still a tie, use additional tie-breakers.

Thoughts:

The difficulty is in...

• Thoroughly explaining the bolded parts in a way that newbies will find intuitive, without sacrificing conciseness.
  (For completeness, and to help users think in these terms, the documentation of all other regex constructs should probably be extended to mention whether they are considered declarative or procedural.)

• Reaching clarity on what "additional tie-breakers", exactly, apply to |.
  (The tie-breaker stuff in S05 seems to have been written largely with LTM of proto-regexes in mind – some of it cannot apply to LTM of |, and some of it is a bit ambiguous w.r.t. to how it should apply to |.)

Also note that Rakudo currently has at least two LTM bugs:

• RT #122951
• RT #130612

(Although those are edge cases, and shouldn't distract from the fact that Rakudo is mostly S05-compliant on this topic, and last I heard Perl 6.c is supposed to implement LTM as specced by S05.)