Tmarks should not be possible on character-set members: -[^'ab'] and ~[^'a'; -'b'] etc.

[cmsmcq]

The grammar of 2022-02-22 (like all of its predecessors, as far as I know) allows tmark both on literals and on inclusions and exclusions. When a literal occurs as a member of a character set, it is thus grammatical to write things like

-[^'ab']
~['a'; -'b'; ^'c']

It is not clear what such constructs might mean.

The simplest solution appears to be to replace literal with ^string in the right-hand side of the rule for member.

It might be thought that we could solve this problem by moving tmark into the rule for terminal from its current locations in the definitions of quoted, encoded, inclusion, and exclusion. (That is, that is what I thought.) This will not work because for tmark to be serialized as an attribute in the XML it must be a descendant of the nonterminal which produces the element.

Proposal: replace

  -member: literal;
           range;
           class.

with

  -member: ^string;
           range;
           class.

[ndw]

I believe that change would lose hex from within ranges.

[cmsmcq]

The nonterminal hex is not currently reachable from range; hex-encoded range boundaries are reached via character:

        range: from, s, -"-", s, to, s.
        @from: character.
          @to: character.
   -character: -'"', dchar, -'"';
               -"'", schar, -"'";
               "#", hex.

[ndw]

That's only a hex that's part of a range. With your proposed change to member, this does not parse:

s: ["0"-"9"; #20] .

because range is "x - y" so I get:

<fail xmlns:ixml="http://invisiblexml.org/NS" ixml:state="failed">
   <line>1</line>
   <column>18</column>
   <pos>17</pos>
   <unexpected>]</unexpected>
   <permitted>'\n', '\r', '\t', '{', ["0"-"9";"a"-"f";"A"-"F"], [Zs]</permitted>
</fail>

[cmsmcq]

You're right; sorry to be so slow on the uptake. I'll need to think about the right way to fix this.

[spemberton]

(adding my reply from the ixml list to this thread) So an inclusion currently looks like this: ["0"-"9"; "+-()."; #d; L] giving <inclusion> <range from='0' to='9'/> <literal string="+-()."/> <literal hex='d'/> <class code='L'/> </inclusion> Since all meaningful (semantic) characters go in attributes, I propose <inclusion> <member from='0' to='9'/> <member string='+-().'/> <member hex='d'/> <member class='L'/> </inclusion> @from and @to would continue to allow as now e.g. from='#d' Look OK?

[cmsmcq]

On 22 March 2022, SP's proposal to resolve this issue was accepted. Issue to be closed once the change is in the published spec.

[ndw]

Steven reports this is resolved.