Add and document \BCPdata to kernel

[josephwright]

Brief outline of the enhancement

As discussed in reutenauer/polyglossia#581 and elsewhere, a common data format for BCP-47 data is needed. This really needs to be initiated and documented at the kernel level, with packages such as polyglossia and babel then re-defining to provide the actual data.

This issue is to allow us to track the design of the interface: currently \BCPdata{...} where ... needs to be refined.

[jspitz]

Just to summarize the proposal so far. The arguments that should be supported are:

1. language (e.g., de)
2. region (e.g., AT)
3. script (e.g., Latn)
4. variant (e.g., 1901)
5. extension.t (transformation, e.g., en-t-ja)
6. extension.u (additional locale information, e.g., ar-u-nu-latn)
7. extension.x (private use area, e.g., la-x-classic)

and possibly

• tag (for the "full" BCP-47 tag as set by babel/polyglossia which however might omit presupposed values)

To access the main language, it was proposed to have a main.* argument as a prefix to the above tags.

@u-fischer furthermore proposed some semantic links such as casing which returns the appropriate tag/subtag for the casing commands.

[josephwright]

@jbezos I assume you are also happy with that list? If so, I'll add to ltfinal and document there and in ltnews for the June release, and adjust the case code so it picks up \BCPdata and thus solves the polyglossia aspect.

(Thanks all for the contributions here: I think we are getting to a very good situation for locale data.)

[u-fischer]

@josephwright will you use a semantic wrapper like casing for \MakeUppercase? Imho it would be sensible as the discussion showed that none of the "primitive" values is the correct one in all cases. I will then adapt hyperref in a similar way.

[josephwright]

@josephwright will you use a semantic wrapper like casing for \MakeUppercase? Imho it would be sensible as the discussion showed that none of the "primitive" values is the correct one in all cases. I will then adapt hyperref in a similar way.

Yes that is the plan I think: makes a lot of sense

[jbezos]

👍

[josephwright]

OK, I will go with \BCPdata{casing}: that I hope should be 'available' from both babel and polyglossia by the time we do the kernel release.

[jbezos]

I’ve just added \BCPdata{casing}. I’ll release it in a few days (probably next Tuesday).

[jspitz]

I’ve just added \BCPdata{casing}. I’ll release it in a few days (probably next Tuesday).

Same here for polyglossia.

[jbezos]

As to main, sorry, I have no time. We have to wait for the next release (in mid May, I think).