experiment using fsts for Unicode tables

[BurntSushi]

I'm interested to see how well FSTs would do for storing Unicode tables, particularly for use in the regex-syntax crate. It could make set operations especially efficient and reduce binary size.

In order for this to work, the fst crate itself can't depend on regex-syntax. I think the right answer here would be to create a new sub-crate, called fst-regex, which provides the Automaton impl. That would trim off the current regex-syntax and utf8-ranges deps, and the mmap feature could be disabled, which would leave only byteorder, which is fine.

@fulmicoton As my sole (public) user, would this pose any problems for you? (I don't think it would.)

[fulmicoton]

@BurntSushi Can you elaborate on mmap will be disabled ?

[BurntSushi]

@fulmicoton I meant disabled inside regex-syntax. It is already an optional dep that is enabled by default. :-) See #27.

[fulmicoton]

No problem for me !

[BurntSushi]

I think the time for this has come and gone. It's unlikely I'd ever be okay with regex-syntax depending on fst, which is a pretty meaty dependency. Moreover, while fsts are reasonably better in terms of space efficiency, they tend to be slower than simple binary search or specialty tries. So I'm going to close this.

FWIW, the ucd-generate tool can output FSTs for many different Unicode things, including properties and Unicode character names.