Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
cc @BurntSushi, this is reply to that reddit comment.
I must say that I work on the higher level than text editors, so I am interested in API generality and ergonomics and have a vague understanding of performance implications.
I write parsers which take input text and produce trees. I'd love to make parsers general. Specifically, advanced editors rarely use
str
as an internal storage. Usually text is stored in some kind of chunked representation, like piece table, gap buffer or tree. While I can write parser againstimpl Iterator<Item=char>
, that is not really convenient, as some form of random-access helps. For example, it's useful to be able to take a substring of the input (to intern identifier text) or to find the next\n
.So, as a user, I want to write code against the trait that is more convenient than iterator, but still allows the code to work with various text representations.
I also am an implementer of this trait: because syntax trees are lossless, you can always extract text representation from a syntax node (by concatenating text of all tokens), and it is useful to ask questions like "how many newlines does this text contains?".
As an implementer, I'd love to provide some fundamental API (slicing and iteration over string chunks), and get convenient string methods like
find
,contains
,trim
for free.In this PR I've tried to sketch the
Str<'a>
trait and implement it for my Tree.Fun fact:
Despite the fact that Java has CharSequence, jFlex operates on
char[]
: jflex-de/jflex#153For this reason, IntelliJ uses a CharSequence-based fork of jFlex:
JetBrains/intellij-deps-jflex@9c9da13