add reverseind(s, i): convert indices in reverse(s) to indices in s

[stevengj]

The main use of reverse(s::String) appears to be regex searching backwards in a string (see the discussion in #6165), but it is insanely hard to correctly convert the resulting indices (of matches in the reversed string) back to indices in the original string (see e.g. #9227). reverseind(s, i) solves this problem: reverse(s)[i] == s[reverseind(s, i)].

For example, in REPLCompletions this replaces code that is either tricky (nextind(partial, endof(partial)) - m.offset, which relies on the fact that the match is ASCII and that ASCII is encoded by a single code unit in all of our encodings) or slow chr2ind(str, length(str)-ind2chr(rstr, last(r))+1) (which is linear time in length(str)).

(It might be useful to check whether reversed strings are being searched in other packages, and whether the indices are handled correctly.)

cc: @dhoegh, @jiahao

[stevengj]

So far, the only example outside of Base that I can find which uses indices of a search result in a reversed string is in URLParse.jl. It appears to get it wrong.

[nalimilan]

Wouldn't it make sense to have a function which would reverse the string, look for matches, and return the indices in the original string directly? Not saying reverseind wouldn't be useful, though.

[stevengj]

(Travis failure looks unrelated.)

[stevengj]

@nalimilan, that could make sense, but seems like it should be a separate PR (that would use reverseind). As long as we are exporting reverse for string processing, I think we should export the index mapping.

(It could even just be an rsearch(s::String, r::Regex) method, although we also have eachmatch and ismatch etcetera. However, it would unfortunately behave somewhat differently than rsearch(s::String, r::String) because it wouldn't be easy to automatically reverse a Regex, so in the former case you'd have to make sure that the user specifies a reversed regex. This was discussed in #6276.)

[stevengj]

Again, the Travis error (on Linux) seems unrelated:

Worker 9 terminated.
ERROR: ProcessExitedException()
in wait at ./task.jl:284
in wait at ./task.jl:194
in wait_full at ./multi.jl:576
in remotecall_fetch at multi.jl:676
in remotecall_fetch at multi.jl:681
in anonymous at task.jl:1450
while loading /tmp/julia/share/julia/test/runtests.jl, in expression

[tkelman]

Worker 9 was on bitarray, that's #9176

[stevengj]

@StefanKarpinski, ok to merge (after rebase)?

[StefanKarpinski]

Seems good to me.

[PallHaraldsson]

[Not sure if disallowed to comment on "Merged" as a "closed" issue..]

Just looking at this it seems "return the corresponding index" means (and must mean?) "return the corresponding byte index". Maybe that needs to be explicit.. I want to get rid of any kind of indexing (as a default). Seems this is one function in my way.. Any good way to know how many functions depend on the implementation (byte-indexing)? I guess, I would have too take a look at any function using UFT8String. I understand any function can access private data - and that is "rude", but I guess that it is the rule in Base and an exception to "rude"-rule..

Hopefully this function is not speed-critical, as it seems I would want to let it return code-point indexes eventually..

[stevengj]

It returns whatever index is appropriate for the given string type. This won't change if indexes are changed to opaque values.