replace: substring on match function argument error

[mdesantis]

Hello,

I'm implementing a simple template engine for Julia (version 0.3.0-767~ubuntu13.10.1). I get this error:

replace("test <% 1 %> test", r"<%.*?%>"s, s -> parse(s[3:end-2]))

ERROR: a SubString must coincide with the end of the original string to be convertible to pointer
 in convert at string.jl:662

It should be a bug, isn't it?

[pao]

More SubString fallout. The parse method is ultimately expecting a null-terminated string. Try:

replace("test <% 1 %> test", r"<%(.*?)%>"s, s -> parse(utf8(s[3:end-2])))

We probably need to teach parse to do that for us.

[mdesantis]

Confirmed that works using utf8():

replace("test <% 1 %> test", r"<%.*?%>"s, s -> parse(utf8(s[3:end-2])))
#=> "test 1 test"

[pao]

See also #5675

[stevengj]

The right thing is for parse to call bytestring on its argument before passing it to ccall.

[pao]

Thanks, @stevengj, that sounds good.

[JeffBezanson]

Those null terminators really are hell. Either we make functions like replace and split slower up-front, or we face copying a string potentially many times. One slightly scary option is to update SubStrings in-place with null-terminated copies as necessary, so copying is never needed more than once for a given string. I'm not sure whether that's worthwhile or advisable though.

[stevengj]

Even scarier option: Just temporarily write a null character into the string during the ccall.

[JeffBezanson]

Wow, that is true evil genius.

[stevengj]

(But if the ccall might throw an exception, you need to have an exception handler to undo the null.)

[StefanKarpinski]

Oh, man. That's so crazy it might work. I'm not sure if exception trapping is going to be low enough overhead though.

[carlobaldassi]

It would also need to check for partially overlapping sub-strings, and use copies in that case.

[stevengj]

@carlobaldassi, that's true, the unusual case of passing multiple substring arguments to the same function would be tricky.

Also, this would be very difficult to do automatically because of perverse situations involving passing callback functions.

[kmsquire]

Even scarier option: Just temporarily write a null character into the string during the ccall.

The main concern here is that in C, strings are mutable.