parsing APIs that indicate whether something was possible, rather than raising an error?

[StefanKarpinski]

See http://stackoverflow.com/questions/21595246/try-catch-or-type-conversion-performance-in-julia. We have lots of integer, float, expression parsing code that all just raises exceptions rather than indicating whether they succeeded or not, but we discourage try/catch as control flow. Should we provide non-error-raising versions of all of these functions that people can use to test whether some string is a valid float, for example?

[JeffBezanson]

See also #3631.
It might be enough to restrict this to parsing, and not worry about conversion in general. Maybe we could return (result, isvalid) tuples. Not sure how best to separate that from the case where you want an answer-or-error though.

[StefanKarpinski]

Parsing is definitely a bit different since it tends to occur with external inputs where it's more normal not to know what the correct type for something is. We could also have a "data parsing" API that parses something, figuring out what the most conservative type for it is – integer, floating-point, big int, big float, string. Something like that. Probably want to throw dates in there too. I don't think it needs to be an expandable list, but it should cover the kinds of basic data that data frames might want to be able to read and automatically determine the type of. I suspect the logic for some of this already must exist in DataFrames.

[ssfrr]

So it seems like the idioms under consideration are:

A

if can_do_the_thing(x)
    y = do_the_thing(x)
else
    # I couldn't do the thing!
end

B

try:
    y = do_the_thing(x)
catch:
    # I couldn't do the thing!
end

C

(y, success) = do_the_thing(x)
if !success
    # I couldn't do the thing!
end

Regarding A, As @JeffBezanson mentions often there's considerable code duplication and inefficiency in checking whether you can do something before doing it. It also exposes the user of the API to race conditions if the data you're operating on is mutable (or for instance checking whether you have permission to write to a file before opening it for writing).

B doesn't have those problems, but according to the SO thread exception catching is slow. Is that a long-term thing or an artifact of some current implementation details? It also sounds like this pattern is discouraged as not idiomatic Julia.

Returning a tuple that includes a status field addresses those issues, but it feels like the exception throwing idiom is more general and cleaner to me. I'm definitely biased from python experience though.

[stevengj]

There is also

y = do_the_thing(x, default)
if y == default
   # I couldn't do the thing!
end

This pattern is already used in several places in Julia (e.g. get).

[JeffBezanson]

Exceptions certainly could be faster, but you often need to parse huge numbers of text values and it can easily be a bottleneck. Exceptions might not ever be fast enough for that.

[stevengj]

I did a quick benchmark, comparing the exception-based isFloat and the non-exception float64_isvalid, on an array of 10^6 strings (50% of which are valid numbers), and it doesn't seem like exceptions are the bottleneck here:

function isFloat(s)
    try
        float64(s)
        return true
    catch
        return false
    end
end
function isFloat{T<:String}(ss::Array{T})
    v = true
    for s in ss
        v = v && isFloat(s)
    end
    return v
end
function isFloat2{T<:String}(ss::Array{T})
    out = [1.0]
    v = true
    for s in ss
        v = v && float64_isvalid(s, out)
    end              
    return v
end
r = rand(10^6); rs = ASCIIString[rand(1:2) == 1 ? "wrong" : string(R) for R in r]
@time isFloat(rs)
@time isFloat2(rs)

(Note: of course, the loops could be short-circuited as soon as a false is encountered, but I intentionally did not do this, for benchmark purposes.) The results are almost identical:

elapsed time: 0.001753407 seconds (96 bytes allocated)
elapsed time: 0.001695101 seconds (112 bytes allocated)

[johnmyleswhite]

FWIW, DataFrames uses the following return values:

• parsed_value
• was_missing
• wasvalid

[JeffBezanson]

@stevengj you are still short-circuiting via &&.

[JeffBezanson]

I'll also add that basically all of the time spent when throwing an exception is for collecting a stack trace.

Switching && to &, I get:

elapsed time: 13.885327767 seconds (16059932 bytes allocated)
elapsed time: 0.137299815 seconds (62236 bytes allocated)

Removing backtrace collection gives:

elapsed time: 0.165405547 seconds (16113012 bytes allocated)
elapsed time: 0.125439228 seconds (154064 bytes allocated)

Painful, indeed! The best way to work around this would be to know in advance which exception handlers want backtraces, which is a bit awkward.

[simonster]

If we did a better job of generating code for union types, would it be possible to return a different type if parsing fails? Then you could either check for that or have your code throw when you try to use the value.

[stevengj]

@JeffBezanson, whoops, I meant to write isFloat(s) && v. When I do that, I get a 100x slowdown, similar to you:

elapsed time: 34.781770763 seconds (16000048 bytes allocated)@time isFloat2(rs)
elapsed time: 0.224690652 seconds (112 bytes allocated)

[vtjnash]

would be closed by #9316 (comment) Nullable{Float64} maybefloat64() suggestion

[JeffBezanson]

tryparse added.