RFC: Convenience syntax to unwrap Nullables (à la Swift if let)

[nalimilan]

I'd like to discuss the opportunity and possible implementations of a syntax similar to Swift's if let. The idea is to offer a compact syntax to check whether a Nullable is null, performing an action on its value if it isn't (with an optional fallback if it is).

This was mentioned recently on julia-users: https://groups.google.com/d/msg/julia-users/pgaw6VnBJ34/bBe2lJ1mAQAJ

I gave a try at writing a macro offering a similar feature in Julia, and as expected it can be made to work quite easily. There are a few issues to discuss, though:

1. First, do people think it would be worth having in Base? I think making it easy to work with Nullable in a rigorous way is a major feature, and this seems to be a quite common opinion nowadays.
2. @if let cannot be used because of parsing issues. That's not a big deal IMO, as I'm not a fan of that naming choice. I've retained @unwrap if, but we could find a better name (if is required to allow else).
3. I've chosen to override the same variable binding using let x=get(x) (i.e. you access the value with the same name as the Nullable). I think this is clearer as there's no reason to need both variables in the same context, and it avoids prompting people to use artificially different names for the "same" thing.
4. I think an alternative solution to this ad-hoc macro would be extend the do syntax to allow for an else part, which would be implemented by passing two functions to get (one for when a value is present, one for when it isn't). The advantage is that it could be useful in other places: for example to allow working efficiently with dicts, with code to update a value if found, and code returning a default value (?= "get or set" operator #2108).

So, here's the macro, and examples of how it can be used:

macro unwrap(ex)
    if !isa(ex, Expr) || ex.head != :if
        throw(ArgumentError("@unwrap expects an if or if...else block"))
    end

    x = esc(ex.args[1])

    if length(ex.args) == 3
        quote
            if !isnull($x)
                let $x = get($x)
                    $(esc(ex.args[2]))
                end
            else
                $(esc(ex.args[3]))
            end    
        end
    else
        quote
            if !isnull($x)
                let $x = get($x)
                    $(esc(ex.args[2]))
                end
            end
        end
    end
end

v = Nullable(1)

@unwrap if v
    res = v + 1
end

@unwrap if v
    res = v + 1
else
    res = 0
end

Cc: @johnmyleswhite

[johnmyleswhite]

I'll read this more carefully in a bit, but I completely agree that we should try to add some sugar for this use case.

[eschnett]

Instead of using a macro you can consider using Julia's do syntax. This could look as follows:

nullable{T}(f, n::Nullable{T}, alt::T) = isnull(n) ? alt : f(get(n))

nullable(n, 0) do x
    x+1
end

The first argument of the function nullable (which could have a different name, of course), when used with the do notation, is the nullable value, the second argument is the fallback value. Different from your example above, the fallback is calculated all the time, not only when the nullable is null.

We could extend Julia's do syntax with an else statement, and then it would look like

nullable{T}(f, alt, n::Nullable{T}) = isnull(n) ? alt() : f(get(n))

nullable(n) do x
    x+1
else
    0
end

The main difference to the suggestion above is that one uses lambda expressions and existing syntax instead of a macro.

[JeffreySarnoff]

+1 for the intent

rather than @unwrap if v ..
something less clinical, like
@onlyif v # if !isnull(v) then .. end
or not a macro, onlyif(v)

In addition to the sugar, how about some honey?

 res = v ? v + 1 : 0
 #   = if !isnull(v) then  v + 1 else 0 end

[nalimilan]

@eschnett Yes, see my point 4. But it requires a small parsing change (which I think it would be justified).

@JeffreySarnoff I don't think that can be made to work, at least not without making quite profound changes in many parts of Julia. Automatic unwrapping of Nullable without any macro doesn't sound like something that would be accepted into Base.

[vchuravy]

I had something similar in mind with allowing map for Nullables (#9446), but the ability of actually specifying a else value is nice.

[johnmyleswhite]

After thinking about this more, I'm ok with it, but I don't know what others want to do with Nullables in the long run.

A few counterpoints (which may just reflect me being confused):

• In the if+else case, it seems like this actually produces code that's longer than you'd have if you get(v, alt).
• I don't believe this simplifies nesting of nullables as cleanly as possible. For an example of what I believe to be the shortest possible code, check out the use of flatMap in this Scala example: http://www.codewithstyle.info/2016/02/scalas-option-monad-versus-null.html

On my end, I've come to think that the best way to deal with the verbosity of nullables is to provide DSL's in which lifting happens totally automatically. But implementing those may be impossible without formal return types or some small core of arrow types at the heart of Julia. So I've kind of given up on trying to make progress.

[eschnett]

There is one major difference between get with fallback and the mechanism proposed here: If you want to perform an operation on the value, then get forces you to perform the same operation on the fallback value. In most cases that wouldn't make sense. Thus the mechanisms here take two inputs, a function (!) to be applied to the nullable value, and a fallback value (or a fallback function that calculates a value).

Regarding longer/shorter code: You can write e.g. nullable(x->x+1, n, 0) instead of using the do syntax.

If you want to wrap the result of the calculation back into a nullable, then you'd want to implement map. (That is again a different case from what is suggested above.) map also allows the do syntax, and there's no need to specify a fallback (since it is null):

map(n) do x
    x+1
end

Or, of course, map(x->x+1, n) for a nullable n.

Or, with the recently proposed "vectorizing" syntax:

(x->x+1).(n)

I don't think this syntax is intuitive for nullables, though.

[JeffreySarnoff]

I traded monads for arrows and chased them awhile; got bored and let go of that quiver.

[nalimilan]

A few counterpoints (which may just reflect me being confused):

In the if+else case, it seems like this actually produces code that's longer than you'd have if you get(v, alt).

@johnmyleswhite Yeah, as @eschnett said, this syntax is only useful for more complex situations where you don't want to simply return a value, but to perform a different action (like raising an error, or computing an alternative result).

I don't believe this simplifies nesting of nullables as cleanly as possible. For an example of what I believe to be the shortest possible code, check out the use of flatMap in this Scala example: http://www.codewithstyle.info/2016/02/scalas-option-monad-versus-null.html

That's interesting, but I must admit I find the C# syntax presented there is much simpler to grasp than Scala's flatMap, and quite shorter. I think we should make more use of ? for working with Nullables.

Taking again inspiration from Swift, the macro I posted below could/should be extended to support multiple/nested Nullable, like this (it's even terser than Swift's due to not allowing for different variable bindings):

@unwrap if x, x.y, x.y.z
    ...
end

which would expand to:

if !isnull(x)
    let x = get(x)
        if !isnull(x.y)
            let y = get(x.y)
                ... # Replace x.y with y in user code
            end
        end
    end
end

Would that suit your needs?

On my end, I've come to think that the best way to deal with the verbosity of nullables is to provide DSL's in which lifting happens totally automatically. But implementing those may be impossible without formal return types or some small core of arrow types at the heart of Julia. So I've kind of given up on trying to make progress.

Yes, I know that's a tough issue...

---

I don't think this syntax is intuitive for nullables, though.

@eschnett I think it's non-intuitive when combined anonymous functions. :-)

[johnmyleswhite]

Your proposal for multi-expression @unwrap sounds reasonable. I bet it'll be really hard to make work, but it would be worth adding to Julia IMO.

[nalimilan]

Why would it be hard? Sounds quite doable to me, though my macro skills are still quite limited...

[johnmyleswhite]

My experience with macros is that there's just a lot of cases to get right. For example, a mixed chain of nullables and non-nullables in x.y.z will probably break your current expansion.

[nalimilan]

Right, it will make things more complex, but a few conditions on types which would be optimized out by the compiler could work.

[johnmyleswhite]

Indeed. I think you should try to make this work and submit a PR, but my vote isn't sufficient for inclusion in Base. That said, I think it's a good start towards getting us some of the sugar we need. (And I also generically have a lot of Swift envy at this point, so I like copying stuff from Swift.)

[nalimilan]

I'd like to get more opinions from core devs before spending time on the more complex solution. Also, the simple macro above is a good start to evaluate whether this kind of syntactic sugar is useful.

[eschnett]

There are two different cases, depending on what the result of the calculation should be:

• the result could be again a nullable, maybe with a new value type
• the result could be a non-nullable value

Which of these cases do you want to address with a macro? The former is similar to map, the latter similar to reduce.

In the first case, you also need to decide whether the calculation that runs in the non-null case is allowed to return null.

[nalimilan]

@eschnett There isn't really a "result" in that syntax. The point isn't to apply an operation and return a value, it's to run code on the value wrapped in a Nullable (or several nested ones) after checking that it's not null. The code can be anything, and may evaluate to nothing. Another syntax would be needed for what you describe.

[eschnett]

@nalimilan In my classification, since the code can return "anything", that's case two.

[eschnett]

@nalimilan When I wrote "result", I meant result in the sense that (almost) all statements in Julia are actually expressions, and can produce a result, and you can assign it to a variable. That "result" might be nothing, or you might ignore it.

The key distinction I'm trying to make is whether the outcome of the calculation should always be another nullable (as you'd expect with a map-type operation), or not (as you'd expect e.g. with a reduction-type operation). Here, clearly you don't want to restrict yourself to always producing a nullable.

This is -- in my mind -- my case 2 above. If you disagree, then that's because I described my case 2 badly (apologies), not because I'm thinking of something else.

[nalimilan]

No worries. It's just that I haven't considered seriously the question of return values, as it's not the primary goal of this syntax. Actually, the macro above does not allow assigning its result to a variable, because if blocks do not seem to allow saving the returned value:

julia> y = if true
           1
       end

julia> dump(y)
Void nothing

This is quite weird, as the return value is indeed printed at the REP. Maybe a bug?

[KristofferC]

Works on 0.4.3. Do you build after #15188?

[nalimilan]

@KristofferC Ah, that was likely it. I haven't rebuilt yet. Let's get back to the design discussion then. :-)

[nalimilan]

@StefanKarpinski @JeffBezanson What's your opinion on this kind of syntactic sugar?

[hayd]

An alternative is to use for like in Scala...

Base.start(::Nullable) = 1
Base.next(n::Nullable, i::Integer) = n.value, i + 1
Base.done(n::Nullable, i::Integer) = isnull(n) | (i > 1)

although for blocks don't return anything :(

[nalimilan]

@hayd Sounds interesting, but AFAICT that would be quite verbose when chaining as in @unwrap if x, x.y, x.y.z. Also, it wouldn't support else clauses.

[ararslan]

I also agree that some convenient way of dealing with this would be nice, though I dislike the particular proposed use of if in the macro syntax, since the "condition" in the if isn't actually a boolean. I feel like that could be pretty confusing as it goes against how if works in a normal context.

[KristofferC]

Seems stale since Nullable has moved out of Base.