fix issue 9959 - Add functional pattern matching for object references

[idanarye]

Polymorphism is usually the way to write code that handles objects of different
types, but it is not always possible or desirable. When the code that uses the
objects is required to treat objects differently based on class, the D way is
using cast&assign inside if:

    if (auto a = cast(A) obj)
    {
        ...
    }
    else if (auto b = cast(B) obj)
    {
        ...
    }

This is not always convenient for two reasons:

• You need to write obj in every if statement. If obj is a more complex
  expression(like the return value of a function) you'll need to store it in an
  variable beforehand. This is not that cumbersome but still worth mentioning.
• An if statement is a statement - which means it does not return a value -
  and creates it's own scope - which means you variables declared in it are not
  accessible outside. Those two constraints mean that if you need to compute a
  value differently based on the object's class and use that value after the
  ifs, you need to declare the variable before the if - and that means you
  can't make it const.

My solution is the function std.algorithm.castSwitch, which is based on Scala's approach.
It is used like this:

    obj.castSwitch!(
            (A a) => ...,
            (B b) => ...,
            )()

It answers both mentioned problems, plus it is more compact readable.

See http://d.puremagic.com/issues/show_bug.cgi?id=9959

[ghost]

I have a hunch that it should be more efficient to get the typeid of the object and then do comparisons with other typeid's via ==.

[idanarye]

@AndrejMitrovic: It'll probably will, because casting works for subclasses as well. And that's the problem - you usually do want it to work for subclasses.

[ghost]

Give me a few minutes and I'll come up with a working example of what I mean. Also this function needs to make sure a class is tested for the most derived type before trying lesser-derived types.

[dnadlinger]

Or just always match the first choice, maybe adding an assert checking that no choices are hidden.

I have had something similar in my personal toolbox for quite some time now (see https://gist.github.com/klickverbot/808936 for an older, less sophisticated version), and I agree it does come in handy from time to time.

std.algorithm might not be the right module for this, though.

[ghost]

Here is a crude implementation, with an added unittest:

import core.exception;

import std.algorithm;
import std.exception;
import std.string;
import std.traits;
import std.typetuple;

// workaround for typeof(null) not working in DerivedToFront
private final abstract class _SentinelNullClass { }

private template GetClassTypes(T...)
{
    alias choice = T[0];
    alias choiceParameterTypes = ParameterTypeTuple!choice;
    static assert(choiceParameterTypes.length <= 1,
            "A choice function can not have more than one argument.");

    static if (choiceParameterTypes.length == 1)
    {
        alias CastClass = choiceParameterTypes[0];
        static assert(is(CastClass == class),
                "A choice function can not accept a non-class-typed argument.");

        alias Type = CastClass;
    }
    else
    {
        // alias Type = typeof(null);  @bug@: DerivedToFront doesn't work with typeof(null)
        alias Type = _SentinelNullClass;
    }

    static if (T.length > 1)
    {
        alias GetClassTypes = TypeTuple!(Type, GetClassTypes!(T[1 .. $]));
    }
    else
    {
        alias GetClassTypes = Type;
    }
}

auto castSwitch(choices...)(Object switchObject)
{
    alias Classes = GetClassTypes!choices;
    alias OrderedClasses = DerivedToFront!Classes;

    ClassInfo classInfo;
    if (switchObject !is null)
        classInfo = typeid(switchObject);

    // try matching exact type
    foreach (Class; OrderedClasses)
    {
        static if (is(Class == _SentinelNullClass))  // typeof(null)
        {
            if (switchObject is null)
            {
                return choices[staticIndexOf!(Class, Classes)]();
            }
        }
        else if (classInfo == typeid(Class))
        {
            return choices[staticIndexOf!(Class, Classes)](cast(Class)cast(void*)switchObject);
        }
    }

    // find first base class
    foreach (Class; OrderedClasses)
    {
        static if (!is(Class == _SentinelNullClass))  // typeof(null) is handled above
        if (auto castedObject = cast(Class)switchObject)
        {
            return choices[staticIndexOf!(Class, Classes)](castedObject);
        }
    }

    // In case nothing matched:
    throw new SwitchError("Input not matched by any choice");
}

///
unittest
{
    class A
    {
        int a;
        this(int a) {this.a=a;}
    }
    class B
    {
        double b;
        this(double b) {this.b=b;}
    }
    class C
    {
        string c;
        this(string c) {this.c=c;}
    }
    class D { }

    Object[] arr = new Object[5];
    arr[0]=new A(1);
    arr[1]=new B(2.5);
    arr[2]=new C("hello");
    arr[3]=new D();
    arr[4]=null;

    auto results=arr.map!(castSwitch!(
                (A a) => "A with a value of %d".format(a.a),
                (B b) => "B with a value of %.1f".format(b.b),
                (C c) => "C with a value of %s".format(c.c),
                (Object o) => "Object of another type",
                () => "null reference",
                ))();

    assert(results[0] == "A with a value of 1");
    assert(results[1] == "B with a value of 2.5");
    assert(results[2] == "C with a value of hello");
    assert(results[3] == "Object of another type");
    assert(results[4] == "null reference");
}

unittest
{
    class A { }
    class B { }

    //Nothing matches:
    assertThrown!SwitchError((new A()).castSwitch!(
                (B b) => 1,
                () => 2,
                )());

    //Choices with multiple arguments are not allowed:
    static assert(!__traits(compiles,
                (new A()).castSwitch!(
                    (A a, B b) => 0)()
                ));

    //Only object arguments allowed:
    static assert(!__traits(compiles,
                (new A()).castSwitch!(
                    (int x) => 0)()
                ));
}

// test derivation
unittest
{
    class A
    {
        int a;
        this(int a) {this.a=a;}
    }

    class B : A
    {
        double b;
        this(double b) {this.b=b; super(1); }
    }

    class C : B
    {
        string c;
        this(string c) {this.c=c; super(1.0); }
    }

    class D : C { this() { super("D class"); } }

    Object[] arr = new Object[5];
    arr[0]=new A(1);
    arr[1]=new B(2.5);
    arr[2]=new C("hello");
    arr[3]=new D();
    arr[4]=null;

    auto results=arr.map!(castSwitch!(
                (A a) => "A with a value of %d".format(a.a),
                (B b) => "B with a value of %.1f".format(b.b),
                (C c) => "C with a value of %s".format(c.c),
                (Object o) => "Object of another type",
                () => "null reference",
                ))();

    assert(results[0] == "A with a value of 1");
    assert(results[1] == "B with a value of 2.5");
    assert(results[2] == "C with a value of hello");
    assert(results[3] == "C with a value of D class");
    assert(results[4] == "null reference");
}

[idanarye]

@AndrejMitrovic: You are using GetClassTypes just so you can do DerivedToFront on the list of classes, to solve the problem of choice overshadowing. I think that a better solution would be to check for overshadowing and generate an error message, like @klickverbot suggested. This will exempt us from having to generate OrderedClasses and Classes - we could do it all directly on choices - and since I'm writing it specifically for castSwitch, _SentinelNullClass will be redundant as well. This will also allow us to check for duplicate cases.

P.S.: I've noticed that I ignored the fact that classes can have interfaces - and by asserting that is(CastClass == class) I've blocked using interfaces in the choices. I fixed it.

[idanarye]

OK, I implemented the direct type checking and the overshadowing testing.

[ghost]

Overshadowing? I just want it to work with the most derived type. If I have switches for class A and derived class B, and I pass an object of static type A but dynamic type B then I expect the B switch to be taken and not issue any errors.

[dnadlinger]

Also see the std.concurrency.receive API.

[idanarye]

@AndrejMitrovic: switch-like constructs - both build-in syntax and functions - are usually first-fit. Be it pattern-matching in functional languages, cond-like functions in lisp dialects or if-else chains. The only exception I'm aware of is in flex, and even then it's longest-fit, not best-fit, and they have a good reason to do that: it's not always possible to order the patterns in a longest-to-shortest order, and using first-fit will make the patterns more complex.

@klickverbot mentioned std.concurrency.receive. It also works with first-fit:

import std.stdio;
import std.variant;
import std.concurrency;

void spawnedFunction()
{
    receive(
            (long l) { writeln("Received a long."); },
            (int i) { writeln("Received an int."); },
           );
}

void main()
{
    auto tid = spawn(&spawnedFunction);
    int x=1;
    send(tid, x);
}

This will print Received a long., even though int is a better fit - because the long handler was first.

This is also the case with two of my other xSwitch pull requests - predSwitch and regexSwitch - where doing a best-fit will be extremely difficult(regexSwitch) if not outright impossible(predSwitch).

Now, using first-fit is usually the easiest method, and sometimes it's the only possible method, and that's it became the standard - but even in cases where best-fit is possible they prefer to use first-fit for consistency, and possibly prevent an error when an early handler catches all the cases that a later handler accepts and provide an error or just a warning.

If B derives from A, you just put the B handler before the A handler. If you put A before B, then the A handler overshadows the B handler, because any case that can be handled by the B handler can also be handled by the earlier A handler.

Now, if you did that in my original implementation, the A handler will be chosen. If you did that in your implementation suggestion, the B handler will be chosen. This means that your implementation would surprise me, and anyone else who thinks that first-fit is the way to go, and my implementation would surprise you and anyone else who thinks that best-fit is the way to go. But a clear error message that says "you can't put this handler before the other handler because it overshadows it" should not surprise anyone - and even if it did, it'll do it at compile time and not as an hard-to-detect runtime bug.

[CyberShadow]

But a clear error message that says "you can't put this handler before the other handler because it overshadows it" should not surprise anyone - and even if it did, it'll do it at compile time and not as an hard-to-detect runtime bug.

I think this is the best solution. Pattern matching is a very common idiom across many programming languages, and it's important to match existing behavior. The fact that there is a compile-time error that removes any chance of ambiguity just seals the deal.

[CyberShadow]

How about mentioning that the user can override this by using an Object-taking delegate, as any class can be cast to Object?

[andralex]

status?

[John-Colvin]

merge conflict, please rebase

[quickfur]

ping
Any updates since last comments?

[idanarye]

@quickfur Sorry, I wanted to get back to this and to #1259 when it was decided to postpone #1392, but I didn't have the chance. There is a little military conflict going on here at Israel and I'm part of the reserve force so I'm occupied right now, but I'll try to get back to it once it's done.

[idanarye]

That last update was just a rebase - this PR is not ready for a merge yet. I still want to add the all-void-handlers and the always-throwing-void-handlers functionalities(like the ones I added to #1392).

[idanarye]

OK: now this PR is ready. Changes from where we left it more than a year ago:

1. void handlers are supported if they guarantee to throw an exception. The check is done at runtime - if the void handler doesn't throw, castSwitch throws a SwitchError.
2. If all the handlers are void, they don't have to throw and castSwitch itself returns void
3. imports moved to the scope where they are used.
4. Added to the example unittest a demonstration of the new void handlers functionality.
5. Added to the other unit tests some checks to verify the new void handlers functionality's behaviour in the null cases.

[idanarye]

@Dicebot Can you remove the needs work label please? This PR is again read for review, and you'll also bump it on the way(it is currently at the bottom of the list since I was the only one commenting here...)

[mihails-strasuns]

Thanks for pinging and thanks for doing needed work :)

[mihails-strasuns]

(I won't be able to review it until this weekend at the very least)

[idanarye]

Why is it still on the bottom of the list? Isn't it supposed to get bumped when a Collaborator comments?

[yebblies]

Why is it still on the bottom of the list? Isn't it supposed to get bumped when a Collaborator comments?

No.

[jacob-carlborg]

I just noticed this pull request. I see that there are two other related pull requests, #1392 and #1259. Wouldn't it be better to try and get a more general solution packed in a single function. This would be more how pattern matching works in functional programming languages.

[idanarye]

@jacob-carlborg I was also thinking that a bigger, united pattern matching function is in order, but I still believe there is room for predSwitch, castSwitch and regexSwitch. I opened a forum thread about it - http://forum.dlang.org/thread/lgwebwiheubnibuhxfwy@forum.dlang.org

[idanarye]

I added support for multi-argument handlers. I still need to add a Tuple unwrapping(so we won't have to use expand when sending in a tuple), update the documentations, and write unit tests.

Since the single-argument castSwitch was pretty much complete and these changes are a big risk, I'm currently adding them as a separate commit. If they are approved I'll squash the commits - it not I can easily go back to the single-argument version.

[idanarye]

OK, the multi-argument handler implementation is complete, with updated documentation and unit tests and some fixes for some bugs that the new unit tests discovered(yey unit testing!).

The main changes from the single-argument version:

• Support for multiple arguments in handlers(Duh!)
• Support for non-object argument types. Supporting these made little sense for the single-argument version, since only objects support runtime type detection, but now that we support multi-argument handlers we might want to send an entire tuple to them(think multi-methods) so it can be useful.
• Instead of the empty tuple matching null as a special case, now typeof(null) matches null, handlers' argument types lists get right-padded with typeof(null)s until they are the same length of the object list. This is backward-compatible with that special case from before(not that backward compatibility to a pending PR matters...) and supports having typeof(null) in the middle of the argument list.
• void* is added as the wildcard - it matches everything.
• If the only switch object is a Tuple, it gets expanded automatically.
• In the unit tests, since there are lots of handlers with unused arguments, I changed the argument names from a, b etc. to _1, _2, etc. I feel that this conveys better that these arguments are not used. Removing the arguments doesn't work - (A) => ... treats A as the argument's name, not type.

@Dicebot: You warned me from over-complicating this, but while the implementation got a bit complicated due to all the edge cases, the interface is kept simple and I believe multiple argument support can be very useful here, for the same reason multi-methods are useful

[mihails-strasuns]

Overall concept looks ok. However I don't like few things:

1. unittest examples look overly formal with all the placeholder names - it is much better to provide fewer examples but ones that operate of some imaginary domain, thus answering not only "how" question but also "why".
2. void* as a wildcard seems rather bad decision. It implies that input can be any reference type (including arrays, pointers etc) which does not seem to be the case. Is there any reason why you can't use no-argument lambda as "match all" case?

Documentation / test thing is probably most difficult to address but is also one of most important.

[idanarye]

@Dicebot:

1. I'll get rid of the placeholders if you don't like that style, but I don't think it's feasible to write short enough examples for an actual (even imaginary) domain. castSwitch requires classes to demonstrate it's potential(even though it supports primitives and structs, they don't support runtime polymorphism) and you can't write several meaningful classes in under 40 lines...

2. The reason why I can't use no-argument lambdas as match-all is that I want to support multi-argument lambdas. So if you send two switch-objects, a zero-arguments lambda will mean "match-all for both", and a single-argument lambda will mean "match-all for the second slot", but what will mean "match-all for the first slot"?

[mihails-strasuns]

Well I'd like other reviewers to chime in on this but in my opinion one example that covers only 10% of functionality but does it clear practical fashion is much better that set of purely theoretical examples that cover 100% of the functionality. Reading latter is not so much different from just reading function documentation.

2. The reason why I can't use no-argument lambdas as match-all is that I want to support multi-argument lambdas. So if you send two switch-objects, a zero-arguments lambda will mean "match-all for both", and a single-argument lambda will mean "match-all for the second slot", but what will mean "match-all for the first slot"?

I see. What do you thing about special stub interface / class type defined in the same module named something like "MatchAny"? Not sure it is actually better though.

[idanarye]

@Dicebot How about this for a practical example:

import std.conv;

interface Vertex {}
class Node : Vertex
{
    Vertex left;
    Vertex right;
    this(Vertex left, Vertex right) { this.left = left; this.right = right; }
}
class Leaf : Vertex
{
    int value;
    this(int value) { this.value = value; }
}

string toString(Vertex vertex)
{
    return vertex.castSwitch!(
            (Node node) => toString(node.left) ~ " " ~ toString(node.right),
            (Leaf leaf) => leaf.value.to!string(),
            )();
}

Vertex tree = new Node(
        new Node(new Leaf(1), new Leaf(2)),
        new Node(new Leaf(3), new Leaf(4)));

assert(toString(tree) == "1 2 3 4");

---

As for adding something like MatchAny - I don't like it either, since it'll only be used for castSwitch. It might be worthwhile to add it to the monster pattern-matching function I'm planning though.

At any rate, this multi-argument things does seem to get a little out of hand. Maybe I should revert to the old, single-argument implementation and leave the multi-argument and non-object-matches functionality to that full blown pattern-matching function I'll make in the future?

[jacob-carlborg]

At any rate, this multi-argument things does seem to get a little out of hand.

What is the use case to match multiple arguments?

[idanarye]

@jacob-carlborg Well, I got the idea from this thread. I figured that castSwitch is quite similar to single-argument dynamic dispatch, and by adding multiple-argument support we can easily get full multi-dispatch. Didn't go as easy as I thought it would...

Mutli-dispatch aids you in making choices based on a list of argument types. For example(the example given in that thread) - collision between two objects in a game, where you want to decide which action to take based on the types of two arguments.

[DmitryOlshansky]

@idanarye Not looking in great detail but multi-arg type switch is really nice primitive.
Also what you folks think of calling it just typeSwitch?

[idanarye]

@DmitryOlshansky I also think multi-arg is nice, but it turned out to be more complex than the single-arg version and to raise some problems that don't exist or that are marginal in the single-arg version. We have to decide if we want the simpler, less-powerful version or the complex, more-powerful version.

As for your renaming suggestion - when you say typeSwitch what pops in my mind is a compile-time switch-like mechanism for choosing a type based on some criteria - something like std.traits' Select.

[mihails-strasuns]

@idanarye I like that example :)

[idanarye]

@Dicebot OK, the tree example is now in the PR. We still need to decide though if we want multi-argument here or not.

[quickfur]

ping

[idanarye]

A decision is required here - should I revert to the old, simple one-arg version or squash and keep the new, complicated multi-args version?

Should I open a forum thread about it?

[dnadlinger]

Another high-level comment: At first glance, it seems like multi-arg support could transparently be added to the single-arg version. Is this so? Then, the obvious solution would be to merge the single-arg version now, and open a discussion about the improved multi-arg one.

[quickfur]

Good idea, I think we should check in the single-arg version first, and then open another PR for the multi-arg version. It's better to have at least a subset of cases working (i.e., single-arg) then to have nothing at all. PRs that try to do too much tend to be overly complicated and/or take too long to review / refine / merge. Checking things in piecemeal, as long as each step leaves us with a usable, self-consistent subset of the ultimate functionality, is a better approach IMO.

[idanarye]

@klickverbot If GitHub comments had a like button I'd press the one next to yours!

I've reverted this commit to the single-arg version, but not before I splitted the branch with the multi-args version. Once we accept the single-arg version(shouldn't be a problem - it already got re factored to the point of acception by the consensus before I decided to add the mutli-args support.

Once this PR gets accepted, I'll rebase the multi-args branch and open a new PR for it.

[DmitryOlshansky]

LGTM - time to merge?

[quickfur]

Let's do it!

[quickfur]

Auto-merge toggled on