Implement multiple dispatch

[EmilyBourne]

Describe the bug
Sometimes we want different functions to run different methods depending on the type. In python this is often handled with the isinstance decorator, however in pyccel this would be unmanageable. We would therefore prefer to use multiple dispatch. Implementation of this would fix issue #802

Test Code
Provide code which does not currently work but which should do when this issue is fixed:

@dispatch('float')
def get_real(a : 'float'):
    return a

@dispatch('complex')
def get_real(a : 'complex'):
    return a.real

The dispatch decorator can either be written ourselves in pyccel or we could add a dependency to multipledispatch (last pypi update Jun 2023)

This code should be translated to an Interface. In C and fortran this should work out of the box, in python we need to print the individual functions with the dispatch decorator when necessary. Ideally these functions should be grouped if the code is identical. E.g:

@dispatch(('int'),('float'))
@template('T',['int','float'])
def get_real(a : 'T'):
    return a

@dispatch('complex')
def get_real(a : 'complex'):
    return a.real

[EmilyBourne]

Blocked by #886

[EmilyBourne]

@yguclu This issue was opened following a discussion with you and @ratnania . Why has it been converted to a discussion?

[yguclu]

I am currently listing Pyccel projects as discussions, as they will need a discussion among developers before the actual implementation.

[EmilyBourne]

Ok. Which points of this issue do you think need clarifying by a discussion? The choice of the python implementation of @dispatch?

[yguclu]

Of course we could implement this in pure Python, following the multipledispatch library. Faster implementations would have to be done in Fortran and C. If we decide to do this, we have to discuss how the generated code should look like.

[EmilyBourne]

I'm not sure I see what needs discussing? The generated code is the easy case as we already have Interfaces in Fortran and C. This already requires us to print a different function for each combination of argument types. The case where the content is different does not require any special treatment

[yguclu]

You are right, most of the machinery is already implemented and is used to handle different argument types for a single function.

I still think that a few things have not been clarified yet. For example, do we need to provide arguments to the @dispatch decorator if we use type annotations (or the @type decorator)? This code duplication looks confusing to me, and error prone. Further, as you mentioned, the Python implementation should be decided.

[EmilyBourne]

An alternative to the multipledispatch library would be the Python built-in functools.singledispatch

It works like this:

from functools import singledispatch


@singledispatch
def func(val):
    raise NotImplementedError


@func.register
def _(val: str):
    print('This is a string')


@func.register
def _(val: int):
    print('This is an int')

This seems like something that would be easy to print, although it requires slightly more effort to parse. However I'm not sure how NumPy types should be managed

[said-hadjout]

We can use the @overload decorator provided by the typing module https://docs.python.org/3/library/typing.html#overload.
I don't think we need to worry about the Python implementation, we can use it as a first step internally for the external libraries, and decide afterward if we want to expose it to the user

[EmilyBourne]

I agree that the typing module could be helpful however I disagree about the Python implementation. It is often frustrating to have to deal with Python as a special case when it should be trivial.

I have had a look at the typing module. At first I thought it took care of the dispatch but instead it seems closer to @template in that it lets you group combinations of arguments. I therefore don't think that this is the right solution as in Pyccel we always ensure that our implementations reproduce the same behaviour as in Python. I don't think this would be the case with an implementation using @overload unless I've missed something?

[said-hadjout]

The Python implementation is needed if we need to expose it to the user, for the moment we can just create our own decorator that does nothing, maybe this way is better

[EmilyBourne]

The Python implementation is needed if we need to expose it to the user

It is also needed for the tests (which was the original reason for opening this issue)

[said-hadjout]

Yes, it is preferable for it to work in Python, but if we can't decide about the Python behavior, we can postpone it until it is clear, because regardless of the behavior we want to give it in Python, the implementation in Fortran/C will be the same

[EmilyBourne]

I agree that the implementation in Fortran/C won't change. But I don't think it should be difficult to reach a consensus on a method that would work for Python so I don't see any reason to write an implementation that we plan to replace when we can just think a little longer and get it right first time.

[EmilyBourne]

I do think we should try and find a solution to this though. This issue comes back repeatedly. Currently we have 4 suggestions:

1. @dispatch from multipledispatch
2. @singledispatch from functools
3. @overload from typing
4. Create our own decorator

[said-hadjout]

yes, it is a problem. if singledispatch does not work with array, then the only solution that I see for the moment is to create our own decorator, and maybe change it in the future if we see the need to

[EmilyBourne]

Maybe we can try and discuss this in tomorrow's Pyccel meeting?
@said-hadjout are you on the discord?

[said-hadjout]

Yes, discussing that in the meeting is better, but no, I'm not on discord

[EmilyBourne]

The link is here: Pyccel Discord Server.

It looks like someone tried to add you at some point but you never accepted the invite. We usually share the links for meetings on there

[said-hadjout]

I'm in