Think about typed do-notation

[sobolevn]

I have removed @pipeline in 0.14 release.
It had lots of problems:

1. It was not working properly with Failure types: We have a problem with @pipeline and unwrap() #90
2. It had problems with unwrapping complex values like IOResult and FutureResult
3. It had problems unwrapping containers of different types (you were not allowed to unwrap both Maybe and Result in the same @pipeline
4. It was encouraging imperative code in a functional codebase

I would love to have some solution instead! But, no ideas about how it should work or look like. So, any ideas are welcome.

[sobolevn]

Related: https://github.com/papaver/pyfnz/blob/master/pyfnz/either.py#L144

[anentropic]

FWIW... Parsy (parser combinator library) has a way of mimicking do notation using generators and yield expressions:
https://parsy.readthedocs.io/en/latest/ref/generating.html#motivation-and-examples

their example:

@generate("form")
def form():
    yield lparen
    exprs = yield expr.many()
    yield rparen
    return exprs

if roughly equivalent to:

form = do
    lparen
    exprs <- expr.many()
    rparen
    exprs

...where form, lparen, rparen and expr.many() are all Parser monads

the correspondence works well enough, in this specific domain, that I was able to translate a bunch of code from Haskell's Megaparsec library into Python in a fairly straightforward way

usefully the generator/do form is not required, you can still use these objects in ordinary expressions like myparser = form | (lparen > other_token < rparen)

I don't know how similar or different this is to what you tried already, or if it's not useful, but if it's helpful at all then I'm glad 😄

[sobolevn]

@anentropic I have tried this approach. It works fine. But, it is impossible to type it properly.

Because, generators require to use Generator or Iterator return types. And these types require a specific generic parameter, ie: Generator[_SendType, _ReturnType, _YieldType]

When this parameter is specified, it becomes impossible to use other types inside your do-notation.

[sobolevn]

Currently I am thinking about this API:

assert do(
   x + y
   for x in wrap(Some(1))
   for y in wrap(Success(2))
) == 3

But, this API still does not work properly with IO values. How can we unwrap IO in this comprehension safely?
It does mean that for z in wrap(IO(1)) should return raw 1 value, but the result of do(...) would have IO type on top of it.

Ideas?! 💯

[sobolevn]

@papaver since I feel very inspired by your implementation I would love to hear your opinion and ideas about it! 🙂

Related: https://github.com/papaver/pyfnz/blob/master/pyfnz/either.py

[anentropic]

I am curious, I didn't fully understand the typing issue with generators+yield expressions.... and this looks like a generator expression as an argument to a function. Does it not have the same problem?

for x in wrap(Some(1)) seems kind of a cumbersome/weird way to extract a single value, compared to x = yield Some(1) (or maybe it needs to be wrapped... x = yield wrap(Some(1)))

(none of this is intended as a criticism, I am just curious about the issues involved)

[sobolevn]

@anentropic maybe I got your idea wrong. Can you please send a simple working demo?

[sobolevn]

Related: dbrattli/OSlash#12

[sobolevn]

gcanti/fp-ts#1262

[papaver]

@sobolevn i ended up going the generator route as it resulted in the cleanest code while still looking like python. all the other implementations i ran into were fairly gross looking and convoluted to force the code to do something it wasn't meant for (like the decorator with dozens of yields). the for comprehension was was able to do a form of 'lifting' for free, cleanly.

i actually use that pattern in code all the time now. most people just don't understand the power of the for comprehension.

    # run validator on cut data, convert errors into issues as well
    def tryValidation(validator, cutData):
        idx, cut, take, shot = cutData
        def toIssue(e):
            return {
                'issue'     : 'ValidationIssue',
                'cut_idx'   : idx,
                'shot'      : shot and shot.code or None,
                'take'      : take and take.code or None,
                'validator' : validator.__name__,
                'error'     : repr(e) }
        return Try(validator, cutData).recover(toIssue)

    # keep track of all the issues found
    issues = []
    def validateCutlist(cutlist, validators):
        [issues.append(issue)
         for validator in validators
         for cutData in cutlist
         for issue in tryValidation(validator, cutData)
             if is_some(issue)]

personally i think typed python is a step backwards so i haven't messed with it. python is extremely powerful functionally without types.

[sobolevn]

We can also try to use something similar to Elixir's notation with with:

Like so:

with (
    result as success_value,
    needs_raw(success_value) as returns_raw,
):
     print(returns_raw)