Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Think about typed do-notation #392

Open
sobolevn opened this issue May 29, 2020 · 18 comments
Open

Think about typed do-notation #392

sobolevn opened this issue May 29, 2020 · 18 comments
Labels
enhancement help wanted question

Comments

@sobolevn
Copy link
Member

@sobolevn sobolevn commented May 29, 2020

I have removed @pipeline in 0.14 release.
It had lots of problems:

  1. It was not working properly with Failure types: #90
  2. It had problems with unwrapping complex values like IOResult and FutureResult
  3. It had problems unwrapping containers of different types (you were not allowed to unwrap both Maybe and Result in the same @pipeline
  4. It was encouraging imperative code in a functional codebase

I would love to have some solution instead! But, no ideas about how it should work or look like. So, any ideas are welcome.

@sobolevn sobolevn added enhancement help wanted question labels May 29, 2020
@sobolevn
Copy link
Member Author

@sobolevn sobolevn commented May 29, 2020

@anentropic
Copy link

@anentropic anentropic commented Jun 9, 2020

FWIW... Parsy (parser combinator library) has a way of mimicking do notation using generators and yield expressions:
https://parsy.readthedocs.io/en/latest/ref/generating.html#motivation-and-examples

their example:

@generate("form")
def form():
    yield lparen
    exprs = yield expr.many()
    yield rparen
    return exprs

if roughly equivalent to:

form = do
    lparen
    exprs <- expr.many()
    rparen
    exprs

...where form, lparen, rparen and expr.many() are all Parser monads

the correspondence works well enough, in this specific domain, that I was able to translate a bunch of code from Haskell's Megaparsec library into Python in a fairly straightforward way

usefully the generator/do form is not required, you can still use these objects in ordinary expressions like myparser = form | (lparen > other_token < rparen)

I don't know how similar or different this is to what you tried already, or if it's not useful, but if it's helpful at all then I'm glad 😄

@sobolevn
Copy link
Member Author

@sobolevn sobolevn commented Jun 9, 2020

@anentropic I have tried this approach. It works fine. But, it is impossible to type it properly.

Because, generators require to use Generator or Iterator return types. And these types require a specific generic parameter, ie: Generator[_SendType, _ReturnType, _YieldType]

When this parameter is specified, it becomes impossible to use other types inside your do-notation.

@sobolevn
Copy link
Member Author

@sobolevn sobolevn commented Jun 25, 2020

Currently I am thinking about this API:

assert do(
   x + y
   for x in wrap(Some(1))
   for y in wrap(Success(2))
) == 3

But, this API still does not work properly with IO values. How can we unwrap IO in this comprehension safely?
It does mean that for z in wrap(IO(1)) should return raw 1 value, but the result of do(...) would have IO type on top of it.

Ideas?! 💯

@sobolevn
Copy link
Member Author

@sobolevn sobolevn commented Jun 25, 2020

@papaver since I feel very inspired by your implementation I would love to hear your opinion and ideas about it! 🙂

Related: https://github.com/papaver/pyfnz/blob/master/pyfnz/either.py

@anentropic
Copy link

@anentropic anentropic commented Jun 25, 2020

I am curious, I didn't fully understand the typing issue with generators+yield expressions.... and this looks like a generator expression as an argument to a function. Does it not have the same problem?

for x in wrap(Some(1)) seems kind of a cumbersome/weird way to extract a single value, compared to x = yield Some(1) (or maybe it needs to be wrapped... x = yield wrap(Some(1)))

(none of this is intended as a criticism, I am just curious about the issues involved)

@sobolevn
Copy link
Member Author

@sobolevn sobolevn commented Jun 25, 2020

@anentropic maybe I got your idea wrong. Can you please send a simple working demo?

@sobolevn
Copy link
Member Author

@sobolevn sobolevn commented Jul 5, 2020

Related: dbrattli/OSlash#12

@sobolevn
Copy link
Member Author

@sobolevn sobolevn commented Jul 22, 2020

gcanti/fp-ts#1262

@papaver
Copy link

@papaver papaver commented Aug 27, 2020

@sobolevn i ended up going the generator route as it resulted in the cleanest code while still looking like python. all the other implementations i ran into were fairly gross looking and convoluted to force the code to do something it wasn't meant for (like the decorator with dozens of yields). the for comprehension was was able to do a form of 'lifting' for free, cleanly.

i actually use that pattern in code all the time now. most people just don't understand the power of the for comprehension.

    # run validator on cut data, convert errors into issues as well
    def tryValidation(validator, cutData):
        idx, cut, take, shot = cutData
        def toIssue(e):
            return {
                'issue'     : 'ValidationIssue',
                'cut_idx'   : idx,
                'shot'      : shot and shot.code or None,
                'take'      : take and take.code or None,
                'validator' : validator.__name__,
                'error'     : repr(e) }
        return Try(validator, cutData).recover(toIssue)

    # keep track of all the issues found
    issues = []
    def validateCutlist(cutlist, validators):
        [issues.append(issue)
         for validator in validators
         for cutData in cutlist
         for issue in tryValidation(validator, cutData)
             if is_some(issue)]

personally i think typed python is a step backwards so i haven't messed with it. python is extremely powerful functionally without types.

@sobolevn
Copy link
Member Author

@sobolevn sobolevn commented Aug 29, 2020

We can also try to use something similar to Elixir's notation with with:

Like so:

with (
    result as success_value,
    needs_raw(success_value) as returns_raw,
):
     print(returns_raw)

@kyprifog
Copy link

@kyprifog kyprifog commented Oct 29, 2020

This is maybe obvious and the reason you chose it, but I like with and as because its consistent with how with is used here:

with open(file, "w+") as f:
       ...

which is also consistent with how Resource is used in scala cats and other monadic behavior. I'm not crazy about the needs_raw notation though and it would make sense to stick with bind to indicate flatmap like:

with (
     fetch_api_keys() as apikeys
     bind(get_user(apikeys)) as user
     bind(get_settings(apikeys)) as settings
     get_local_config() as local_config
     do_stuff(user, settings, local_config)
) as final

@kyprifog
Copy link

@kyprifog kyprifog commented Nov 12, 2020

@sobolevn any other thoughts about this? Is anyone working on this yet?

@sobolevn
Copy link
Member Author

@sobolevn sobolevn commented Nov 12, 2020

Nope, this issue is not in the works right now. I like the generator expression approach:

do(
  x + y + z
  for x in c(Some(1))
  for y in c(Some(2))
  for z in c(Some(3))
)

With the support of async generators for Future.

Do you want to work on it?

@sobolevn
Copy link
Member Author

@sobolevn sobolevn commented Dec 6, 2020

@bgrounds
Copy link

@bgrounds bgrounds commented Jul 14, 2021

Currently I am thinking about this API:

assert do(
   x + y
   for x in wrap(Some(1))
   for y in wrap(Success(2))
) == 3

But, this API still does not work properly with IO values. How can we unwrap IO in this comprehension safely?
It does mean that for z in wrap(IO(1)) should return raw 1 value, but the result of do(...) would have IO type on top of it.

Ideas?! 💯

A couple of thoughts here:

  • I don't believe you can typically mix different types of monads in the same do block (in pure functional languages that enforce strong static types). This is the problem that monad transformers solve. In this particular case, you'd probably just convert your Maybe value into a Result value and use 2 Result values in the do block.
  • I think the result of do(...) should be wrapped in IO. 3 is a pure value, but x + y happens in the impure IO context. Once you enter IO, you cannot get back; you just have to keep working in IO.
  • what is wrap? Is it just a way to turn a monadic type into a generator? I wonder if there's a way to bake generator functionality into the monadic types themselves so you could avoid the wrap function...
  • I think using generator expressions for do notation is a really cool idea. I don't know if they're equally powerful, but this reminds me of LINQ in C#. Language-Ext is a comprehensive C# library that uses LINQ for do notation, which could be a handy reference for this work.

@bagrounds
Copy link

@bagrounds bagrounds commented Jul 15, 2021

Illustration in Haskell:

( do
  x <- note "maybeX was Nothing!" (Just 1)
  y <- Right 2
  return (x + y) ) == Right 3

https://replit.com/@bagrounds/do-notation#main.hs

@internetimagery
Copy link

@internetimagery internetimagery commented Oct 6, 2021

@sobolevn sobolevn mentioned this issue Feb 27, 2022
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement help wanted question
Development

No branches or pull requests

7 participants