New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a safe pattern and special forms for options #8358

Closed
wants to merge 1 commit into
base: devel
from

Conversation

Projects
None yet
@PMunch
Contributor

PMunch commented Jul 18, 2018

This adds the safer pattern of access from superfunc/maybe, along with adding the maybe logic defined in Toccata to avoid using boolean expressions alltogether.

@mratsim

This comment has been minimized.

Show comment
Hide comment
@mratsim

mratsim Jul 18, 2018

Collaborator

Can't we just overload case for Option?

Collaborator

mratsim commented Jul 18, 2018

Can't we just overload case for Option?

@mratsim

This comment has been minimized.

Show comment
Hide comment
@mratsim

mratsim Jul 18, 2018

Collaborator

Or I like @dom96 suggestion on IRC to call it match

Collaborator

mratsim commented Jul 18, 2018

Or I like @dom96 suggestion on IRC to call it match

@dom96

This comment has been minimized.

Show comment
Hide comment
@dom96
Member

dom96 commented Jul 18, 2018

@mratsim

This comment has been minimized.

Show comment
Hide comment
@mratsim

mratsim Jul 18, 2018

Collaborator

Relevant: >>= name was rejected in PR #6404 for the flatmap name.

Also linked to RFC #7476

Collaborator

mratsim commented Jul 18, 2018

Relevant: >>= name was rejected in PR #6404 for the flatmap name.

Also linked to RFC #7476

@PMunch

This comment has been minimized.

Show comment
Hide comment
@PMunch

PMunch Jul 18, 2018

Contributor

Ah didn't know >>= was rejected elsewhere, that was just a copy from the implementation from @superfunc. And I don't think overloading case is an option as it's a builtin..

Contributor

PMunch commented Jul 18, 2018

Ah didn't know >>= was rejected elsewhere, that was just a copy from the implementation from @superfunc. And I don't think overloading case is an option as it's a builtin..

@bluenote10

This comment has been minimized.

Show comment
Hide comment
@bluenote10

bluenote10 Jul 18, 2018

Contributor

It would be nice if optionCase (or whatever name it will be) is usable as both a statement and an expression. Is this the case? If so, the docs should clarify that (unless I missed it).

Contributor

bluenote10 commented Jul 18, 2018

It would be nice if optionCase (or whatever name it will be) is usable as both a statement and an expression. Is this the case? If so, the docs should clarify that (unless I missed it).

@citycide

This comment has been minimized.

Show comment
Hide comment
@citycide

citycide Jul 18, 2018

Contributor

Most of the IRC discussion seems on point — the logical macros are 🤷‍♂️ but the case one is pretty good. I dislike the names though and hope we can find something better.

This kind of thing is why I think a generic pattern matching construct would be so awesome, ideally in Nim itself as an upgrade to case or a new match expression. patty doesn't currently support a few important features like nesting, multiple patterns per branch, etc. Is this possible in userland and just a current limitation or is it something that needs to be more internal?

Contributor

citycide commented Jul 18, 2018

Most of the IRC discussion seems on point — the logical macros are 🤷‍♂️ but the case one is pretty good. I dislike the names though and hope we can find something better.

This kind of thing is why I think a generic pattern matching construct would be so awesome, ideally in Nim itself as an upgrade to case or a new match expression. patty doesn't currently support a few important features like nesting, multiple patterns per branch, etc. Is this possible in userland and just a current limitation or is it something that needs to be more internal?

Show outdated Hide outdated lib/pure/options.nim
Show outdated Hide outdated lib/pure/options.nim
@andreaferretti

This comment has been minimized.

Show comment
Hide comment
@andreaferretti

andreaferretti Jul 19, 2018

Collaborator

@citycide Neither nesting nor multiple patterns per branch are an instrinsic limitation. In fact, everything that is listed on the README of Patty as currently missing is doable with macros only. It would also be nice to include this macro to support options.

It is just that I could not find the time to work on these other features. Nesting is probably the main one, because it requires the macro to become recursive. After this restructuring is done, adding different kind of base patterns, such as options, arrays and seqs, should be doable easily

Collaborator

andreaferretti commented Jul 19, 2018

@citycide Neither nesting nor multiple patterns per branch are an instrinsic limitation. In fact, everything that is listed on the README of Patty as currently missing is doable with macros only. It would also be nice to include this macro to support options.

It is just that I could not find the time to work on these other features. Nesting is probably the main one, because it requires the macro to become recursive. After this restructuring is done, adding different kind of base patterns, such as options, arrays and seqs, should be doable easily

@PMunch

This comment has been minimized.

Show comment
Hide comment
@PMunch

PMunch Jul 19, 2018

Contributor

Okay, I've completely rewritten most of this now. It has a match macro for case-based pattern matching, it has a conditional continuation operator .? that works both for values and in ifs. And it has a complete set of comparators and logic operators defined for options.

There is one issue in that Nim refuses my overload of >= and simply rewrites it to <= while flipping the argument. This completely breaks the logic of comparators on options. If it's not made possible in the compiler to do this then we must simply rename them to something else. Maybe >=?, <=?, ==? etc.?

Contributor

PMunch commented Jul 19, 2018

Okay, I've completely rewritten most of this now. It has a match macro for case-based pattern matching, it has a conditional continuation operator .? that works both for values and in ifs. And it has a complete set of comparators and logic operators defined for options.

There is one issue in that Nim refuses my overload of >= and simply rewrites it to <= while flipping the argument. This completely breaks the logic of comparators on options. If it's not made possible in the compiler to do this then we must simply rename them to something else. Maybe >=?, <=?, ==? etc.?

@andreaferretti

This comment has been minimized.

Show comment
Hide comment
@andreaferretti

andreaferretti Jul 20, 2018

Collaborator

Just wanted to mention that using match for pattern matching will collide with patty. Of course patty has lower precedence than stdlib when it comes to naming.

Nevertheless, there is an overlap here - options may be just another pattern that patty recognizes.

What should we do?

[ ] Rename match in patty and have two different forms of pattern matching for options vs anything else
[ ] Include option pattern matching in patty
[ ] Deprecate patty and include other forms of pattern matching in stdlib
[ ] Other?

Collaborator

andreaferretti commented Jul 20, 2018

Just wanted to mention that using match for pattern matching will collide with patty. Of course patty has lower precedence than stdlib when it comes to naming.

Nevertheless, there is an overlap here - options may be just another pattern that patty recognizes.

What should we do?

[ ] Rename match in patty and have two different forms of pattern matching for options vs anything else
[ ] Include option pattern matching in patty
[ ] Deprecate patty and include other forms of pattern matching in stdlib
[ ] Other?

@mratsim

This comment has been minimized.

Show comment
Hide comment
@mratsim

mratsim Jul 20, 2018

Collaborator

Same remark as #8369:

or and and are better than optOr and optAnd, but due to potential confusion for Option[bool], I would like a different name for option combinators. Rust uses and_then and or_else for example.

Collaborator

mratsim commented Jul 20, 2018

Same remark as #8369:

or and and are better than optOr and optAnd, but due to potential confusion for Option[bool], I would like a different name for option combinators. Rust uses and_then and or_else for example.

@PMunch

This comment has been minimized.

Show comment
Hide comment
@PMunch

PMunch Jul 20, 2018

Contributor

@mratsim, it's not possible without modifying the internals though to create new non-symbol operators.. Having some confusion for Option[bool] is in my opinion much better than user a regular call operator.

@andreaferretti, we can change the name if people want. match just seemed to be the consensus before my latest changes.

Contributor

PMunch commented Jul 20, 2018

@mratsim, it's not possible without modifying the internals though to create new non-symbol operators.. Having some confusion for Option[bool] is in my opinion much better than user a regular call operator.

@andreaferretti, we can change the name if people want. match just seemed to be the consensus before my latest changes.

@andreaferretti

This comment has been minimized.

Show comment
Hide comment
@andreaferretti

andreaferretti Jul 20, 2018

Collaborator

@PMunch I have nothing against match, but if we introduce pattern matching for options in stdlib it may make sense to introduce a more general construct

Collaborator

andreaferretti commented Jul 20, 2018

@PMunch I have nothing against match, but if we introduce pattern matching for options in stdlib it may make sense to introduce a more general construct

@PMunch

This comment has been minimized.

Show comment
Hide comment
@PMunch

PMunch Jul 20, 2018

Contributor

Ah, that makes sense. This isn't quite pattern matching though. It is just a wrapper around an if and a case statement. All the actual matching is done by the case statement, which I guess is already a general construct.

match ourOption:
  some _ of 0..high(int): echo "It's huge!"
  some x: echo "It's a measly ", x
  none: echo "It's nothing at all"

would get converted to:

let tmp = ourOption
if tmp.isSome:
  case tmp.val:
  of  0..high(int):
    echo "It's huge!"
  else:
    let x = tmp.val
    echo "It's a measly ", x
else:
  echo "It's nothing at all"
Contributor

PMunch commented Jul 20, 2018

Ah, that makes sense. This isn't quite pattern matching though. It is just a wrapper around an if and a case statement. All the actual matching is done by the case statement, which I guess is already a general construct.

match ourOption:
  some _ of 0..high(int): echo "It's huge!"
  some x: echo "It's a measly ", x
  none: echo "It's nothing at all"

would get converted to:

let tmp = ourOption
if tmp.isSome:
  case tmp.val:
  of  0..high(int):
    echo "It's huge!"
  else:
    let x = tmp.val
    echo "It's a measly ", x
else:
  echo "It's nothing at all"
@andreaferretti

This comment has been minimized.

Show comment
Hide comment
@andreaferretti

andreaferretti Jul 20, 2018

Collaborator

@PMunch well, that's exactly what patty does - rewrite into a case statement - but for other types of values otehr than options (objects and variant objects)

Collaborator

andreaferretti commented Jul 20, 2018

@PMunch well, that's exactly what patty does - rewrite into a case statement - but for other types of values otehr than options (objects and variant objects)

@PMunch

This comment has been minimized.

Show comment
Hide comment
@PMunch

PMunch Jul 20, 2018

Contributor

@andreaferretti ah interesting, I'm not really qualified to answer your question though. I'm happy as long as we get a safe and easy pattern for options. The best case would of course be if we could shadow an identifier as well. So if a user tried to do something like this:

let noneOpt = none(int)
match noneOpt:
  some _: echo noneOpt.get
  none: echo noneOpt.get

It would actually throw an error instead of compiling and later throwing a runtime error.

@bluenote10 forgot to answer this. The matching didn't support that when you asked, but they do now.

Contributor

PMunch commented Jul 20, 2018

@andreaferretti ah interesting, I'm not really qualified to answer your question though. I'm happy as long as we get a safe and easy pattern for options. The best case would of course be if we could shadow an identifier as well. So if a user tried to do something like this:

let noneOpt = none(int)
match noneOpt:
  some _: echo noneOpt.get
  none: echo noneOpt.get

It would actually throw an error instead of compiling and later throwing a runtime error.

@bluenote10 forgot to answer this. The matching didn't support that when you asked, but they do now.

Show outdated Hide outdated lib/pure/options.nim
Show outdated Hide outdated lib/pure/options.nim
Show outdated Hide outdated lib/pure/options.nim
Show outdated Hide outdated lib/pure/options.nim
Show outdated Hide outdated lib/pure/options.nim
@krux02

Generally I have to say, I don't like this PR at all. I am sorry to say this, because you probably invested a lot of time in it. I can see that, but overall it feels lake a big hack that tries to make everything compile to something. It really circumvents the safety net that static typing normally brings.

Show outdated Hide outdated lib/pure/options.nim
Show outdated Hide outdated lib/pure/options.nim
Show outdated Hide outdated lib/pure/options.nim
Show outdated Hide outdated lib/pure/options.nim
Show outdated Hide outdated lib/pure/options.nim
Show outdated Hide outdated lib/pure/options.nim
@andreaferretti

This comment has been minimized.

Show comment
Hide comment
@andreaferretti

andreaferretti Jul 20, 2018

Collaborator

I have to say I am not very convinced by this PR. It seems to add 4 things:

  • some converters (A -> Option[A], Option[A] -> bool, ...) which I feel are only detrimental to type safety
  • some utility function. I like some of them (map, filter, or, flatMap, flatten) but others seem to be there just because of similarity with boolean, even if they don't make much sense (and, not)
  • a macro .? for chaining access to fields that may be options - I like this
  • a macro match for pattern matching - I also like this, but I feel that a more general one is needed

My suggestion would be to split this PR into smaller ones that can be discussed independently - I wouldn't like to accept this in toto

Collaborator

andreaferretti commented Jul 20, 2018

I have to say I am not very convinced by this PR. It seems to add 4 things:

  • some converters (A -> Option[A], Option[A] -> bool, ...) which I feel are only detrimental to type safety
  • some utility function. I like some of them (map, filter, or, flatMap, flatten) but others seem to be there just because of similarity with boolean, even if they don't make much sense (and, not)
  • a macro .? for chaining access to fields that may be options - I like this
  • a macro match for pattern matching - I also like this, but I feel that a more general one is needed

My suggestion would be to split this PR into smaller ones that can be discussed independently - I wouldn't like to accept this in toto

@Araq

This comment has been minimized.

Show comment
Hide comment
@Araq

Araq Aug 1, 2018

Member

Note that proposed (by you and me) pattern matching in 'if' has NOTHING common with pseudo-boolean handling. It's just terse form of pattern-matching 'case' as I shown in the desugaring example.

Well so let's add pattern-matching to 'case' then. if takes a bool condition, not a pattern, I still think pattern matching in 'if' doesn't fit Nim's style.

Member

Araq commented Aug 1, 2018

Note that proposed (by you and me) pattern matching in 'if' has NOTHING common with pseudo-boolean handling. It's just terse form of pattern-matching 'case' as I shown in the desugaring example.

Well so let's add pattern-matching to 'case' then. if takes a bool condition, not a pattern, I still think pattern matching in 'if' doesn't fit Nim's style.

@Bulat-Ziganshin

This comment has been minimized.

Show comment
Hide comment
@Bulat-Ziganshin

Bulat-Ziganshin Aug 1, 2018

@Araq the idea is simple - allow

if PATTERN <- EXPR:
  ACTION

as the shorter equivalent of

case EXPR:
of PATTERN:
  ACTION

Is that suitable for Nim or not - is up to you. Of course, it can be implemented only when we will have pattern matching in case.

Bulat-Ziganshin commented Aug 1, 2018

@Araq the idea is simple - allow

if PATTERN <- EXPR:
  ACTION

as the shorter equivalent of

case EXPR:
of PATTERN:
  ACTION

Is that suitable for Nim or not - is up to you. Of course, it can be implemented only when we will have pattern matching in case.

@Araq

This comment has been minimized.

Show comment
Hide comment
@Araq

Araq Aug 1, 2018

Member

Well that arrow notation seems completely arbitrary to me. At least come up with a great syntax please when you push for this non-idiomatic feature. ;-)

Member

Araq commented Aug 1, 2018

Well that arrow notation seems completely arbitrary to me. At least come up with a great syntax please when you push for this non-idiomatic feature. ;-)

@mratsim

This comment has been minimized.

Show comment
Hide comment
@mratsim

mratsim Aug 2, 2018

Collaborator

Let's start with pattern matching for case as everyone seems to agree on it.

Then we can think about more syntactic sugar. I like if let because no symbol that are hard to search, and Nim assignation is let, not <-.

Collaborator

mratsim commented Aug 2, 2018

Let's start with pattern matching for case as everyone seems to agree on it.

Then we can think about more syntactic sugar. I like if let because no symbol that are hard to search, and Nim assignation is let, not <-.

@andreaferretti

This comment has been minimized.

Show comment
Hide comment
@andreaferretti

andreaferretti Aug 2, 2018

Collaborator

I, for one, do not agree in having pattern matching for case. Not that I would dislike it, but doing it with a macro is a great opportunity to show the extensibility of Nim.

It would be really nice to have some form of pattern matching - more general than patty - in the standard library as an importable macro, rather than baking it into the compiler

Collaborator

andreaferretti commented Aug 2, 2018

I, for one, do not agree in having pattern matching for case. Not that I would dislike it, but doing it with a macro is a great opportunity to show the extensibility of Nim.

It would be really nice to have some form of pattern matching - more general than patty - in the standard library as an importable macro, rather than baking it into the compiler

@mratsim

This comment has been minimized.

Show comment
Hide comment
@mratsim

mratsim Aug 2, 2018

Collaborator

@andreaferretti, iirc the goal is for Araq to add caseMatchStmt which similar to forLoopStmt currently, can allow macros to work on case statement.

See this example for forLoopStatement:

macro enumerate(x: ForLoopStmt): untyped =
  expectKind x, nnkForStmt
  # we strip off the first for loop variable and use
  # it as an integer counter:
  result = newStmtList()
  result.add newVarStmt(x[0], newLit(0))
  var body = x[^1]
  if body.kind != nnkStmtList:
    body = newTree(nnkStmtList, body)
  body.add newCall(bindSym"inc", x[0])
  var newFor = newTree(nnkForStmt)
  for i in 1..x.len-3:
    newFor.add x[i]
  # transform enumerate(X) to 'X'
  newFor.add x[^2][1]
  newFor.add body
  result.add newFor

for a2, b2 in enumerate([1, 2, 3, 5]):
  echo a2, " ", b2

Then anyone can metaprogram the case statement for their own types. There is a real need to resolve type though like macro case[Option](x: caseMatchStmt[Option]): untyped = ...

Collaborator

mratsim commented Aug 2, 2018

@andreaferretti, iirc the goal is for Araq to add caseMatchStmt which similar to forLoopStmt currently, can allow macros to work on case statement.

See this example for forLoopStatement:

macro enumerate(x: ForLoopStmt): untyped =
  expectKind x, nnkForStmt
  # we strip off the first for loop variable and use
  # it as an integer counter:
  result = newStmtList()
  result.add newVarStmt(x[0], newLit(0))
  var body = x[^1]
  if body.kind != nnkStmtList:
    body = newTree(nnkStmtList, body)
  body.add newCall(bindSym"inc", x[0])
  var newFor = newTree(nnkForStmt)
  for i in 1..x.len-3:
    newFor.add x[i]
  # transform enumerate(X) to 'X'
  newFor.add x[^2][1]
  newFor.add body
  result.add newFor

for a2, b2 in enumerate([1, 2, 3, 5]):
  echo a2, " ", b2

Then anyone can metaprogram the case statement for their own types. There is a real need to resolve type though like macro case[Option](x: caseMatchStmt[Option]): untyped = ...

@krux02

This comment has been minimized.

Show comment
Hide comment
@krux02
Contributor

krux02 commented Aug 2, 2018

@dom96

So I think a lot of the discussion here has focused on pattern matching so far. This PR does a lot more than just that, and as it stands I would reject all of it except the .? macro.

Can we close this PR and create a new one with this macro? We can then discuss pattern matching in a separate issue/forum thread (I strongly suggest the latter).

@krux02

This comment has been minimized.

Show comment
Hide comment
@krux02

krux02 Aug 3, 2018

Contributor

@dom96 just close it.

Contributor

krux02 commented Aug 3, 2018

@dom96 just close it.

@PMunch

This comment has been minimized.

Show comment
Hide comment
@PMunch

PMunch Aug 4, 2018

Contributor

Oh wow, I've been away for two weeks on vacation and there are a lot of replies to go through. I'm still polishing my article on why this way of treating options is a good idea. But it's mostly about the concepts, the usage of or, and, or any other names doesn't really matter for that. But they do mean that it's more ergonomic to use them (in my opinion). When it comes to the matcher it really was just something I threw together as it wasn't all that much extra work. The most important part of it is the two distinct branches making it safer to use options. I'll read through more of these comments tomorrow and try to finish up my article on options, it will hopefully make a more coherent argument than all these comments.

Contributor

PMunch commented Aug 4, 2018

Oh wow, I've been away for two weeks on vacation and there are a lot of replies to go through. I'm still polishing my article on why this way of treating options is a good idea. But it's mostly about the concepts, the usage of or, and, or any other names doesn't really matter for that. But they do mean that it's more ergonomic to use them (in my opinion). When it comes to the matcher it really was just something I threw together as it wasn't all that much extra work. The most important part of it is the two distinct branches making it safer to use options. I'll read through more of these comments tomorrow and try to finish up my article on options, it will hopefully make a more coherent argument than all these comments.

@PMunch

This comment has been minimized.

Show comment
Hide comment
@PMunch

PMunch Aug 6, 2018

Contributor

I've now finished up my article on options, and why I think adding all of these are a good idea: https://peterme.net/optional-value-handling-in-nim.html

Contributor

PMunch commented Aug 6, 2018

I've now finished up my article on options, and why I think adding all of these are a good idea: https://peterme.net/optional-value-handling-in-nim.html

@andreaferretti

This comment has been minimized.

Show comment
Hide comment
@andreaferretti

andreaferretti Aug 7, 2018

Collaborator

I have read @PMunch article, and I am still unconvinced.

For one, the reason for introducing and for options relies on the similarity to how bash handles contructs like do_this && do_that. Now in bash it is cumbersome to do otherwise, os one relies on and being short-circuiting to execute side effects conditionally. Personally I consider this to be a hack and far less readable than explicit control flow (yes, with an if).

About match, I am still of the same opinion: it is a useful construct, but I would not want it implemented in the standard library in a way that only works for options.

Finally, the comparators like <? may be nice in some contexts but seem definitely a niche thing. By the same token, one may want to extend every predicate to options - I do not see why < or > should be given special treatment. I think the best place to implement this DSL would be a Nimble package of its own.

So in the end I still think that the best course of action is to extract the .? macro as a PR for sugar.nim and implement the rest of these operations inside a Nimble package.

Collaborator

andreaferretti commented Aug 7, 2018

I have read @PMunch article, and I am still unconvinced.

For one, the reason for introducing and for options relies on the similarity to how bash handles contructs like do_this && do_that. Now in bash it is cumbersome to do otherwise, os one relies on and being short-circuiting to execute side effects conditionally. Personally I consider this to be a hack and far less readable than explicit control flow (yes, with an if).

About match, I am still of the same opinion: it is a useful construct, but I would not want it implemented in the standard library in a way that only works for options.

Finally, the comparators like <? may be nice in some contexts but seem definitely a niche thing. By the same token, one may want to extend every predicate to options - I do not see why < or > should be given special treatment. I think the best place to implement this DSL would be a Nimble package of its own.

So in the end I still think that the best course of action is to extract the .? macro as a PR for sugar.nim and implement the rest of these operations inside a Nimble package.

@PMunch

This comment has been minimized.

Show comment
Hide comment
@PMunch

PMunch Aug 7, 2018

Contributor

The implementation of or and and is similar to how they work in bash, but that is not the reason for implementing them here. In bash everything returns a return code, and writes what in other languages would be the return value to a stream. This return code works like an option with && and || to allow conditional chaining of commands. But since it only works on return codes in bash it is far less powerful than the general concept. Probably the simplest task that shows some of that power is the default value for lookups:

let name = userTable.getOpt("name") or some("John Doe")

This could of course be done with inline if statements in Nim:

let name = if userTable.hasKey("name"): userTable.get("name") else: "John Doe"

But in my opinion this is a bit on the verbose side, and it shows the check-blindness concept I mentioned in the article. If I mess up and put the get statement on the wrong side I will get a guaranteed run-time error that the compiler won't be able to catch. Sure, it is my fault for writing stupid code, but there simply isn't a way to write this kind of stupid code when using options.

Now that does little more than what the proposed either template would do, except it returns an option, making it composable so we could do something like this:

let name = userTable.getOpt("name") or settings.getOpt("defaultName")
echo either(name, "No name found and no default name defined")

Or if we wanted some error checking and wanted to use the matcher:

let name = userTable.getOpt("name") or settings.getOpt("defaultName")
match name:
  some x:
    echo "Welcome ", x
  none:
    echo "No name found and no default name defined"

I share your concern with match but I think that can be addressed by a name change until we have a more robust solution. It is an integral part of working with options this way as it assures that you can only use the value of the option if you have actually made sure that it exists.

The scepticism around the comparators is also fair, they are more of a niche feature and as I mention in the article they are more there to flesh out the concept and make it a bit easier to work with options. I implemented optCmp which took an arbitrary comparator, but I noticed I wasn't able to pass ==, !=, <=, etc to it so I opted to make those special in order to avoid the user having to wrap them in a lambda if they wanted to use them.

Contributor

PMunch commented Aug 7, 2018

The implementation of or and and is similar to how they work in bash, but that is not the reason for implementing them here. In bash everything returns a return code, and writes what in other languages would be the return value to a stream. This return code works like an option with && and || to allow conditional chaining of commands. But since it only works on return codes in bash it is far less powerful than the general concept. Probably the simplest task that shows some of that power is the default value for lookups:

let name = userTable.getOpt("name") or some("John Doe")

This could of course be done with inline if statements in Nim:

let name = if userTable.hasKey("name"): userTable.get("name") else: "John Doe"

But in my opinion this is a bit on the verbose side, and it shows the check-blindness concept I mentioned in the article. If I mess up and put the get statement on the wrong side I will get a guaranteed run-time error that the compiler won't be able to catch. Sure, it is my fault for writing stupid code, but there simply isn't a way to write this kind of stupid code when using options.

Now that does little more than what the proposed either template would do, except it returns an option, making it composable so we could do something like this:

let name = userTable.getOpt("name") or settings.getOpt("defaultName")
echo either(name, "No name found and no default name defined")

Or if we wanted some error checking and wanted to use the matcher:

let name = userTable.getOpt("name") or settings.getOpt("defaultName")
match name:
  some x:
    echo "Welcome ", x
  none:
    echo "No name found and no default name defined"

I share your concern with match but I think that can be addressed by a name change until we have a more robust solution. It is an integral part of working with options this way as it assures that you can only use the value of the option if you have actually made sure that it exists.

The scepticism around the comparators is also fair, they are more of a niche feature and as I mention in the article they are more there to flesh out the concept and make it a bit easier to work with options. I implemented optCmp which took an arbitrary comparator, but I noticed I wasn't able to pass ==, !=, <=, etc to it so I opted to make those special in order to avoid the user having to wrap them in a lambda if they wanted to use them.

@andreaferretti

This comment has been minimized.

Show comment
Hide comment
@andreaferretti

andreaferretti Aug 7, 2018

Collaborator

Just a small note: you may have not noticed I have issues with and and not. You explain in detail the use of or, but I am completely fine with or (it is a common operation found in many other languages - for instance that would be orElse in Scala)

Collaborator

andreaferretti commented Aug 7, 2018

Just a small note: you may have not noticed I have issues with and and not. You explain in detail the use of or, but I am completely fine with or (it is a common operation found in many other languages - for instance that would be orElse in Scala)

@PMunch

This comment has been minimized.

Show comment
Hide comment
@PMunch

PMunch Aug 7, 2018

Contributor

Yeah I agree that not should be removed, it really has no use-case I can think of. and on the other hand is essentially the same as && in bash, whether it's useful really depends on the context of the has-ity. If you return an option with an error code (similar to what bash does) then and often makes more sense than or as you essentially want to continue if the option doesn't have a value. Plus having only one or the other would be really strange as they are two halves of the same concept.

Contributor

PMunch commented Aug 7, 2018

Yeah I agree that not should be removed, it really has no use-case I can think of. and on the other hand is essentially the same as && in bash, whether it's useful really depends on the context of the has-ity. If you return an option with an error code (similar to what bash does) then and often makes more sense than or as you essentially want to continue if the option doesn't have a value. Plus having only one or the other would be really strange as they are two halves of the same concept.

@jduey

This comment has been minimized.

Show comment
Hide comment
@jduey

jduey Aug 9, 2018

Hi folks. I'm the BDFL of Toccata. I'm gobsmacked that anyone would think an idea of mine is interesting enough to include in another language. And your discussion here has given me some food for thought. I'm not that knowledgeable about Nim, but I've been intrigued by it for some time. Happy to answer any questions anyone has about Toccata, though. :)

jduey commented Aug 9, 2018

Hi folks. I'm the BDFL of Toccata. I'm gobsmacked that anyone would think an idea of mine is interesting enough to include in another language. And your discussion here has given me some food for thought. I'm not that knowledgeable about Nim, but I've been intrigued by it for some time. Happy to answer any questions anyone has about Toccata, though. :)

@Araq

This comment has been minimized.

Show comment
Hide comment
@Araq

Araq Aug 9, 2018

Member

@jduey Can you please explain to me why bool loses information but int does not? No offense but it seems completely arbitrary to me. Most programming languages lack units of measure which seems to be the "real" problem you are concerned about.

Member

Araq commented Aug 9, 2018

@jduey Can you please explain to me why bool loses information but int does not? No offense but it seems completely arbitrary to me. Most programming languages lack units of measure which seems to be the "real" problem you are concerned about.

@jduey

This comment has been minimized.

Show comment
Hide comment
@jduey

jduey Aug 9, 2018

I can try. I don't see it as 'int' not losing any info as much as 'bool' loses all info except for a single bit. An 'int' still has a magnitude and 32 or 64 bits of info. If a programmer wants an int to carry more info, such as units, they can wrap it in a type.

There's also a question of how big a step to take at once. Eliminating bools was a small enough step I could feel confident in taking it. Eliminating unadorned ints is a much bigger step. And now that you mention it, I might have to put some thought into that.

jduey commented Aug 9, 2018

I can try. I don't see it as 'int' not losing any info as much as 'bool' loses all info except for a single bit. An 'int' still has a magnitude and 32 or 64 bits of info. If a programmer wants an int to carry more info, such as units, they can wrap it in a type.

There's also a question of how big a step to take at once. Eliminating bools was a small enough step I could feel confident in taking it. Eliminating unadorned ints is a much bigger step. And now that you mention it, I might have to put some thought into that.

@PMunch

This comment has been minimized.

Show comment
Hide comment
@PMunch

PMunch Aug 9, 2018

Contributor

Oh hi @jduey, nice to see you here :)

@Araq, I mentioned this in my article. IMHO boolean-blindness is, as you mention, a bit silly. All values lose information, it's not special for booleans. I also link to another article that points this out and generalizes the problem to something he calls part-blindness. This concept is however a bit too broad and is dealt with in Nim in other ways. But back-tracking to boolean blindness most people tend to talk about it when performing checks. Things like:

if x.isNil:
  echo x[]
else:
  echo "X is nil!"

Now this obviously won't work, as I've switched the if bodies around by "accident". This is what I refer to as check-blindness in my article, the fact that the context of the boolean check is lost. By using options and the special case .? and match the context becomes part of the block and the value can only be extracted when it is valid. This means that the above statement becomes something like this:

match some(x) !=? some(nil):
  some y: echo y[]
  none: echo "x is nil!"

This shows that the context of the check is not lost, it lives alongside the option, and the value we care about can only be used when it is valid.

PS: I'm planning on adding overloads to the postfixed comparators and the right hand side of the and and or to avoid many of the some(x) statements you now need. This would eliminate the two in the above statement.

Contributor

PMunch commented Aug 9, 2018

Oh hi @jduey, nice to see you here :)

@Araq, I mentioned this in my article. IMHO boolean-blindness is, as you mention, a bit silly. All values lose information, it's not special for booleans. I also link to another article that points this out and generalizes the problem to something he calls part-blindness. This concept is however a bit too broad and is dealt with in Nim in other ways. But back-tracking to boolean blindness most people tend to talk about it when performing checks. Things like:

if x.isNil:
  echo x[]
else:
  echo "X is nil!"

Now this obviously won't work, as I've switched the if bodies around by "accident". This is what I refer to as check-blindness in my article, the fact that the context of the boolean check is lost. By using options and the special case .? and match the context becomes part of the block and the value can only be extracted when it is valid. This means that the above statement becomes something like this:

match some(x) !=? some(nil):
  some y: echo y[]
  none: echo "x is nil!"

This shows that the context of the check is not lost, it lives alongside the option, and the value we care about can only be used when it is valid.

PS: I'm planning on adding overloads to the postfixed comparators and the right hand side of the and and or to avoid many of the some(x) statements you now need. This would eliminate the two in the above statement.

@PMunch

This comment has been minimized.

Show comment
Hide comment
@PMunch

PMunch Aug 14, 2018

Contributor

Okay, did some more rewrites. Now you can use all the logic operators and comparator operators with the right-hand value being a value and not an option. Just means you don't have to put some everywhere in your code. Also added some simple wrapper macros that will auto-wrap non-option procedures.

match is now gone, say hello to require! require doesn't turn everything into a case statement, and can't do the value matching thing, that can be done elsewhere. What require does is take in one statement, or a list of statements, and evaluates them one by one until one returns a none. If none of them return a none it will run the just branch and allow you to map the different values to symbols. Something like this:

require ["abc".find('o'), "def".find('f')]:                               
  some [firstPos, secondPos]:                                             
    echo "Found 'o' at position: ", firstPos, " and 'f' at position ",    
      secondPos                                                           
  none: echo "Couldn't find either 'o' or 'f'"

IMO this is more convenient than match and hopefully not as controversial.. Another option would of course be to move this to a new module optionutils, that way they are a bit more opt-in.

Contributor

PMunch commented Aug 14, 2018

Okay, did some more rewrites. Now you can use all the logic operators and comparator operators with the right-hand value being a value and not an option. Just means you don't have to put some everywhere in your code. Also added some simple wrapper macros that will auto-wrap non-option procedures.

match is now gone, say hello to require! require doesn't turn everything into a case statement, and can't do the value matching thing, that can be done elsewhere. What require does is take in one statement, or a list of statements, and evaluates them one by one until one returns a none. If none of them return a none it will run the just branch and allow you to map the different values to symbols. Something like this:

require ["abc".find('o'), "def".find('f')]:                               
  some [firstPos, secondPos]:                                             
    echo "Found 'o' at position: ", firstPos, " and 'f' at position ",    
      secondPos                                                           
  none: echo "Couldn't find either 'o' or 'f'"

IMO this is more convenient than match and hopefully not as controversial.. Another option would of course be to move this to a new module optionutils, that way they are a bit more opt-in.

@dom96

This comment has been minimized.

Show comment
Hide comment
@dom96

dom96 Aug 14, 2018

Member

Sorry, but I don't understand the rationale for this complete rewrite. Hundreds of lines of code are using get and you're deprecating it here for seemingly no compelling reason.

I'm happy to accept a pattern matching macro (and the .? operator), but the rest should stay as it is. At this point you might just be better off creating a new PR for these two things (and I would encourage creating separate PRs)

Member

dom96 commented Aug 14, 2018

Sorry, but I don't understand the rationale for this complete rewrite. Hundreds of lines of code are using get and you're deprecating it here for seemingly no compelling reason.

I'm happy to accept a pattern matching macro (and the .? operator), but the rest should stay as it is. At this point you might just be better off creating a new PR for these two things (and I would encourage creating separate PRs)

@PMunch

This comment has been minimized.

Show comment
Hide comment
@PMunch

PMunch Aug 14, 2018

Contributor

What? This isn't a complete rewrite.. I had to force push over my copy after I rebased against the current devel. The deprecation has been there from the start (or at least close to it).. Could of course remove it, but since no-one mentioned it I thought people were fine with it. The reason to deprecate it is to let people use the more powerful constructs that avoids silly runtime errors.

As I mentioned in my last comment I removed the pattern matching part of match and renamed it to require and now it only does the unpacking of options when they have a value.

What about the new wrapping macros? Yay or nay? And what don't you like with the comparators and logic operators?

Contributor

PMunch commented Aug 14, 2018

What? This isn't a complete rewrite.. I had to force push over my copy after I rebased against the current devel. The deprecation has been there from the start (or at least close to it).. Could of course remove it, but since no-one mentioned it I thought people were fine with it. The reason to deprecate it is to let people use the more powerful constructs that avoids silly runtime errors.

As I mentioned in my last comment I removed the pattern matching part of match and renamed it to require and now it only does the unpacking of options when they have a value.

What about the new wrapping macros? Yay or nay? And what don't you like with the comparators and logic operators?

@PMunch

This comment has been minimized.

Show comment
Hide comment
@PMunch

PMunch Aug 14, 2018

Contributor

Got tired of discussing this since it was basically just the same couple arguments over and over. So I removed everything but require, .?, and the new wrappers. Also removed the deprecation warnings and added back in the old ==. Hopefully this is less scary and will actually get added :)

Contributor

PMunch commented Aug 14, 2018

Got tired of discussing this since it was basically just the same couple arguments over and over. So I removed everything but require, .?, and the new wrappers. Also removed the deprecation warnings and added back in the old ==. Hopefully this is less scary and will actually get added :)

@PMunch

This comment has been minimized.

Show comment
Hide comment
@PMunch

PMunch Aug 17, 2018

Contributor

Split options into options and optionutils and moved all the map, flatMap, .?, etc. to the new module. Also changed the name from require to allSome per Araqs request.

Contributor

PMunch commented Aug 17, 2018

Split options into options and optionutils and moved all the map, flatMap, .?, etc. to the new module. Also changed the name from require to allSome per Araqs request.

@PMunch

This comment has been minimized.

Show comment
Hide comment
@PMunch

PMunch Aug 17, 2018

Contributor

Had to un-fuck the git history of this PR. I made some mistakes when trying to make it up to date with devel so it would merge properly.

Contributor

PMunch commented Aug 17, 2018

Had to un-fuck the git history of this PR. I made some mistakes when trying to make it up to date with devel so it would merge properly.

@PMunch

This comment has been minimized.

Show comment
Hide comment
@PMunch

PMunch Oct 3, 2018

Contributor

I've now split this into four separate PRs as requested: #9160, #9161, #9162, #9163. Please move discussion to the corresponding PRs.

Contributor

PMunch commented Oct 3, 2018

I've now split this into four separate PRs as requested: #9160, #9161, #9162, #9163. Please move discussion to the corresponding PRs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment