Keep a notion of the in keyword inside the Abstract Syntax Tree.

[nojaf]

Is your feature request related to a problem? Please describe.
When a user tries to format the following code with Fantomas, we try and preserve the in keyword. (At least we do in SynExpr.LetOrUse)

let a = 4 in
let b = 5 in
a + b

let x =
    let y = 1 in
    let z = 2 in
    y - z

Currently, this is part of "trivia process" and this information cannot be found in the Untyped Abstract Syntax tree.

ParsedInput.ImplFile
  (ParsedImplFileInput
     ("tmp.fsx", true, QualifiedNameOfFile Tmp$fsx, [], [],
      [SynModuleOrNamespace
         ([Tmp], false, AnonModule,
          [SynModuleDecl.Let
             (false,
              [Binding
                 (None, NormalBinding, false, false, [],
                  PreXmlDoc ((1,4), FSharp.Compiler.XmlDoc+XmlDocCollector),
                  SynValData
                    (None, SynValInfo ([], SynArgInfo ([], false, None)), None),
                  SynPat.Named
                    (SynPat.Wild tmp.fsx (1,4--1,5) IsSynthetic=false, a, false,
                     None, tmp.fsx (1,4--1,5) IsSynthetic=false), None,
                  SynExpr.Const
                    (SynConst.Int32 4, tmp.fsx (1,8--1,9) IsSynthetic=false),
                  tmp.fsx (1,4--1,5) IsSynthetic=false,
                  DebugPointAtBinding tmp.fsx (1,0--1,9) IsSynthetic=false)],
              tmp.fsx (1,0--1,9) IsSynthetic=false);
           SynModuleDecl.Let
             (false,
              [Binding
                 (None, NormalBinding, false, false, [],
                  PreXmlDoc ((2,4), FSharp.Compiler.XmlDoc+XmlDocCollector),
                  SynValData
                    (None, SynValInfo ([], SynArgInfo ([], false, None)), None),
                  SynPat.Named
                    (SynPat.Wild tmp.fsx (2,4--2,5) IsSynthetic=false, b, false,
                     None, tmp.fsx (2,4--2,5) IsSynthetic=false), None,
                  SynExpr.Const
                    (SynConst.Int32 5, tmp.fsx (2,8--2,9) IsSynthetic=false),
                  tmp.fsx (2,4--2,5) IsSynthetic=false,
                  DebugPointAtBinding tmp.fsx (2,0--2,9) IsSynthetic=false)],
              tmp.fsx (2,0--2,9) IsSynthetic=false);
           SynModuleDecl.DoExpr
             (DebugPointAtBinding tmp.fsx (3,0--3,5) IsSynthetic=false,
              SynExpr.App
                (NonAtomic, false,
                 SynExpr.App
                   (NonAtomic, true, SynExpr.Ident op_Addition, SynExpr.Ident a,
                    tmp.fsx (3,0--3,3) IsSynthetic=false), SynExpr.Ident b,
                 tmp.fsx (3,0--3,5) IsSynthetic=false),
              tmp.fsx (3,0--3,5) IsSynthetic=false);
           SynModuleDecl.Let
             (false,
              [Binding
                 (None, NormalBinding, false, false, [],
                  PreXmlDoc ((5,4), FSharp.Compiler.XmlDoc+XmlDocCollector),
                  SynValData
                    (None, SynValInfo ([], SynArgInfo ([], false, None)), None),
                  SynPat.Named
                    (SynPat.Wild tmp.fsx (5,4--5,5) IsSynthetic=false, x, false,
                     None, tmp.fsx (5,4--5,5) IsSynthetic=false), None,
                  SynExpr.LetOrUse
                    (false, false,
                     [Binding
                        (None, NormalBinding, false, false, [],
                         PreXmlDoc
                           ((6,8), FSharp.Compiler.XmlDoc+XmlDocCollector),
                         SynValData
                           (None, SynValInfo ([], SynArgInfo ([], false, None)),
                            None),
                         SynPat.Named
                           (SynPat.Wild tmp.fsx (6,8--6,9) IsSynthetic=false, y,
                            false, None, tmp.fsx (6,8--6,9) IsSynthetic=false),
                         None,
                         SynExpr.Const
                           (SynConst.Int32 1,
                            tmp.fsx (6,12--6,13) IsSynthetic=false),
                         tmp.fsx (6,8--6,9) IsSynthetic=false,
                         DebugPointAtBinding
                           tmp.fsx (6,4--6,13) IsSynthetic=false)],
                     SynExpr.LetOrUse
                       (false, false,
                        [Binding
                           (None, NormalBinding, false, false, [],
                            PreXmlDoc
                              ((7,8), FSharp.Compiler.XmlDoc+XmlDocCollector),
                            SynValData
                              (None,
                               SynValInfo ([], SynArgInfo ([], false, None)),
                               None),
                            SynPat.Named
                              (SynPat.Wild tmp.fsx (7,8--7,9) IsSynthetic=false,
                               z, false, None,
                               tmp.fsx (7,8--7,9) IsSynthetic=false), None,
                            SynExpr.Const
                              (SynConst.Int32 2,
                               tmp.fsx (7,12--7,13) IsSynthetic=false),
                            tmp.fsx (7,8--7,9) IsSynthetic=false,
                            DebugPointAtBinding
                              tmp.fsx (7,4--7,13) IsSynthetic=false)],
                        SynExpr.App
                          (NonAtomic, false,
                           SynExpr.App
                             (NonAtomic, true, SynExpr.Ident op_Subtraction,
                              SynExpr.Ident y,
                              tmp.fsx (8,4--8,7) IsSynthetic=false),
                           SynExpr.Ident z, tmp.fsx (8,4--8,9) IsSynthetic=false),
                        tmp.fsx (7,4--8,9) IsSynthetic=false),
                     tmp.fsx (6,4--8,9) IsSynthetic=false),
                  tmp.fsx (5,4--5,5) IsSynthetic=false, NoDebugPointAtLetBinding)],
              tmp.fsx (5,0--8,9) IsSynthetic=false)], PreXmlDocEmpty, [], None,
          tmp.fsx (1,0--9,0) IsSynthetic=false)], (true, true)))

Describe the solution you'd like
Could we extend SynBinding or another node to preserve this information?

Describe alternatives you've considered
Our current workaround is detecting the IN token via the Tokenizer.

Additional context
I'd like to give this a shot myself if approved.

[auduchinok]

Would checking that IN is between bindings and body ranges in a LetOrUse expression help? If you've managed to tokenize IN there and there's a valid body expression afterwards in the SyntaxTree, then it should be token you're after.

Off-topic: (questionable) what would be really great for Fantomas is to be able to find cases where in keyword is not actually needed by the light syntax and not preserve it.

[nojaf]

@auduchinok that is how we do it today.

[auduchinok]

Cool. :)
Since there can be only one in in such an expression, and everything else would produce a syntax error, it should work fine. I couldn't come up with a counter-example that would break this approach. Is there a case where it doesn't work for you? Or is it to try to improve perf? I wonder because not sure what other keywords you could be interested in and in which contexts.

[nojaf]

Yes, this is purely to improve perf as we jump through some hoops to restore the in keyword.
I would assume that the compiler at some point has this information when parsing the code.
(Related Fantomas issue: fsprojects/fantomas#1114).

Other keywords that might be interesting is abstract member versus abstract, see fsprojects/fantomas#1106.

[auduchinok]

There may be multiple bindings in a single LetOrUse expression, so in range should belong to it instead of a particular inner binding, consider this example:

do
    let rec f = ()
    and g = () in
    ()

mkLocalBindings creates LetOrUse expressions and a range option could be passed to it in places like

fsharp/src/fsharp/pars.fsy

Lines 3281 to 3282 in 5f8a77b

	| defnBindings IN typedSeqExpr %prec expr_let
	{ mkLocalBindings (unionRanges (rhs2 parseState 1 2) $3.Range, $1, $3) }

Speaking about the example above: is it harder for you to detect in than, say, rec and and?

[nojaf]

Well that sample doesn't work yet 😅 (See fsprojects/fantomas#1176).
I guess detecting is more or less the same, my main question/proposal here is that I don't need to detect it at all, that is just some flag in an AST node.

[nojaf]

@cartermp any thoughts on this?

[cartermp]

I think this is a reasonable request. Don't think it would be high priority, but I think we'd certainly take a look at a PR.