Introduce #use API in FCS (#load with fragment-by-fragment evaluation)

[matthid]

@dsyme Continuation from dotnet/fsharp#1392

Repro steps

test.fsx

failwith "test"

let scriptContents = System.IO.File.ReadAllText("test.fsx")
//session.EvalInteraction("#line 1 \"test.fsx\"")
session.EvalInteraction(scriptContents)

Expected behavior

System.Exception: runtime error
   at <StartupCode$FSI_0006>.$FSI_0006.main@()
Stopped due to error

Actual behavior

 failwith "test"
 ^^^^^^^^^^^^^^^

stdin(5,1): error FS0030: Value restriction. The value 'it' has been inferred to have generic type
    val it : '_a
Either define 'it' as a simple data term, make it a function with explicit arguments or, if you do not intend for it to be generic, add a type annotation.

Related information

This should be a workaround to properly load a script with interaction directive support, but its not working.

Should I add the string "; ();;" to the script contents? This turns out to be more like a hack than a solution.

[matthid]

Appending \n() seems to work. I'm not sure if this handles all cases.

The only way I could make it work was:

let scriptContents = 
  sprintf "#line 1 @\"%s\"\n" scriptFile + 
  System.IO.File.ReadAllText scriptFile + 
  "\n()"
x.EvalInteraction scriptContents

Not sure if this is the recommended way of doing things.

[dsyme]

@matthid If I start fsi.exe and enter the input

failwith "test";;

then we get the same behaviour as "Actual" above - so EvalInteraction is just doing what fsi.exe has always done. So it looks like the correct behaviour for EvalInteraction given that input.

Note that fsc.exe and #load will accept a file containing the above, i..e the value restriction is not applied. This is because of the following:

• For fsi.exe, the result of the last expression in the interaction (the failwith) is implicitly assigned to a value called "it". The value restriction is applied to this value.
• For fsc.exe, the result of expression is discarded, so the value restriction is not applied.

[matthid]

For fsi.exe, the result of the last expression in the interaction (the failwith) is implicitly assigned to a value called "it". The value restriction is applied to this value.

Question is: How can I do the same? EvalInteration was already the suggested way for implementing a "real" #load, but as shown here it's not equivalent.

[dsyme]

For fsi.exe, the result of the last expression in the interaction (the failwith) is implicitly assigned to a value called "it". The value restriction is applied to this value.

Question is: How can I do the same?

From the behaviour you report above in the "Actual" section it looks to me like EvalInteraction is doing this (i.e. implicitly assigning the last expression to a value called "it" and applying the value restriction to this value)

EvalInteration was already the suggested way for implementing a "real" #load, but as shown here it's not equivalent.

Perhaps I don't understand what you mean by a "real" #load. EvalScript is the equivalent of a #load. EvalInteraction is the equivalent of using the input text as an interaction (incuding #cd, binding-to-it etc.).

If you want the latter (sometimes called "#use" on a script, rather than "#load" - FSI doesn't support the #use directive but does support the --use:script.fsx command line argument) then it looks like you're doing the right thing: reading the text of the script and calling EvalInteraction. However the specific behavior above appears correct when processing text as interactions.

[matthid]

This is related to dotnet/fsharp#1392. I want to have a #load with behavior (3) from dotnet/fsharp#1392 (comment), but I want it lazy in an evaluation session (after I setup some state via EvalInteraction).

dotnet/fsharp#1392 (comment) suggest using EvalInteraction. This issue just tries to show that it's not enough.

[dsyme]

@matthid OK, so this is a feature request for FCS right? You're after a behavior that can't be done with either F# Interactive or the F# compiler today?

Specifically, you want an EvalInteraction that doesn't actually run the initialization code of the interaction but instead just checks/compiles it and then uses on-demand initialization?

If so, could you explain the scenario a bit more to help me understand what's driving the feature request. It sounds like it would be a "don't evaluate eagerly" flag on EvalScript and EvalInteraction.

[matthid]

Specifically, you want an EvalInteraction that doesn't actually run the initialization code of the interaction but instead just checks/compiles it and then uses on-demand initialization?

I'm not sure.

OK, so this is a feature request for FCS right?

Ideally I'd like to see this implemented in the compiler (see Use case 1 below).

You're after a behavior that can't be done with either F# Interactive or the F# compiler today?

No. It can already be done. It's just fsi script.fsx. Ideally I'd like to see the exact same behavior when using "#evaluate 'script.fsx'" (ie evaluating line by line instead of compiling the whole thing). I think people actually expect "#load" to do the same thing already (at least for scripts, but that might be only my personal feeling)

Maybe it makes more sense after the use cases...

Use case 1:

See

Consider having two scripts which use Paket or nuget to bootstrap (note: this is untested dummy code):
test1.fsx

// ...
webClient.Download("nuget.client.dll")
#r "nuget.client.dll"
Nuget.Client.Restore("deps.config")
// .. more restore action

#r "package1.dll"
#r "package2.dll"
#r "package3.dll"

// Regular script logic of test1.fsx
// Use other interactive directions like #cd and stuff

test2.fsx

// ...
webClient.Download("nuget.client.dll")
#r "nuget.client.dll"
Nuget.Client.Restore("deps.config")
// .. more restore action

#r "package1.dll"
#r "package2.dll"
#r "package3.dll"

// Regular script logic of test2.fsx
// Use other interactive directions like #cd and stuff

Now you want to "extract" the common bootstrapping logic to a shared script -> Doesn't work with #load.

Use case 2

Consider test1.fsx from above. Now consider that you want to provide a scripting environment via FCS with predefined variables (ie executing EvalInteraction before loading the script). But still allow people to use the same bootstrapping code (ie at the same time allow test1.fsx to be executed as is).

[dsyme]

@matthid Thanks

Ah OK, thanks

The primary FCS issue seems to be that EvalInteraction doesn't do fragment-by-fragment evaluation. Let's change this issue to cover that . That should be simple enough to fix.

The #load discussion is I think orthogonal. The #load mechanism is restricted to be what it is on purpose (i.e. it's not a #use, nor did we support a seaprate '#use). The primary rationale for this is to make sure we can give sane typechecking/errors/intellisense/autocomplete/.... , and to give proper resolution of diamond dependencies (same file #load'd from multiple paths only gets referenced once). It also allows type checking to be efficient, since we only check dependent files once (and recheck when they change).

Yes this means you can't factor out common fragments of scripts that create DLLs etc. I think the right thing is to propose on fslang.uservoice.com to add a #use directive, and provide a PR with its implementation. The Paket scenario is compelling and it would be reasonable to support it. The implementation would not be hard I think.

A key question is how the type checker in Visual Studio, Xamarin etc. would treat the #use fragments. My assumption is that it would effectively be textual inclusion, but wouldn't actually execute the fragments.

Cheers
don

[matthid]

Thanks for the detailed answer! Now we managed to get to the root problem :)

The #load mechanism is restricted to be what it is on purpose

I assumed something like that, but couldn't figure out why...
Honestly I expected one of the reasons to be to force people to write libraries instead of scripts, but that doesn't seem to be the case?

The #load discussion is I think orthogonal.

That discussion is now over with #use...

One note after reading your answer: If we can't handle it in the tooling I'm actually against introducing #use. However from an architectural standpoint it still think would make sense to implement it in "visualfsharp" and expose it via FSharp.Compiler.Service (that would cover Use case 2, which is my primary interest actually). This has a lower impact, we could test the functionality and later expose #use if it still makes sense. Does that sound reasonable?

[dsyme]

Honestly I expected one of the reasons to be to force people to write libraries instead of scripts, but that doesn't seem to be the case?

Yes, that's part of the thinking too. #load helps enable a transition from script to library, #use doesn't since diamond dependencies and textual inclusion allow for all sorts of coding patterns.

However from an architectural standpoint it still think would make sense to implement it in "visualfsharp" and expose it via FSharp.Compiler.Service (that would cover Use case 2, which is my primary interest actually).

We can just add it to FCS - It's reasonable for EvalInteraction to effectively be a multi-fragment #use. If we decidde to light up #use in fsi.exe proper then we can backport to visuafsharp.

[matthid]

@dsyme Thanks. I'm fine with that. Will try to hack something together :) (Can take some days/weeks though)