Fable support #688

ncave · 2017-01-21T20:39:54Z

Mostly #ifdefs added, but not too many (IMO)
Uses a custom build step for code generation: fcs\build CodeGen.Fable
Parsing multiple files works
Untyped AST parsing works
Typed AST parsing also works
Compiling Fable with Fable works
Fable2 Demo REPL page works (Fable1 REPL here)
Next Step: To be determined by community, open for suggestions.

dsyme · 2017-01-22T18:58:26Z

@ncave Extraordinary. I'll take a close look

Is the demo page hosted somewhere? How big is the overall JS for the bootstrapped compiler?

Random future thought: Do you think there's any chance we could eventually produce .NET binaries with the JS compiler (and send them to .NET clients?) I've no idea when/where that would be useful, it's just a thought.

Also, I wonder if this might eventually make it into the Microsoft Visual F# repo and the standard F# command line compiler. That would be quite interesting :) Or is it always best to package it via Fable?

dsyme · 2017-01-22T19:00:48Z

Export.FCS.sln

@@ -0,0 +1,27 @@
+Microsoft Visual Studio Solution File, Format Version 12.00


It's probably best if all files that have something to do with Fable have Fable in the name where possible?

Sure. It's not really related to the PR, it's just a quick hack to export metadata using FCS, so it should be moved out.

dsyme · 2017-01-22T19:02:14Z

build.fsx

@@ -272,14 +272,27 @@ Target "CodeGen.NetCore" (fun _ ->
    let fsLex fsl out = runInDir (toolDir + "fslex.exe") "%s --unicode %s -o %s" fsl lexArgs out
    let fsYacc fsy out m o = runInDir (toolDir + "fsyacc.exe") "%s %s %s %s %s -o %s" fsy lexArgs yaccArgs m o out

+#if FABLE_COMPILER


Just to mention that in the visualfsahrp (and soon fsharp, and soon this) repo FsSrGen has been replaced by a script fssrgen.fsx

I know, it's there just until the new tooling gets released and FCS is migrated to use it.

dsyme · 2017-01-22T19:03:53Z

build.fsx

+    // comments the #line directive as it is not supported by Fable
+    ["lex.fs"; "pplex.fs"; "illex.fs"; "ilpars.fs"; "pars.fs"; "pppars.fs"]
+    |> Seq.map (fun fileName -> IO.Path.Combine (workDir, fileName))
+    |> RegexReplaceInFilesWithEncoding @"# (?=\d)" "//# " Text.Encoding.UTF8


Any specific reason this doesn't work in fable?

Fable uses the line directive already to do source mapping, so we can't. The simplest way is to just comment it out.

dsyme · 2017-01-22T19:03:54Z

build.fsx

+    // comments the #line directive as it is not supported by Fable
+    ["lex.fs"; "pplex.fs"; "illex.fs"; "ilpars.fs"; "pars.fs"; "pppars.fs"]
+    |> Seq.map (fun fileName -> IO.Path.Combine (workDir, fileName))
+    |> RegexReplaceInFilesWithEncoding @"# (?=\d)" "//# " Text.Encoding.UTF8


Any specific reason this doesn't work in fable?

Yes, see this.

dsyme · 2017-01-22T19:04:52Z

src/absil/il.fs

@@ -8,14 +8,18 @@ module (*internal*) Microsoft.FSharp.Compiler.AbstractIL.IL


 open Internal.Utilities
+#if FABLE_COMPILER


You may as well make this unconditional e.g.

open Microsoft.FSharp.Collections // needed for Fable open Microsoft.FSharp.Core // needed for Fable

It may be a good idea to retain the conditional as there is some type overwriting that is only needed when compiled with Fable.

Please use FSLANG_NO_IMPLICITOPEN to indicate a dialect of F# that doesn't have implicit opening of these namespaces, thanks

It's not that there is no implicit opening, it's the order of opening after the adapters are loaded which overwrites some core types with custom implementations.

dsyme · 2017-01-22T19:05:06Z

src/absil/il.fs

 open Microsoft.FSharp.Compiler.AbstractIL
 open Microsoft.FSharp.Compiler.AbstractIL.Diagnostics
 open Microsoft.FSharp.Compiler.AbstractIL.Internal
 open Microsoft.FSharp.Compiler.AbstractIL.Internal.Library
 open System.Collections
 open System.Collections.Generic
 open System.Collections.Concurrent
-


Please remove whitespace cahnges

dsyme · 2017-01-22T19:05:48Z

src/absil/il.fs

@@ -77,21 +81,21 @@ let rec splitNamespaceAux (nm:string) =

 /// Global State. All namespace splits ever seen
 // ++GLOBAL MUTABLE STATE
+#if FABLE_COMPILER


Why not conditionally define

type ConcurrentDictionary<'K,'V> = Dictionary<'K'V>

?

No reason, just looked cleaner this way. This too can go away after Dictionary constructors can be handled properly by Fable.

dsyme · 2017-01-22T19:07:11Z

src/absil/il.fs

-         (x.Scope = y.Scope) && 
-         (x.Name = y.Name) && 
-         (x.Enclosing = y.Enclosing)
+    override x.Equals(yobj:obj) = 


Why is this change needed? When would the cast fail?

I don't recall the actual reason, probably unsupported casting by Fable.

dsyme · 2017-01-22T19:07:49Z

src/absil/il.fs

-    member x.IsNewSlot              = match x.mdKind with | MethodKind.Virtual v -> v.IsNewSlot  | _ -> invalidOp "not virtual"
-    member x.IsCheckAccessOnOverride= match x.mdKind with | MethodKind.Virtual v -> v.IsCheckAccessOnOverride   | _ -> invalidOp "not virtual"
-    member x.IsAbstract             = match x.mdKind with | MethodKind.Virtual v -> v.IsAbstract | _ -> invalidOp "not virtual"
+    member x.IsFinal                = match x.mdKind with | MethodKind.Virtual v -> v.IsFinal    | _ -> false //invalidOp "not virtual"


Why are these changes needed? I see they are pretty harmless changes though

I'll revisit that.

dsyme · 2017-01-22T19:08:35Z

src/absil/il.fs

@@ -3661,7 +3666,11 @@ type ILTypeSigParser(tstring : string) =
                // fetch the arity
                let arity = 
                    while (int(here()) >= (int('0'))) && (int(here()) <= ((int('9')))) && (int(peek()) >= (int('0'))) && (int(peek()) <= ((int('9')))) do step()
+#if FABLE_COMPILER


Can we just always use the Fable version here?

Sure, I just tried to be conservative with changes to lower the impact.

dsyme · 2017-01-22T19:08:59Z

src/absil/il.fs

@@ -1552,7 +1557,7 @@ and [<Sealed>] ILTypeDefs(f : unit -> (string list * string * ILAttributes * Laz
    let mutable array = InlineDelayInit<_>(f)
    let mutable dict = InlineDelayInit<_>(fun () -> 
            let arr = array.Value
-            let t = Dictionary<_,_>(HashIdentity.Structural)
+            let t = Dictionary<_,_>(3, HashIdentity.Structural)


What's up here - doesn't Fable have a default size constructor?

Main reason is Fable (and Javascript maps/sets) have no support for collections with equality comparers. Technically map/set trees with IComparer can be used as a substitute, but probably the performance will not be too good, more like a list instead of a map (not sure how important collection performance is here though). Very few places in FCS actually use a custom equality comparer, so for now we're getting away with just ignoring them, that's one of the corners that have been cut and has to be fixed. Technically this should eventually go away as it's just a deficiency in Fable.

dsyme · 2017-01-22T19:09:58Z

Made a few initial comments. I suppose the big challenge is going to be keeping this compiling and working as changes flow in from the Visual F# TOols

smoothdeveloper · 2017-01-21T21:11:15Z

src/absil/il.fs

@@ -1552,7 +1557,7 @@ and [<Sealed>] ILTypeDefs(f : unit -> (string list * string * ILAttributes * Laz
    let mutable array = InlineDelayInit<_>(f)
    let mutable dict = InlineDelayInit<_>(fun () -> 
            let arr = array.Value
-            let t = Dictionary<_,_>(HashIdentity.Structural)
+            let t = Dictionary<_,_>(3, HashIdentity.Structural)


should 3 be arr.Length?

Maybe, but that's not the default value for this contructor. In any case, this will probably go away, as it's just a work around a deficiency in Fable (can't do comparers), when it's resolved I'll take it out.

smoothdeveloper · 2017-01-21T21:12:00Z

src/absil/il.fs


    member aref.QualifiedName = 
        let b = new System.Text.StringBuilder(100)
        let add (s:string) = (b.Append(s) |> ignore)
-        let addC (s:char) = (b.Append(s) |> ignore)
+        let addC (s:char) = (b.Append(string s) |> ignore)


not sure it impacts much the perf but maybe make a conditional for this as well?

Don argued against making it conditional here.

ncave · 2017-01-23T05:54:41Z

@dsyme @smoothdeveloper Thank you for your comments!
Some of the unexplicable ifdefs are just to get around deficiencies in the compiler, while some others are actually a problem in FCS (see #676 for example, which is forcing the addition of dummy constructors all over the place).

@dsyme

Demo page is not hosted anywhere yet but you can host it locally with Node after it's compiled with Fable. The bundled JS size is around 8 MB (unminified) so it should get smaller when minified. Compilation time for scripts is not as bad as I thought it would be, just a few seconds, but since the whole thing is completely unoptimized, it should get better when the hot path is optimized.
Sure, I see no technical reason to not be able to produce .NET binaries with the JS compiler. Not sure if that's applicable in a browser setting, but perhaps in Node setting. A bit more of FCS will need to be compiled, but shouldn't be a problem if there is interest.
For a full-featured replacement as a tool it will need the rest of FCS to be compiled as well (file system, build system, project support, etc.), but yes, no reason why it can't be migrated to the upstream repos.

dsyme · 2017-01-31T01:05:14Z

@ncave Looks like this is green :) Do you have more work you'd like to do on this, or should we work towards a final review?

ncave · 2017-01-31T06:09:42Z

@dsyme The PR can probably use some work to clean it up a bit, but if I recall correctly, you mentioned that there are some large upstream changes incoming, so it may be a good idea to wait until those are merged in and adapt the PR to them before moving forward.

dsyme · 2017-02-01T15:25:42Z

.vscode/launch.json

@@ -0,0 +1,21 @@
+{


Should this be included in the PR?

I'll take it out. It was just handy to have a debugging environment out of the box, but it does not belong in the PR.

dsyme · 2017-02-01T15:27:09Z

src/absil/illib.fs

 let (===) x y = LanguagePrimitives.PhysicalEquality x y

+#if !FABLE_COMPILER


It may be better to use a conditional based on feature like #if FX_NO_GETCURRENTPROCESS or the like

Looks like FX_NO_GETCURRENTPROCESS is not used anywhere else. I've tried to keep the conditionals to a minimum and not introduce new ones unnecessarily. IMO, there are quite a few BCL APIs not supported by Fable, it would become crowded very fast if we introduce a different conditional for each one that is not supported. But if you feel strongly about it, I'll change it.

Ideally I'd like the F# compiler codebase to not mention Fable at all , but rather be ``#if`'d on a set of characteristics that Fable happens to have. I'm expecting that there will other F#-to-XYZ attempts in the future, e.g. to the JVM, or another F#-to-Javascript compiler (e.g. WebSharper), or F#-to-native.

The more we can use FX_NO_XYZ... or FSLANG_NO_BYREFS the more the compiler code will be "neutral" with respect to these different efforts.

To give context: we've had this before, e.g. for Silverlight we scattered hundreds of #if SILVERLIGHT across the codebase. They hung around for years and years even after we stopped cross-compiling - and we had cause to keep some of them because some carried across to .NET Core, some were feature-specific etc. The lesson I learned from that is to always ask people to be specific about why #if creep in: if there are just 5-10 uses of an `#ifit's ok, but there are about 180 uses ofif FABLE_COMPILER`` - so we should take the time to divide them into the true underlying reasons now

That's fine, if that's your preference I'll try to replace all #if with feature conditionals.

dsyme · 2017-02-01T15:28:51Z

src/absil/illib.fs

 /// An efficient lazy for inline storage in a class type. Results in fewer thunks.
+#if FABLE_COMPILER
+type InlineDelayInit<'T when 'T : not struct>(f: unit -> 'T) = 


When you use #if FABLE_COMPILER please add a comment indicating what Fable doesn't support, or use a feature define like #if FSLANG_NO_MUTABLE_STRUCTS. The latter helps document that the F# language as supported by Fable has some specific semantic differences to F# as supported by .NET.

dsyme · 2017-02-01T15:29:18Z

src/absil/illib.fs


    let order = LanguagePrimitives.FastGenericComparer<string>
-


Please remove whitespace changes where possible

dsyme · 2017-02-01T15:30:13Z

src/absil/illib.fs

@@ -808,10 +831,17 @@ type LayeredMap<'Key,'Value  when 'Key : comparison> = Map<'Key,'Value>
 type Map<'Key,'Value when 'Key : comparison> with
    static member Empty : Map<'Key,'Value> = Map.empty

+#if FABLE_COMPILER


Add explanatory comment for the #if please

dsyme · 2018-09-26T18:16:42Z

Thanks all, see https://github.com/fable-compiler/repl2/issues/48 :)

alfonsogarciacaro · 2019-01-28T10:31:16Z

@ncave It seems the fsharp/FSharp.Compiler.Service repo is lagging behind Microsoft/visualfsharp. Would it be possible to change your fork to that repo? Does it make any sense at all?

ncave · 2019-01-28T15:31:43Z

@alfonsogarciacaro This has been discussed before. Of course we can, and would be nice to not have to wait until latest (and greatest) gets merged downstream, but there is very little chance (if any) for this PR to ever be merged over there. Not that it has much better chances here either (for understandable reasons), so yeah, perhaps we can just open a new one in a Microsoft/visualfsharp fork without submitting a PR. I'll consider it.

ncave · 2019-02-02T22:35:35Z

Rebased to latest (26.0.1).

ncave · 2019-02-23T01:52:59Z

Rebased to latest.

ncave · 2019-02-26T02:59:23Z

Rebased to latest (27.0.1).

ncave · 2019-03-29T21:47:48Z

Rebased to latest (28.0.0).

ncave · 2019-05-20T02:39:14Z

Closing this PR in favor of this one upstream.
It was getting long in the tooth anyway ... 😀

ncave mentioned this pull request Jan 21, 2017

[WIP] Fable JS support fable-compiler/Fable#657

Merged

ncave force-pushed the fable branch from 200e802 to 4f75bd8 Compare January 22, 2017 15:10

dsyme reviewed Jan 22, 2017

View reviewed changes

smoothdeveloper reviewed Jan 22, 2017

View reviewed changes

dsyme closed this Jan 30, 2017

dsyme reopened this Jan 30, 2017

ncave force-pushed the fable branch 4 times, most recently from 63c12b9 to e646aa8 Compare February 1, 2017 07:14

dsyme reviewed Feb 1, 2017

View reviewed changes

ncave force-pushed the fable branch 2 times, most recently from e214fbd to dcecef2 Compare October 20, 2018 23:26

alfonsogarciacaro mentioned this pull request Oct 30, 2018

Question on Fable.Repl fable-compiler/Fable#1621

Closed

ncave force-pushed the fable branch from 607c3e6 to cbd2578 Compare November 2, 2018 03:42

ncave force-pushed the fable branch from 2af6133 to a1a9cfa Compare January 28, 2019 04:44

ncave force-pushed the fable branch 4 times, most recently from ddbeea5 to bef8db3 Compare February 2, 2019 22:20

ncave force-pushed the fable branch from f613d5c to 9406356 Compare February 5, 2019 11:59

ncave force-pushed the fable branch from fff19ed to 73df5f0 Compare February 23, 2019 01:49

ncave force-pushed the fable branch from 73df5f0 to 099be25 Compare February 26, 2019 02:58

ncave force-pushed the fable branch from 7e5dbbe to 1a0c731 Compare March 13, 2019 10:29

ncave force-pushed the fable branch 2 times, most recently from c3cd85d to fd3a482 Compare March 23, 2019 17:43

ncave added 2 commits March 29, 2019 10:37

Change LexBuffer type from char to uint16

0633603

Fable support

57a54e0

ncave force-pushed the fable branch from 426860f to 57a54e0 Compare March 29, 2019 21:45

ncave added 2 commits April 7, 2019 20:40

Fixed optimized bindings

8e5555c

metadata update

ba93df4

ncave closed this May 20, 2019

granicz mentioned this pull request Oct 4, 2019

Propose transpilation guidelines fsharp/fsharp.github.io#96

Merged

		@@ -0,0 +1,27 @@
		Microsoft Visual Studio Solution File, Format Version 12.00

		@@ -8,14 +8,18 @@ module (internal) Microsoft.FSharp.Compiler.AbstractIL.IL


		open Internal.Utilities
		#if FABLE_COMPILER

		let (===) x y = LanguagePrimitives.PhysicalEquality x y

		#if !FABLE_COMPILER

Fable support #688

Fable support #688

Conversation

ncave commented Jan 21, 2017 • edited Loading

dsyme commented Jan 22, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ncave Jan 23, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ncave Feb 4, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ncave Jan 23, 2017 • edited Loading

Choose a reason for hiding this comment

dsyme commented Jan 22, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ncave Jan 23, 2017 • edited Loading

Choose a reason for hiding this comment

ncave commented Jan 23, 2017 • edited Loading

dsyme commented Jan 31, 2017

ncave commented Jan 31, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dsyme Feb 4, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dsyme commented Sep 26, 2018

alfonsogarciacaro commented Jan 28, 2019

ncave commented Jan 28, 2019 • edited Loading

ncave commented Feb 2, 2019

ncave commented Feb 23, 2019

ncave commented Feb 26, 2019

ncave commented Mar 29, 2019

ncave commented May 20, 2019

ncave commented Jan 21, 2017 •

edited

Loading

ncave Jan 23, 2017 •

edited

Loading

ncave Feb 4, 2017 •

edited

Loading

ncave Jan 23, 2017 •

edited

Loading

ncave Jan 23, 2017 •

edited

Loading

ncave commented Jan 23, 2017 •

edited

Loading

dsyme Feb 4, 2017 •

edited

Loading

ncave commented Jan 28, 2019 •

edited

Loading