Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add nullable reference types [RFC FS-1060] #577

Open
4 of 6 tasks
0x53A opened this issue Jun 2, 2017 · 69 comments
Open
4 of 6 tasks

Add nullable reference types [RFC FS-1060] #577

0x53A opened this issue Jun 2, 2017 · 69 comments

Comments

@0x53A
Copy link
Contributor

0x53A commented Jun 2, 2017

RFC: https://github.com/fsharp/fslang-design/blob/main/RFCs/FS-1060-nullable-reference-types.md

[Updated description by @dsyme]

C# 8.0 is likely to ship with a feature where new C# projects opt-in by default to making reference types "without-null" by default), with explicit annotations for reference types "with-null". The corresponding metadata annotations produced by this feature are likely to start to be used by .NET Framework libraries.

F#-defined types are "without-null" by default, however .NET-defined types like string and array and any other types defined in .NET libraries are "with-null" by default. The suggestion is to make new F# 5.0 projects opt-in to having .NET reference types be "without-null" by default when mentioned in F# code, and to interoperate with .NET metadata produced by C# 8.0 indicating when types are "with-null".

Terminology and working assumptions

These terms are used interchangeably:

  • "null as a abnormal value", "string without null", "non-nullable string"

Likewise

  • "null as a normal value", "string with null", "nullable string"

For the purposes of discussion we will use string | null as the syntax for types explicitly annotated to be "with null". You will also see string? in some samples

We will assume this feature is for F# 5.0 and is activated by a "/langlevel:5.0" switch that is on by default for new projects.

Proposed Design Principles

We are at the stage of trying to clarify the set of design principles to guide this feature:

  1. We should aim that F# should remain "almost" as simple to use as it is today. Indeed, the aim should be that the experience of using F# as a whole is simpler, because the possibility of nulls flowing in from .NET code for types like string is reduced and better tracked.

  2. The value for F# here is primarily in flowing non-nullable annotations into F# code from .NET libraries, and vice-versa, and in allowing the F# programmer to be explicit about the non-nullability of .NET types.

  3. Adding with-null/without-null annotations should not be part of routine F# programming

  4. There is a known risk of "extra information overload" in some tooling, e.g. tooltips. Nullability annotations/information may need to be suppressed and simplified in some types shown in output in routine F# programming. There is discussion about how this would be tuned in practice

  5. F# users should primarily only experience/see this feature when interoperating with .NET libraries (the latter should be rarely needed)

  6. The feature should produce warnings only, not hard errors

  7. The feature is backwards compatible, but only in the sense all existing F# projects compile without warning by default. Placing F# 4.x code into F# 5.0 projects may give warnings.

  8. F# non-nullness is reasonably "strong and trustworthy" today for F#-defined types and routine F# coding. This includes the explicit use of option types to represent the absence of information. The feature should not lead to a weakening of this trust nor a change in F# methodology that leads to lower levels of safety.

Notes from original posting

Note that there are a few related proposals (see below), but I couldn't find an exact match in this repo.

One reason I am adding this proposal is that I have lately been working in a mixed C# / F# solution, where I DO need to work with null more often than I like.

The other reason is that we should be aware of the parallel proposal for C# (dotnet/csharplang#36).

My main question is: Is there a minimal implementation for F#, that eases working with C# types NOW, without blocking adoption of future C# evolutions?

The existing way of approaching this problem in F# is ...

Types declared in F# are non-nullable by default. You can either make them nullable with AllowNullLiteralAttribute, or wrap them in an Option<'T>.

Types declared either in C#, or in a third-party F# source with AllowNullLiteralAttribute are always nullable, so in theory you would need to deal with null for every instance. In practice, this is often ignored and may or (often even worse) may not fail with a null-reference exception.

I propose we ...

I propose we add a type modifier to declare that this instance shall never be null.
This will be most useful in a function parameter.

Because I do not want to discuss the actual form of that modifier (attribute, keyword, etc), I will use 'T foo as meaning non-nullable 'T, similar to 'T option.

Example:

let stringLength (s:string foo) = s.Length

Calling this method with a vanilla string instance would produce a hard error.

How can a nullable type be converted to a non-nullable?

There should be at least a limited form of flow analysis.

Two examples would be if-branches and pattern matching:

let s = ... // vanilla (nullable) string

(* 1 - pattern matching *)

match s with
| null -> 0
| s -> stringLength s

// Note that this does not change the original variable s - the new s shadows the old s. I can also use a different variable name:

match s with
| null -> 0
| s2 ->
    // this fails:
    // stringLength s
    // this works:
    stringLength s2



(* 2 - explicit null-checking *)

if s = null then
    ()
else
    // now the compiler knows s is non-nullable
    stringLength s

There probably also needs to be a shorthand to force a cast, similar to Option.Value, which will throw at runtime if the option is null.

Runtime behavior

The types are erased to attributes.
You can't overload between nullable and non-nullable (may in combination with inline?)
When a non-null type modification is used in a function parameter, the compiler should insert a null-check at the beginning of the method.
The compiler should also add a null-check for all hard casts.

Pros and Cons

The advantages of making this adjustment to F# are ...

stronger typing.

The disadvantages of making this adjustment to F# are ...

yet another erased type modifier, like UOM, aliases.

Extra information

Estimated cost (XS, S, M, L, XL, XXL): L

Related suggestions: (put links to related suggestions here)

#552: Flow based null check analysis for [<AllowNullLiteralAttribute>] types and alike

Affidavit (must be submitted)

Please tick this by placing a cross in the box:

  • This is not a question (e.g. like one you might ask on stackoverflow) and I have searched stackoverflow for discussions of this issue
  • I have searched both open and closed suggestions on this site and believe this is not a duplicate
  • This is not something which has obviously "already been decided" in previous versions of F#. If you're questioning a fundamental design decision that has obviously already been taken (e.g. "Make F# untyped") then please don't submit it.

Please tick all that apply:

  • This is not a breaking change to the F# language design
  • I would be willing to help implement and/or test this
  • I or my company would be willing to help crowdfund F# Software Foundation members to work on this
@dsyme dsyme changed the title Add non-nullable instantiations of nullable types Add non-nullable instantiations of nullable types, and interop with proposed C# 8.0 non-nullable reference types Nov 16, 2017
@dsyme
Copy link
Collaborator

dsyme commented Nov 16, 2017

Adjusted title to reflect that this will need to cover interop with proposed C# 8 non-nullable reference types https://blogs.msdn.microsoft.com/dotnet/2017/11/15/nullable-reference-types-in-csharp/

@cartermp
Copy link
Member

cartermp commented Jun 21, 2018

@dsyme given that this is going to be a headlining feature for C# 8 (and all of .NET that compiles with it), I suggest we add proposed-priority as a label.

@cartermp
Copy link
Member

cartermp commented Jun 22, 2018

Additional notes after speaking with @TIHan about how we might want to do this:

Syntax

Use ? in the type annotation, just like C#:

let doot (str: string?) = // Nullable
    ...

let hoot (str: string) = // Non-nullable
    ...

When the compiler detects the attibute emitted by nullable types, it will infer the reference type to be nullable. The syntax should be reflected in tooling as well (QuickInfo, SignatureHelp, etc.).

Behavior

From a "referentially transparent" point of view, the behavior of this should be the same as C#. Under the covers, C# will be doing vastly different things because they did not start with declared reference types being non-null by default like F#. We'd work with the C# team to make sure every behavioral edge case is the same, but the underlying implementation for F# is likely to be far simpler.

AllowNullLiteral

This is a strictly better solution than decorating types with [<AllowNullLiteral>]. It's very rare that someone wishes to assign null to a reference type all over the place. Rather, we think that it's ad-hoc nullability that people want. This feature accomplishes that.

Given this, a goal of this feature should be to allow F# developers to phase out their usage of [<AllowNullLiteral>] and eventually allow us to deprecate the feature in a future future version of F# (i.e., much further in the future).

Additionally, we might want to consider types that are nullable equivalent to types that are decorated with [<AllowNullLiteral>]. Nullable types that are also decorated with [<AllowNullLiteral>] would be redundant, so we could consider having a warning for that. It's hard to say how much additional work could be done here (e.g., code fix to remove [<AllowNullLiteral>] under some circumstances).

@dsyme
Copy link
Collaborator

dsyme commented Jun 22, 2018

Use ? in the type annotation, just like C#:

let doot (str: string?) = // Nullable

let hoot (str: string) = // Non-nullable

@cartermp As a starting desire that is reasonable. However there are significant problems.

One is that in F# code today string is definitely nullable, because it is from a .NET DLL. This leaves an awkward compat problem. Additionally there is a risk that ? annotations will become pervasive and awkward in F# code, because they "flow in" whenever interoperating with .NET. This will create conceptual baggage for the F# programmer.

For example, if .NET defines an interface today:

type IA = 
     abstract M : System.String -> System.String

Are those non-nullable strings or not? And can I implement that using this without getting a nullability warning?

{ new IA with 
      member x.M(s:string) = s
}

In principle the answer is "no" as we must take into account nullability:

{ new IA with 
      member x.M(s:string?) = s
}

But this breaks code. Emitting a warning for this kind of code is plausible but it would require very careful tuning. We could plausibly make it work, but nullability warnings would surely need to be off-by-default. And non-nullability information would not be emitted at interop boundaries.

As mentioned by the OP, another approach would to progressively opt-in to explicit non-nullability for .NET-defined types. So

Use ? in the type annotation, just like C#:

let doot (str: string?) = // explicitly nullable, warnings if used as non-nullable

let hoot (str: string) = // nullable but not explicitly so, no warnings associated with this

let woot (str: string!) = // non-nullable, warnings if matched against null

let hoot2 (str: FSharpThing) = // non-nullable but not explicitly so

Stepping back a bit, I think we should first try to formulate some guiding principles. Here is a starting set:

  1. F# should remain as simple to use as it is today
  2. The value for F# here is primarily in flowing non-nullable annotations into F# code from .NET libraries, and vice-versa, and in allowing the F# programmer to be explicit about the non-nullability of .NET types.
  3. Adding nullability/non-nullability annotations should not be part of routine F# programming (see principle 1)
  4. Nullability annotations/information should not appear in types shown in output in routine F# programming (see principle 1)
  5. F# users should primarily only experience/see this feature when interoperating with .NET libraries (the latter should be rarely needed)
  6. The feature is backwards compatible, all existing F# projects compile without warning by default
  7. The feature should produce warnings only, not hard-errors

I'm not saying those are the exact principles we want. But if we draw some bounding principles like these then design discussions may become tractable. In particular I think clearly articulating (2) is really important - what is the actual exact value this feature is bringing to the F# programmer? These annotations are valuable because they might help the F# programmer catch errors, e.g. where they pass a null to a C# library. Likewise C# people will be more often forced to interact with F# code using non-nullable semantics.

At the technical level, and as mentioned by @0x53A, it will also be crucial to consider how you get from string? to string, i.e. how you prove something is non-null. In F# by far the most natural place for this is pattern matching,

    match s with 
    | null -> ...
    | s2 -> ...  // s2 suddenly has non-null annotation

but pressure will also come to hack in a range of other ways to prove non-nullness, starting with

    match s with 
    | null -> ...
    | _ -> ...  // s suddenly has non-null annotation even though it is not re-bound

and then this:

    if s = null then 
        ...
    else
        ...

and then this

    if String.IsNullOrEmpty(s) then 
         ...
    else
        ...  // s suddenly has non-null annotation

and then many other variations. The second and third are vaguely plausible (but have problems) but the last and its more elaborate forms are deeply foreign to F# and the entire typed language design tradition it embodies. It is also likely to cause havoc in the F# compiler to try to implement this sort of thing.

We should also be aware that allowing FSharpThing? will create tensions. For example, it will create tension with F# option types. Suddenly users are faced with a choice about the "right" way to express optionality in F# code. Many will naturally gravitate to the one that is the least number of characters, which means there is no longer "one right way". This might mean Nullable<FSharpThing> is preferable.

@0x53A
Copy link
Contributor Author

0x53A commented Jun 22, 2018

I would really like, even if off-by-default and configurable through a compiler switch, a way to insert hard null-checks at all public boundaries.

That is

let f (s : string!) =
    // do something with s

should be translated to

let f ([<TheAttributeCSharpUsesToKeepTrackOfNullability>] s : string) =
    if s = null then raise (ArgumentNullException("s"))
    // do something with s

C# explicitly opted to not do this, probably for performance reasons. But not having this at least as an option would mean that you have a type system you can't really trust at runtime.

There is no semantic impact of the nullability annotations, other than the warnings. They don’t affect overload resolution or runtime behavior, and generate the same IL output code.

(https://blogs.msdn.microsoft.com/dotnet/2017/11/15/nullable-reference-types-in-csharp/)

@smoothdeveloper
Copy link
Contributor

smoothdeveloper commented Jun 22, 2018

@dsyme I guess you are aware of it, but want to confirm in context of your answer, C# will not consider all non annotated to be non null, unless you enable a new compiler flag, which you should have on by default on new projects, and ideally should convert projects to have (kind of the same way you don't necessarily want all your vb projects with "strict off" and tend to set "strict on" whenever possible).

I've used a .net language with non nil reference semantics (cobra language) and I must say that I liked it a lot, used properly, it provides a zero cost abstraction compared to usage of option type.

Now, I'm not saying that F# should go that way, but it is good to consider the fact that it is coming to C# as maybe more than "just" an interop concern and potential positive aspects if F# were to also introduce that specific compiler flag in future.

As of now, I'm not clear if the shorthand syntax is a must have for this feature in terms of having things ready for most basic interop, I'd consider more important to establish clearer picture of aim of this support, and @dsyme's point 2 is spot on.

I'd add to that point 2 that having F# types flow to more general .NET the same nil/nonnil should also be a goal, I don't think it breaks the ethos of how C# is going to handle this on their end (no runtime behaviour, just sane warnings to find potential bugs and be more explicit about intent).

@0x53A in cobra you can do equivalent of this:

let f (nillable: string?) =
   let nonNillableString = nillableString to !
   // ...

that to ! would introduce the runtime check, and reads like a cast (done via to) in that language, I guess in F# we may consider :!> to do that along with a fsharp.core function akin to box / unbox.

Looking forward to ideas and proposals in this thread 😃

@cartermp
Copy link
Member

cartermp commented Jun 22, 2018

@dsyme Regarding principles:

F# should remain as simple to use as it is today

There's only so far as this can go. The primary platform that F# runs on is moving in this direction, and it will only be a matter of time before reference types are considered non-null by default.

I would actually argue that this is easier to deal with conceptually, because the implicit null carried with any reference type whenever interacting with non-F# code is far too easy to forget and be bitten by later (see this issue where we had a crashing Annotation View in VS due to forgetting to account for null in otherwise normal F# code).

Unfortunately, it's still a bit more complicated that just this, because if you ignore all the warnings (we're going warning-level, not error-level, that's something I will pick my hill to die on) you can still crash. But at least it's pointing out that the world is a bit more complicated rather than hiding that from you only to make you deal with a NullReferenceException later.

Thus, I think this principle is fine, but we're shifting the goal posts a bit.

The value for F# here is primarily in flowing non-nullable annotations into F# code from .NET libraries, and vice-versa, and in allowing the F# programmer to be explicit about the non-nullability of .NET types.

Agreed.

Adding nullability/non-nullability annotations should not be part of routine F# programming (see principle 1)

I agree, but with this caveat. The concept of a reference type in .NET is changing. Yes, at the IL level everything is still basically the same, but this concept is going to flow directly into CoreFX and the BCL. It's a reality to deal with.

Now I will say that given the nature of non-null by default for F# constructs, there will be considerably less annotations in F# code than in C# code. But at the boundary layers of F# code that must interact with C# and .NET, I would expect annotations to come into play.

Nullability annotations/information should not appear in types shown in output in routine F# programming (see principle 1)

I would definitely expect to see nullability annotations in type signatures that appear in tools and in FSI. These should reflect the source code in the text editor. Otherwise, I think I agree, but I'm not sure what you mean.

Are you referring to this, or something else?

F# users should primarily only experience/see this feature when interoperating with .NET libraries (the latter should be rarely needed)

Yes. I think that this doesn't change the way people would write F# "kernel" code in a system today; that is, using DUs and Records to hold data.

The feature is backwards compatible, all existing F# projects compile without warning by default

Agreed. Wherever this lands (say F# 4.6), the default for new projects is to have this be on, and existing projects build with an older F# version will have it be off. I would assume that we adopt the same rules that C# does:

From The C# proposal:
In summary, you need to be able to opt in/out of:

  • Nullable warnings
  • Non-null warnings
  • Warnings from annotations in other files

Anything less on our end would be untenable.

The feature should produce warnings only, not hard-errors

Yes. I would refuse to have this implemented if it were enforced at an error level.

Regarding your example on interfaces:

Are those non-nullable strings or not?

My understanding is that if the library is compiled with C# 8, then yes these are non-nullable strings. That is, a string and an assembly-level attribute is emitted by the compiler. Languages that respect this attribute would then be responsible for interpreting the string types as non-nullable. But languages that do not respect the attribute will just see nullable strings.

In the interface implementation examples you give, yes this would be a warning, and I think it should be on by default for new F# projects, just like it would for C#.

Regarding flow analysis:

C# will "give up" in some places. @jaredpar could comment on some examples.

I agree that there will be some pressure to try and add more as annoying little edge cases come up. But this is something we can do progressively.

Regarding the tension between this and F# options, and having one way to do things

I think the ship has already sailed on this front. We already have "typesafe null" with options and struct options, "implicit null" with reference types, null itself, and Nullable<'T> from the BCL. You could also argue that Choice and Result are somewhat similar in nature.

What this does is it changes reference types from "implicitly null" to "says if it can be null or not".

The right way to show this in syntax and how to rationalize its usage when compared with Option could be tricky, as there seems to be a longstanding idiom of the following:

let tryGetValue (s: string) =
    match s with
    | null -> None
    | _ -> Some s

I would argue that this is inferior to nullability annotations, because we're conflating null with "conceptually nothing", when null is a specialized consideration when programming for .NET. Making that leap as a new F# programmer isn't too difficult, but it's still a bit to learn, and I remember thinking "this sort of sucks" when learning. Now in F#-only code, options are wonderful, and when learning F# it became quite clear that I could evict null from my worries so long as I was behind the boundaries of my system.

To that end, I'm not convinced that there's too much tension here, certainly not if this is documented well.

@cartermp
Copy link
Member

cartermp commented Jun 22, 2018

Another consideration, perhaps even a principle, is that this feature would also need to work well with Fable. I believe this is not particularly difficult, as TypeScript's nullable types offer a view into how that works for web programming, and nullable types as-proposed in C# are modeled after them a bit, especially the opt-in nature of them.

Speaking of how things are done in TypeScript, this is the syntax:

function f(s: string | null): string { ... }

Guarding against null with a union is fantastic syntax.

Note that they use ! at the end of an identifier to say, "ignore the fact that this may be null". I think a similar mechanism is a decent idea.

@cartermp
Copy link
Member

cartermp commented Jun 23, 2018

Just to summarize how some other languages are approaching this:

Swift - Optional

Optionals in Swift are variables that can be nil. These are not the same as F# options.

There are two forms of their syntax:

func short(item: Int?) { ... }
func long(item: Optional<Int>) { ... }

To use the underlying value, you must use any of the following syntax:

Using control structures to conditionally bind the wrapped value:

let x: Int? = 42

// 'if let', but could by 'guard let' or 'switch'
if let item = x {
    print("Got it!")
} else {
    print("Didn't get it!")
}

Using ?. to safely dot into an item:

if myDictionary["key"]?.MyProperty == valueToCompare {
    ...
} else {
    ...
}

Using ?? to provide a default value if the item is nil:

let item = myDictionary["key"] ?? 42

Unconditional unwrapping (unsafe):

let number = Int("42")!
let item = myDictionary["key"]!MyProperty

Additionally, later versions of Swift/Obj-C added nullability annotations, because anything coming from Obj-C would automatically be annotated as identifier!; i.e., an implicitly unwrapped optional. Luckily, with C# 8, it doesn't appear that this problem is applicable.

Overall, this kludges together nullability and optional as we know it in F#.

Kotlin - Null Safety

In Kotlin, the type system distinguishes references that can be null and those that cannot. These are denoted with ?:

var a: String = "hello"
a = null // Compile error

var b: String? = "hello"
b = null // OK

Accessing a value from a type that is nullable is a compiler error:

val s: String? = "hello"
s.length // Error

Through flow analysis, you can access values from nullable types once you've checked that it's non-null:

val s: String? = "hello"
if (s != null && b.length > 0) { // Note that this is legal after the null check
    print("It's this long: ${b.length}
}

However, flow analysis does not account for any arbitrary function that could determine nullability:

fun isStrNonNull(str: String?): Boolean {
    return str != null
}

fun doStuff(str: String) {
    println(str)
}

fun main(args: Array<String>) {
    val str: String? = "hello"
    
    if (isStrNonNull(str)) {
        doStuff(str) // ERROR: Inferred type is String? but String was expected
    } else {
        println("womp womp")
    }
}

You can also use ?. to access values from a potentially null reference:

val l = str.?length // Type is Int?, returns 'null' if 'str' is null

You can also use the elvis operator ?: to conditionally assign another value if the reference is null and you don't want the resulting type to also be nullable:

val l = null?.length ?: -1 // Type is Int

You can also use !! to assert that a reference is non-null to access things in an unsafe manner:

val l = str!!.length

This will throw a NullPointerException if str is null.

You can also safely case a nulable reference to avoid a ClassCastException:

val x: Int? = a as? Int

Here, x is null if the cast is unsuccessful.

This is very close to how C# is approaching the problem, with the major difference being that Kotlin emits errors instead of warnings.

TypeScript - Nullable Types

In TypeScript, there are two sepcial types: null and undefined. By default, they are assignable to anything. However, when compiled with --strictNullChecks, TypeScript code will disallow assignment like that:

let s = "hello";
s = null; // ERROR, 'null' is not assignable to 'string'
let sn: string | null = "hello";
sn = null; // OK

sn = undefined; // ERROR, 'undefined' is not assignable to 'string | null'

let snu: string | null | undefined = "hello";
sn = undefined // OK

The idea that a type could be null is handled by a union type. When compiling with this flag, programmers must specify types like this.

To use the value, you need to guard against null:

function yeet(sn: string | null) {
    if (sn == null) {
        ... // handle it
    } else {
        ... // Use 'sn' as if it were a string here
    }
}

However, like in Kotlin, the compiler can't track all possible ways something could or could not be null.

To handle this, you can assert that something is non-null with the ! operator:

function doot(name: string | null) {
    return name!.charAt(0)
}

Because this is an assertion, the programmer is responsible for ensuring the value is non-null. If it is null, fun behavior will occur at runtime.

My opinion

TypeScript handles this the most elegantly, and with all other things held constant, I'd prefer it be done like this in F#. It doesn't feel like Yet Another Special Case to account for, but rather an extension of an already-existing feature, and feels a bit more "baked in" than how other languages handle it.

However, at the minimum, we'd need the ability to specify ad-hoc unions (#538), which means the scope of the feature would be larger. And that doesn't get into the other implications of a syntax like this.

@cartermp
Copy link
Member

cartermp commented Jun 23, 2018

Another thought: having this be configurable to also be an error independently of making all warnings errors, along the lines of what @0x53A was saying. I could imagine that F# users would want this to be a hard error in their codebases, but not have all warnings also be that way.

@alfonsogarciacaro
Copy link

alfonsogarciacaro commented Jun 24, 2018

This seems it could be a very nice feature for Fable users too. Just for the record, I'll leave some comments about dealing with null when interacting with JS from F#/Fable:

  • As commented for TypeScript, JS has both null and undefined. Fable in general uses strict equality checking in the generated JS, but just for null checks, we are using non-strict equality (in JS x == null evals to true either if x is null or undefined). So in principle we can say in Fable eyes null and undefined are the same thing.

  • Fable erases options (Some 1 becomes 1 and None becomes null). At one point I had some hopes that we could use this fact to implement null checks using F# options, but now I'm not so much convinced about this because: a) option erasure is an implementation detail that could change in the future, b) sometimes options are actually wrapped at compile time and c) JS bindings don't usually take this into account (see below). So I don't think a feature like the one proposed here would clash with options in Fable.

Another reason to erase options in Fable was to use them to represent TypeScript optional properties. Against, I don't think this would be any redundancy, and we'll probably still use options for this, because missing properties eval to undefined.

  • In general, Fable has similar null-related problems when interacting with JS as F#/C#: strings can also be nullable (well, numbers and booleans can be nullable too in JS, but usually solve that problem but putting the fingers in my ears and saying LALALA very loud) and ts2fable decorates interfaces for JS bindings with AllowNullLiteralAttribute, so types coming from JS are assumed to be nullable. However, besides optional properties, ts2fable doesn't wrap strings or types coming from JS with option.

TBH, I haven't been involved very much in latest ts2fable development so I'm not sure if it's doing something to represent TS nullable types in F#.

In summary, I'm assuming that anything you do to make interop with C# safer it'll be probably good for JS interop too. Fable interoperability is usually compared with Elm, which uses something called "channels" to make sure you won't get any null-reference error at runtime. Fable makes interop easier and more direct, but less safe in this sense. Having null-checks enforced by the compiler is likely to be very welcome by users who prefer to make the interop safer.

@dsyme
Copy link
Collaborator

dsyme commented Jun 25, 2018

A basic shift of the platform to non-null-reference-types-as-the-default can, I think, only be good for F#.

I guess I'll admit I'm still sceptical C# and the platform will shift that much. The more it shifts the better of course. However I have used C# lately to write the first version of a code generator and the pull towards using nulls is very, very strong in hundreds of little ways. For example, any use of Json.NET to de-serialize data immediately infects your code with nulls. While this is also true when using Json.NET with F#, there are other options available in the F# community for JSON deserialization which give you strong and much more trustworthy non-nullness. So I'm sceptical C#'s version of non-nullness is going to be "trustworthy", even in the sense that the typical C# programmers feels they can trust the feature in their own code, let alone other people's code. The whole C# language and ecosystem is geared towards making "null" very pervasive. I'm also a little sceptical we will see that many non-nullness annotations appearing in framework libraries, but let's see what they come up with...

Anyway, I suggest this principle:

We don't undermine the "strong" notion of non-nullness in the F# world by somehow overly embracing a "weak" notion of non-nullness flowing in from C#

BTW some examples of where the complexity added by this feature will be in-the-face of users to a worrying degree:

  • Optional arguments. e.g. Are you allowed the stomach churning ?x : string?.

    As an aside the typescript syntax makes the question about optional arguments clearer: the meaning of ?x: (string | null) is clearer than ?x: string?. The syntax also sets incentives differently, people would be less inclined to use the annotation as an adhoc representation of the absence of information. Indeed perhaps int | null is a reasonable syntax for Nullable<int> in F#.

  • Array types. How do you write a might-be-null array type now? e.g. string[]?, array?<string>, string[] | null etc.

    Either way suddenly you now have four variations:

    array<string>
    array?<string>
    array<string?>
    array?<string?>
    

    Looking at these I think again I prefer the typescript syntax:

    array<string>
    array<string> | null
    array<string | null>
    array<string | null> | null
    

    Note that the vast majority of F# code uses the first and should be able to continue to do that.

  • What will Array.zeroCreate<string> return? array<string> or array<string | null>? I suppose the latter, but then we will likely need an Array.zeroCreateUnchecked or Array.zeroCreateNonNull or something.

  • What will Unchecked.defaultof<string> return? string (since it is "unchecked") or string | null (since, hey, it's null). Will we need a Unchecked.nonnull<string>? What do you write to cast with-null -> with-null?

Agreed....the default for new projects is to have this be on, and existing projects build with an older F# version will have it be off.

Note that F# Interactive scripts will now give warnings (because they have no mechanism to opt-in to particular language features or language version level).

@dsyme
Copy link
Collaborator

dsyme commented Jun 25, 2018

@cartermp I'll make a catalog of proposed design principles in the issue description

@0x53A
Copy link
Contributor Author

0x53A commented Jun 25, 2018

Note that F# Interactive scripts will now give warnings (because they have no mechanism to opt-in to particular language features or language version level).

something similar to #light "off"?

@dsyme
Copy link
Collaborator

dsyme commented Jun 25, 2018

Nullability annotations/information should not appear in types shown in output in routine F# programming (see principle 1)

I would definitely expect to see nullability annotations in type signatures that appear in tools and in FSI. These should reflect the source code in the text editor. Otherwise, I think I agree, but I'm not sure what you mean. Are you referring to this, or something else?

I think I need to get a feeling for how often string | null (or string? if that's the syntax) would appear in regular F# coding.

For example, the tooltips for LINQ methods are already massive and horrific. It is really, really problematic if we start showing

member Join : source2: (Expression<seq<string | null> | null> | null) * key1:(Expression<Func<(string | null), T | null> | null> | null)) * key2:(Expression<Func<(string | null), T | null> | null> | null)) 

instead of the already-challenging

member Join : source2: Expression<seq<string>> * key1:(Expression<Func<string, T>>)  * key2:(Expression<Func<string, T>>)

just because there are no non-nullness annotations in the C# code. We have to be careful not to blow things up.

We have quite a lot of flexibility available to us in when and how we choose to surface annotation information. For example, in tooltips we already show with operator (+) instead off the full gory details of static member constraints. This is much more usable and has definitely been the right choice.

Anyway we can talk through the details later, however looking at the above example I'm certain we will need to be suppressing nullness annotations in many situations flowing from .NET in some situations.

@dsyme
Copy link
Collaborator

dsyme commented Jun 25, 2018

something similar to #light "off"?

Yes, scripts should probably really be able to have all of this sort of thing:

#langlevel "5.0"
#define "DEBUG"
#options "/optimize-"

@cartermp
Copy link
Member

cartermp commented Jun 25, 2018

@dsyme With respect to tooling, I'd rather we plan to show annotations (and build them out as such), and then find ways to reduce them down based on feedback. That is, I'd rather start from a place of showing all information and then reducing down to an understandable notation rather than start from a place of eliding information.

@dsyme
Copy link
Collaborator

dsyme commented Jun 25, 2018

That is, I'd rather start from a place of showing all information and then reducing down to an understandable notation rather than start from a place of eliding information.

The example above is easily enough to show that we need to elide (and there are much, much worse examples - honestly, under default settings some method signatures would run to a page long). There's absolutely no about that if we are remotely serious about design principle (1) "don't lose simplicity", and we've already walked this territory before. So we can take that as the first round of feedback I think, given I'm a user of the language as well :)

Really, I'm going to be very insistent about taking design principle (1) seriously. Simplicity is a feature, not something you tack on afterwards.

@cartermp
Copy link
Member

cartermp commented Jun 25, 2018

In that example I would not expect that signature to change, since the default for all reference types in this new world is non-nullable. That is, any existing reference type will not have a changed signature unless we modify it to be as such. AFAIK, this is not the plan for the BCL (except in the small number of cases where it explicitly returns or accepts null), and it shouldn't be for FSharp.Core either.

I agree that signatures in any context could get out of control, and I could see something like this be possible in tooltips to at least help with that:

member Foo : source: (seq<'T>) * predicate: ('T -> 'U-> bool) * item: ('U)

    where 'T, 'U are nullable

Generic parameters:
'T is BlahdeeBlah

Not sure how things in signature files would be made better. That said, I also think there's a slight positive in having it be more tedious and difficult to represent nullable data to reinforce that this is not the default way anymore.

@dsyme
Copy link
Collaborator

dsyme commented Jun 25, 2018

In that example I would expect that signature to not change, since the default for all reference types in this new world is non-nullable. That is, any existing reference type will not have a changed signature unless we modify it to be as such. AFAIK, this is not the plan for the BCL, and it shouldn't be for FSharp.Core either.

How can that be the case? IQueryable.Join is defined in a .NET assembly. All we see is the .NET metadata, a IL type signature. There is no annotation in that .NET metadata if, for example, targeting .NET 4.7.2 or .NET Core 2.1 or .NET Standard 2.0, and we surely have to assume nullability for both inputs and outputs if there is no annotation - half the Framework accepts null as a valid (and in some cases necessary) value..

I assume some future version of .NET Standard will come with a much more constrained IQueryable.Join, with lots of useful non-nullable annotations, which is fine. Or equivalently there may be an upcoming version with a big fat annotation saying "hey, it's ok to assume non-nullable for this DLL unless we say otherwise!". But even then I suspect an awful lot of methods in framework functionality will list null as a valid input.

Basically for every single reference or variable type X flowing in from .NET assemblies we will surely have to assume "X | null" in the absence of other information. We can't assume "non-nullable" unless there's information saying that's a reasonable assumption. Whether we choose to turn that into warnings and/or display the information is another matter. It could be that for entirely unannotated .NET 4.x/.NET Standard 2.x/.NET Core/C# 7.0 assemblies we do more suppression than for annotated assemblies.

I could see something like this be possible in tooltips to at least help with that:
member Foo : source: (seq<'T>) * predicate: ('T -> 'U-> bool) * item: ('U)
where 'T, 'U are nullable

Yes, something like that. Other tricks are

  • only showing outer-level annotations
  • only show annotations if they are relevant to an error
  • compress even further on an input/output basis and just say "some inputs may accept null"

Not sure how things in signature files would be made better.

These would have to be explicit

That said, I also think there's a slight positive in having it be more tedious and difficult to represent nullable data to reinforce that this is not the default way anymore.

Yes, and I'm happier with string | null over string? for this reason

@smoothdeveloper
Copy link
Contributor

smoothdeveloper commented Jun 25, 2018

How can that be the case? IQueryable.Join is defined in a .NET assembly. All we see is the .NET metadata, a IL type signature.

Maybe the C# implementation will add an assembly level attribute describing if the assembly is compiled with non null ref enabled. If this is the case, presence of this assembly attribute would turn on the extended signature logic.

@cartermp
Copy link
Member

cartermp commented Jun 25, 2018

Re: BCL

After talking with @terrajobst, the plan is to do something where types are annotated as nullable if the input type is valid or the return type is null. However, there are numerous places where they just throw an exception if an input is null, or they never return null explicitly. Thus, the current plan does not involve blanket annotations for everything.

I'm not sure of the vehicle, but because it's unlikely that we'll be updating older .NET Framework versions, I suspect that some sort of shim will be involved.

But I think the overall goal is to make this feature also "work" on the BCL as much as they can, since it's arguably the most valuable place to have it "enabled". And since you can target older BCL versions with C# 8.0, this would have to come in some way.

@cartermp
Copy link
Member

cartermp commented Jun 25, 2018

@dsyme Also just to confirm, if we were to try to adopt a TypeScript-like syntax, would that mean that #538 is also a prerequisite to this feature, or would this just be a specialized thing that we expand later?

@cartermp
Copy link
Member

cartermp commented Jun 25, 2018

More considerations (some from the C# proposal):

  • Allowing the programmer to assert non-nullability via special syntax when flow analysis cannot determine non-nullability?

Examples:

// ! postfix to assert nullability
let len (s: string | null) =
    if not(String.IsNullOrWhiteSpace s) then
        s!.Length
    else -1

// !! postfix to assert nullability
let len (s: string | null) =
    if not(String.IsNullOrWhiteSpace s) then
        s!!.Length
    else -1

// "member" access, a la java and Scala
let len (s: string | null) =
    if not(String.IsNullOrWhiteSpace s) then
        s.get().Length
    else -1

// Cast required
let len (s: string | null) =
    if not(String.IsNullOrWhiteSpace s) then
        (s :> string).Length
    else -1
  • Casting between S[] --> T?[] and S?[] --> T[] produces a warning?

Living with [<AllowNullLiteral>] and null type constraint:

Do we want to "alias" the types/type signatures? E.g.

[<AllowNullLiteral>]
type C< ^T when ^T: null>() = class end

Is now this in tooltips:

type C<'T | null> =
    new: unit -> C<'T | null> | null

// As opposed to the current style
// Note that [<AllowNullLiteral>] doesn't surface here
type C<'T (requires 'T: null)> =
    new: unit -> C<'T>

And when instantiated:

let cNull = null
let c = C<string | null>()

Shows:

val cNull: 'a | null
val c : C<string | null>

// As opposed to the current style
// Note that the type constraint is not shown

val cNull : 'a (requires 'a: null)
val c : C<string>

Or find a way to make the nullable syntax be aliased by existing things today, i.e., do the reverse?

My current opinion

  • Introduce ! to assert non-nullability, since C#/TypeScript/Swift already do this, so it's "familiar in that sense (though this may be confusing within a CE)
  • Yes on the array casting warnings
  • Try to "alias" what we already have in signatures to use nullability syntax

@jaredpar
Copy link

jaredpar commented Jun 26, 2018

@smoothdeveloper

Maybe the C# implementation will add an assembly level attribute describing if the assembly is compiled with non null ref enabled.

There will be a way to detect via metadata that an assembly was compiled with C#'s nullable reference type feature enabled.

@cartermp
Copy link
Member

cartermp commented Jun 26, 2018

Oooof, weirdness with the null constraint. So, given the following:

[<AllowNullLiteral>]
type C() = class end
type D<'T when 'T: null>() = class end

If C were not decorated, it would be a compile error to parameterize D with it. That would make you think that the null constraint on D could be made equivalent to 'T | null, but the compiled output says otherwise:

    [Serializable]
    [CompilationMapping(SourceConstructFlags.ObjectType)]
    public class D<T> where T : class
    {
        public D()
            : this()
        {
        }
    }

As you'd expect, you can parameterize D with any reference type today (unless it is declared in F# and is not decorated with [<AllowNullLiteral>]). That means that in a C# 8.0+ world, both nullable and non-nullable reference types could be used to parameterize D.

However, as per the C# proposal:

The class constraint is non-null. We can consider whether class? should be a valid nullable constraint denoting "nullable reference type".

Ooof - it's a bit mind-warping that in F# syntax we declare something as effectively nullable, but it emits as something that is non-null. If class? were done for C# 8.0, then we'd need to either change how the null constraint in F# compiles, or have it be a specialized F# thing that remains for back compat and have people move towards something like this:

type D<'T | null>() = class end

I don't really see a way forward with this unless we change existing F# behavior somehow. What I'd like to have happen is the following:

  • [<AllowNullLiteral>] is a no-op, but doesn't emit a warning when you use it (or maybe it does?)
  • The null type constraint is equivalent to 'T | null

However, that loosens requirements around null when compared with F# code today, which doesn't feel right. Alternatively:

  • [<AllowNullLiteral>] behaves exactly as it does today
  • The null constraint behaves exactly as it does today
  • There is an additional 'T | null constraint available ("nullable constraint")
  • If you attempt to parameterize a type that does not have the nullable constraint with a nullable type (e.g., string?), then you get a warning

I'm already dying on the inside at thinking about how you can use a "null constraint" or a "nullable constraint", though 😢.

@jwosty
Copy link
Contributor

jwosty commented Jul 9, 2018

@isaacabraham In a perfect world, yes, that would be awesome. .NET ideally never should have had implicit nulls, but it can never be undone without majorly breaking changes, even from the language level. Old F# code that compiled would suddenly not compile anymore if referencing C# code compiled with the old compiler. E.g., take this combination which currently compiles:

// C# 7
public class CSharpClass {
    public static string GetString() => "foobar";
}
// F# 4.1
printfn "%s" (CSharpClass.GetString())

If, say, F# 5 interpreted types that aren't marked CLR non-nullable as Option (or Nullable), upgrading just the F# snippet would break this code. You'd have to recompile the C# with the new compiler. I don't think it's possible to do this in a non-breaking way unfortunately.

@isaacabraham
Copy link

isaacabraham commented Jul 9, 2018

@jwosty taken from above:

Data which we don't know about (C#7 and below -> maps to a nullable string, exactly what we have today)

In other words, in the absence of any "nullable" marker / attribute that would be emitted by the C# compiler, we would treat things exactly as today. There would be no change when working with code that was compiled with C#7.

Only when that C# assembly was rebuilt with C#8 would things change from the F#5 perspective:

  1. Either the value was marked as non-nullable, in which case F# would prevent us checking against / assigning to null.
  2. Or the value was marked as nullable, in which case F# would surface it as an option.

@zpodlovics
Copy link

zpodlovics commented Jul 9, 2018

@isaacabraham I may misunderstood what you said - whenever I saw Option type I always associate to the F# option type and not both F# option and F# valueoption/voption type (which is not yet in the core).

Let assume I have a array of nullable int32. What type the List.tryFind should return? Let assume I have a list of nullable string. What type the List.tryFind should return? null is a prefectly legal value for for option value here. What will be the difference between having (finding) a null value in the array and not founding with tryFind? How the functional operation eg.: map supposed work if not even possible to represent every type habitant? Operation that supposed to have the same number of elements suddenly returns different number of elements?

However a list of non-nullable int32 and a list of non-nullable string does not have null as a habitant, so non-nullable type option type should not allowed to have null as a value. By definition option type must able to represent every type habitant as value.

Merging different concepts because it looks somewhat familiar, and it's easy to do will make thing even more complicated.

@zpodlovics
Copy link

zpodlovics commented Jul 29, 2018

Update: It looks like I missed the memo about the nullable value types implementation. However I still feel a bit surprised with the implemented behaviour.

The following examples was tried with 5.12.0.301-0xamarin8+ubuntu1604b1 and 4.1.33-0xamarin7+ubuntu1604b1:

type [<Struct>] S<'T> =
  val Value: 'T
  new(v) = { Value = v}

let a = Array.zeroCreate<S<System.Nullable<System.Int32>>> 10
let found1 = Array.tryFind (fun i -> i = S<_>(System.Nullable<System.Int32> 10) ) a
let found2 = Array.tryFind (fun i -> i = Unchecked.defaultof<S<System.Nullable<System.Int32>>>) a

What will be the difference between not founding with tryFind and having (finding) a null value in the array?

val found1 : S<System.Nullable<System.Int32>> option = None
val found2 : S<System.Nullable<System.Int32>> option = None

This is probably the best comment I have seen in the review:

"Design a fully sound (apart from runtime casts/reflection, akin to F# units of measure) and fully checked non-nullness system for F# (which gives errors rather than warnings), then interoperate with C# (giving warnings around the edges and for interop)"

@cartermp
Copy link
Member

cartermp commented Jul 29, 2018

@zpodlovics I'm not sure I follow.

  • The F# RFC is named "Nullable reference types", not "Nullable types"
  • F# already has System.Nullable support for value types (which we should be calling "Nullable value types" moving forward)
  • Use of nullable value types in F# already presents challenges, and nullable reference types won't be making those any better or worse due to being a different feature

@zpodlovics
Copy link

zpodlovics commented Jul 29, 2018

@cartermp Thanks for the update. It looks like I missed the memo about the nullable value types implementation. The default behaviour was too surprising to me so I assumed its not yet or not fully implemented. Later I found the custom operators and the test cases...

@matthid
Copy link

matthid commented Jul 31, 2018

I just read through the suggestion and the discussion here and I feel like nobody is really too happy with any solution. Therefore let me just try to suggest a bit of craziness (probably too hard to implement but maybe some food for the discussion).

What if we don't have a "real" option type but we make the feature "look" like it is similar to an option:

type Optionish<'a> =
   | null
   | Ref of a

Note that this Optionish type will not exist anywhere and there is no way to define it yourself.

I'm not sure about the name (we probably would choose another one), but I guess it has to be different from Option. It would behave basically very similar to what @cartermp specified as 'a | null, so you can directly use 'a members on variables and use it in places where 'a is expected (with the new compiler warning), but you can use it in pattern matching and you can use it in a way that feels more natural (and without warnings):

match v with
| null -> ...
| Ref i -> ...

This basically should "feel" like using Option.

Without thinking this through too much, but we might be able to somehow overload Some and None in pattern matching on Optionish types, but I guess that will make type inference a bit more difficult.

It basically is quite similar to string | null but it hopefully feels a lot less "hacked" in.

Could this work out or do I miss something here?

Edit:
It's basically a "unsafe" version of "option" where you can do everything you want and only get warnings instead of errors and the feature would disappear on compilation (besides the attributes).

@cartermp
Copy link
Member

cartermp commented Jul 31, 2018

@matthid one of the biggest reasons for this feature is to warn on existing code that isn't null safe today.

For example:

let len (s: string) = s.Length

...

let value = getStringValueSomewhere() // Could by null

printfn "%d" (len value)

As you're well aware (and may have been bitten by in the past!), this code could easily throw an NRE.

With the feature as-proposed in the RFC, len value would emit a warning because value is possibly null, which I believe we can agree is a useful thing.

For this to also happen with Optionish<'T> (love the name 😄), would this be implicitly convertible between reference types? Would it be possible to realize this with the following null-safe code?

let xn = SomeNullableReferenceType()

match xn with
| null -> // handle null
| x -> x.Something()

i.e., can I use the same syntax throughout, implicitly convert, or something else?

FWIW I would much rather find a way to make things be based on Option or option-ish things, but I haven't thought of a way for that to work yet.

@matthid
Copy link

matthid commented Jul 31, 2018

@cartermp I guess we would need a bit of experimenting but my initial feeling is that for the first example let value that if getStringValueSomewhere returns an Optionish you would get a warning on len value.

Regarding the second example my initial feeling would be that x should be an Optionish again (as it feels unnatural to change the type here), so in a first attempt I would suggest emitting a warning at x.Something with the fix to be explicit in the match (it depends a bit on how we decide to match here. My Ref was only a suggestion).

I can see how we could argue that this might lead to "too many warnings", but maybe not? In any case I feel like this initial implementation would actually be simpler and changing the "type" of a variable along the code is not something we are familiar with in F#, but it is possible that we add that for Optionish when we encounter too many warnings.

It's actually similar with Option we kind of get punished for writing if Option.isSome v then v.Value else ... and it feels wrong, even if it is perfectly safe, kind of the same goes for your match.
So the thinking would be: Kind of get "punished" for working with optionish too long, either match on it or convert it to proper option. This obviously sets a lot of trust in the ecosystem actually shifting.

If we decide that

match xn with
| null -> // handle null
| x -> x.Something()

is the "proper" way to match on optionish then I'd expect that

match xn with
| x -> x.Something()
| null -> // handle null

should work as well after this change (I hope that makes it a bit more clear on how this Optionish is meant).

@cartermp
Copy link
Member

cartermp commented Jul 31, 2018

Taking a quick step back, the main concerns here (aside from principles listed at the top) are:

  • Make null explicit
  • Let people know when their code isnull-unsafe
  • Don't make people change code that is perfectly null-safe
  • Offer a way to opt in/out of nullability warnings to varying degrees

Would this sort of approach be able to satisfy each of these points?

It clearly does for the first, but I'm not sure about the other three. The 2nd and 3rd point could be a possibility if we define an implicit conversion between this structure and any reference type, respecting the various attributes that C# 8.0 can emit. I'm not sure how the ergonomics of that would be, though, since the type would have to be ephemeral. So this would be more akin to units of measure. Some further questions for that would be how else to check for null. Today, I can use isNull and other boolean-based checks and be perfectly safe. Would that also be okay with this structure?

I'm also unsure how tunability would work here. If I wanted to turn off all warnings, then how would it look using one of these and a "normal" reference type interchangeably?

@matthid
Copy link

matthid commented Jul 31, 2018

Don't make people change code that is perfectly null-safe

I think it is pretty clear that this is not achievable in the general case. The question is to what degree we push people to be explicit about it. I feel like Optionish<'a> can be pretty much the same as 'a | null the difference is that we can start with being more "explicit" at first and then add features (ie remove warnings) along the way. With 'a | null it feels quite invasive from the beginning (and I mean from a language decision, not from the code changes which might be exactly the other way around).

It's also a design decision of what code would "ideally" look like and in what direction we push people regarding the language design. Given the 3rd point it seems to have been decided that "match on null" and "if isNull" is exactly what we want. I'm not sure I agree with this and people here have spoken about the "option" programming flow. My suggestion is trying to push it a bit into that programming model while still having all options on the table.

Anyway, I'd like to add that I have not thought this through entirely. I just tried to formulate some thoughts that crossed my mind while reading the discussion here on using Option and what syntax to use. So I don't have an answer to all the questions. I just hoped someone with more knowledge would either point out my errors or would be able to finish the thought :) There are just too many edge cases here...

@cartermp
Copy link
Member

cartermp commented Jul 31, 2018

Yeah, lots of edge cases. But this sort of stuff is definitely useful! I think the door is quite open to thinking about how to approach this, and perhaps some "level" of null-safety can be accomplished with a structure like this. @dsyme had some thoughts along those lines, which are called out in the RFC (just not in much detail)

@charlesroddie
Copy link

charlesroddie commented Oct 7, 2018

Some feedback on the RFC.

Non-nullability Assertions

For example, imagine a C# 8.0 project consuming an older assembly that was compiled with an older C# compiler that has no notion of nullability, or it was compiled such that reference types are defined in a scope where NonNullTypes(false).

treating all components we cannot guarantee as non-nullable as nullable is, we feel, "in the spirit of F#", which mandates safety and non-nullness wherever possible

I don't agree with this. When you use standard reference types from C# in F# now, they are treated as non-nullable. If programmers know they may be nullable then they can do null checks, but there is no warning from just using the objects as non-null without checking.

This behaviour should continue. It is practical now, and will become safer as nullable types get marked as nullable, leaving fewer nullables that are not marked as such.

There will be two categories, nullables and non-nullables. Oblivious types should be treated the same as non-nullables. The categorization is a warning system for likely nullables, more than a guarantee of non-nullability.

Non-nullability Assertions
Not have it? This is untenable

We won't need non-nullability assertions. Since we should mark oblivious types as non-nullable, nullable types will really be nullable. It's fine to warn on usage as a non-nullable type since it's appropriate to pattern match here or convert to an option and then pattern match.

There will of course be a function A|null -> A (somewhere in between Option.getValue and id) and that is good enough for this use-case. This is all we need.

! is too easy and the only reason to make it that easy is that there are a lot of nullables that are really non-nullable. notnull{ } is way too hard and the only reason to make it so hard is that it's too easy to get tagged as nullable. But F# code should only tag things as nullable when they have been explicitly created as nullable in F# code, or come from outside as definitely nullable. Almost all F# code should get tagged as non-nullable automatically.

Emitting F# options types for C# consumption

This becomes much less important. F# option consumption in C# has never been good enough to use. Now we will have a good way to do it for most cases.

I see this as being a large improvement to interop, and affecting code on the boundary of F#/C#. It will probably be a good pattern to immediately convert C# nullables to options (explicitly, a simple function A | null -> A option) and then convert options to nullables when exposing to C# (A option -> A | null).

@cartermp
Copy link
Member

cartermp commented Oct 7, 2018

@charlesroddie What you're saying here goes against the entire point of safety with nullability:

If programmers know they may be nullable then they can do null checks, but there is no warning from just using the objects as non-null without checking.

This behaviour should continue. It is practical now, and will become safer as nullable types get marked as nullable, leaving fewer nullables that are not marked as such.

Today, reference types are not non-nullable. They are implicitly nullable, and this is where the source of bugs lie. Failing to warn on the dereference of a nullable type would be throwing out one of the largest benefits of the feature set: knowing that your code has a bug in it. We will not take the route of failing to warn in this case. That would make F# even less null-safe than C#.

Oblivious types should be treated the same as non-nullables.

We will not be doing this. A principle of F# is safety by default, and assuming something is non-nullable when we can't know if it is violates that principle. Especially when you consider that F# programmers rely heavily on type inference, knowingly inferring a state of the world where you can be surprised with exceptions is not a good thing.

The only alternative I will consider for this is to treat them as explicitly oblivious, like in C#. But given type inference in F#, this is a difficult one to pull off.

We won't need non-nullability assertions.

We do. This doesn't have anything to do with null-obliviousness. It has to do with the compiler understanding if something is non-nullable based on checks performed in code. Nearly all "normal" checks will be, and there will be additional attributes that can be used to signal to the compiler that something is checking for null. But covering all possible cases is not computationally feasible, so there needs to be an escape hatch for the rare cases when the compiler can't figure out something that you can.

The function you are referring to is a non-nullability assertion. It will throw if something is null.

This becomes much less important. F# option consumption in C# has never been good enough to use. Now we will have a good way to do it for most cases.

F# options emit with null for the None case, which means they must be treated as a nullable reference type, or else we risk a source-breaking change for C# consumers of F# code, which is not an acceptable scenario. The only thing we can do (that I'm aware of right now) that doesn't outright break existing code is to add a constraint of 'T to be non-null, such that code like Some null emits a warning now.

@charlesroddie
Copy link

charlesroddie commented Oct 7, 2018

Thanks for the reply. I am only starting to read the conversations about this.

It seems that we want, when consuming C# properties, to map type Knowledge = KnownSometimesNull | Unknown | KnownNeverNull to type Behavior = AccountForNull | AssumeNonNull. Where to put the Unknown middle band. Currently everything is Unknown and effectively mapped to AssumeNotNull in F# tooling.

Hopefully C# takeup is swift and comprehensive and there will be very few Unknowns. Then Unknown -> AccountForNull is a small cost for the benefit of complete safety.

If the Unknown band is very large and consists mostly of NeverNulls, then it will be a very large cost.

@cartermp
Copy link
Member

cartermp commented Oct 7, 2018

Agreed, and the transition period will present challenges for all parties involved. For example, we can't go and rewrite the .NET Framework to use this, so we're working on annotation files that act as a shim for the BCL. CoreFX will likely have this at first too, but I expect that OSS contributors will immediately start rewriting the signatures of the real APIs once C# 8 and F# 5 are released.

Striking the balance between ease of transition and letting people know about their unsafe code today is going to be difficult. I expect we'll be tuning pieces of the design when we have a more complete prototype to ship in our nightly feed. We'll also be watching the C# feedback.

@7sharp9
Copy link
Member

7sharp9 commented Nov 12, 2018

This thread is quite long so I may have missed a previous discussion, but mixing optional params and nullable params with the ? in member definitions is going to lead to a lot of confusion, as in the optional syntax

            ProvidedProperty(
                name, 
                propertyType, 
                getterCode = (fun args -> Expr.FieldGet(args.[0], field)),
                ?setterCode = blah

@cartermp
Copy link
Member

cartermp commented Nov 12, 2018

@7sharp9 I suggest raising concerns here: fsharp/fslang-design#339

This point is actually written down in the RFC as a downside, but given that the other proposed syntax was a breaking change, we're in a bit of a tight spot.

@7sharp9
Copy link
Member

7sharp9 commented Nov 12, 2018

Done.

@dsyme dsyme changed the title Add non-nullable instantiations of nullable types, and interop with proposed C# 8.0 non-nullable reference types Add nullable reference types [RFC FS-1060] Oct 13, 2021
@reinux
Copy link

reinux commented Nov 14, 2021

Any movement on this front?

@xperiandri
Copy link

xperiandri commented Apr 2, 2022

I agree with @isaacabraham that the best way would be to merge F# Options world and .NET non-nullable world like this:

  1. F# constides all .NET nullable types as Option<T> (or ValueOption<T> or T explicitly with cast or typed let binding)
  2. F# no longer emits FSharpOption<T> but produces T? types in IL. So that neither FSharpOption<T> nor FSharpValueOption<T> will exist in assemblies built with F# 7

@Happypig375
Copy link
Contributor

Happypig375 commented Apr 2, 2022

not possible due to back compat; not possible due to generics representation between struct and class.

@xperiandri
Copy link

xperiandri commented Apr 2, 2022

Easily fixed the same way as C# does with compiler flag and MSBuild property <Nullable>enable</Nullable> that return behavior to the previous one

@xperiandri
Copy link

xperiandri commented Apr 2, 2022

Want an old behavior, use FSharp.Core 6.x- and F# 7+ complier with <Nullable>enable</Nullable>
Want a new behavior, use FSharp.Core 7.0+ and F# 7+ compiler without anything

@charlesroddie
Copy link

charlesroddie commented Apr 3, 2022

This thread should be locked since there is an RFC (see the OP for link), with corresponding discussion thread and WIP implementation linked in the RFC.

@realvictorprm
Copy link
Member

realvictorprm commented Apr 3, 2022

@abelbraaksma
Copy link

abelbraaksma commented Apr 4, 2022

The follow-up discussion is now here: fsharp/fslang-design#339. And the RFC, also added to the original post above, is here, with links to a prototype.

Considering that most discussion is now in #339, it is probably best to keep it in one place (though I've no opinion on closing this thread or not).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests