Suggestion: non-nullable type #185

Closed
fdecampredon opened this Issue Jul 22, 2014 · 358 comments

Projects

None yet
@fdecampredon

Introduce two new syntax for type declaration based on JSDoc

var myString: !string = 'hello world'; //non-nullable
var myString1: ?string = 'hello world'; // nullable 
var myString2: string = 'hello world'; // nullable 
var myString3 = 'hello world'; // nullable

by default type are nullable.

Two new compiler flag :

  • inferNonNullableType make the compiler infer non-nullable type :
var myString3 = 'hello world' // typeof myString is '!string', non-nullable
  • nonNullableTypeByDefault (I guess there might be a better name) :
var myString: !string = 'hello world'; // non-nullable
var myString1: string = 'hello world'; // non-nullable 
var myString2: ?string = 'hello world'; // nullable 
var myString3 = 'hello world' // non-nullable
@danquirk danquirk added the Suggestion label Jul 22, 2014
@electricessence

I suggest using a type other than string as an example since it by nature is nullable. :P
I can perceive non-nullable types being problematic since the user and compiler of "!" expects the type to always be non-null, which can never be truly asserted in JavaScript. A user might define something like this:

function(myNonNull:!myClass):void {
  myNonNull.foo();
}

And because it's defined as non-null, everything might be happy for the compiler, but then someone else who uses it in javascript passes something null and kaboom.

Now that said, maybe the solution could be that for public facing methods, it could automatically assert not null. But then the compiler could also assert that you cannot have public properties (or private for that matter really) that can have a !nonnull declaration since they could not be enforced.

This may go deeper into the discussion of code contracts for this to be properly enforced.

@aleksey-bykov

Forgive my critics, I think there is very little need in non-nullable types if/as-soon-as algebraic data types are here. The reason people use null to represent a missing value is because there is no better way to do that in JavaScript and in most OOP languages alike. So imaging ADTs are already here. Then, as for the old libs written before non-nullables, having them won't make life any better. As for the new libs, with ADT's in place one can very accurately model what a value can take according to the business domain specification without using nulls at all. So I guess what I am saying is that ADT is way more powerful tool to address the same problem.

@samwgoldman

Personally, I just wrote a little Maybe<T> interface and use discipline to ensure that variables of that type are never null.

@fdecampredon

I suggest using a type other than string as an example since it by nature is nullable. :P
I can perceive non-nullable types being problematic since the user and compiler of "!" expects the type to always be non-null, which can never be truly asserted in JavaScript. A user might define something like this:

function(myNonNull:!myClass):void {
myNonNull.foo();
}
And because it's defined as non-null, everything might be happy for the compiler, but then someone else who uses it in javascript passes something null and kaboom.

I don't really understand you can also define :

function myFunc(str: string): int {
 return str && str.length;
}

and if someone pass an int to that function it will ends up with an error also, an advantage of typescript is to delegate to the compiler pass things that you would check manually in javascript, having another check for nullable/non-nullable type seems reasonable for me. By the way SaferTypeScript and ClosureCompiler already do that sort of check.

@fdecampredon

With union types, we could have a pretty simpler specification for that.
Let's say we have now a basic type 'null', we can have a 'stricter' mode where 'null' and 'undefined' is not compatible with any type, so if we want to express a nullable value we would do :

var myNullableString: (null | string);
var myString = "hello";
myNullableString = myString //valid
myString = myNullableString // error null is not assignable to string;

With the 'strict mode' activated typescript should check that every variable non nullable is initialized, also by default optional parameter are nullable.

var myString: string; // error
var myNullableString: (null | string); // no error

function myString(param1: string, param2?: string) {
  // param1 is string
  // param2 is (null | string)
}
@robertknight

@fdecampredon +1

IIRC from what Facebook showed of Flow which is using TypeScript syntax but with non-nullable types by default they support a shorthand for (null | T) as in your original post - I think it was ?T or T?.

@robertknight
var myString: string; // error

That could potentially be quite annoying in the case where you want to initialize a variable conditionally, eg.:

var myString: string;
if (x) {
myString = a;
} else if (y) {
myString = b;
} else {
myString = c;
}

In Rust for example, this is fine as long as the compiler can see that myString will get initialized before it is used but TypeScript's inference doesn't support this at the moment.

@fdecampredon

Honestly doing something like var myString = '' instead of var myString: string does not bother me so much, but sure that kind of rule is always possible.

@johnnyreilly

@fdecampredon +1 for this - I like the idea very much. For code bases that are 100% JavaScript this would be a useful compile-time only constraint. (As I understand your proposal there's no intention for generated code to enforce this?)

@fdecampredon

As for shorthand for (null | string) sure ?string is fine.
And sure @johnnyreilly it's only a compile time check

@samwgoldman

Sum types make non-nullable types by default a very interesting possibility. The safety properties of non-nullable by default can't be overstated. Sum types plus the planned "if/typeof destructuring" (not sure what this should be called) even make it type safe to integrate nullable and non-nullable APIs.

However, making types non-nullable by default is a huge breaking change, which would require changing almost every existing third-party type definition file. While I am 100% for the breaking change, no one person is able to update the type definitions that are out there in the wild.

It's good that a great consensus of these definitions are collected in the DefinitelyTyped repo, but I still have practical concerns about this feature.

@fdecampredon

@samwgoldman the idea is to have non-nullable types only under a special compiler flag like nonImplicitAny this flag could be named strict or nonNullableType. So there would be no breaking changes.

@samwgoldman

@fdecampredon What about the type definitions for non-TypeScript libraries, like those at DefinitelyTyped? Those definitions are not checked by the compiler, so any 3rd party code that could return null would need to be re-annotated in order to work correctly.

I can imagine a type definition for a function that is currently annotated as "returns string," but sometimes returns null. If I depended on that function in my nonNullableType'ed code, the compiler doesn't complain (how could it?) and my code is no longer null-safe.

Unless I'm missing something, I don't think this is functionality that can be turned on and off with a flag. It seems to me that this is an all-or-nothing semantic change to ensure interoperability. I would be happy to be proven wrong, though, because I think a flag-switched feature is more likely to happen.

As an aside, there isn't much information available yet on Facebook's Flow compiler, but from the video recording of the presentation, it seems like they went with non-nullable by default. If so, at least there is some precedence here.

@fdecampredon

Ok let's assume there is a shorthand ? type for type | null | undefined.

@fdecampredon What about the type definitions for non-TypeScript libraries, like those at DefinitelyTyped? Those definitions are not checked by the compiler, so any 3rd party code that could return null would need to be re-annotated in order to work correctly.

I can imagine a type definition for a function that is currently annotated as "returns string," but sometimes returns null. If I depended on that function in my nonNullableType'ed code, the compiler doesn't complain (how could it?) and my code is no longer null-safe.

I don't see the problem, sure some definition files won't be valid with the nonNullableType mode, but most of the time good library avoid to return null or undefined so the definition will still be correct with majority of the cases.
Anyway I personally rarely can pick a DefinitelyTyped definition without having to check/modify it you'll just have a little bit of extra work to add a ? prefix with some definitions.

Unless I'm missing something, I don't think this is functionality that can be turned on and off with a flag. It seems to me that this is an all-or-nothing semantic change to ensure interoperability. I would be happy to be proven wrong, though, because I think a flag-switched feature is more likely to happen.

I don't see why we could not have a flag switched feature, the rules would be simple :

  • in normal mode ? string is equivalent to string and null or undefined are assignable to all the types
  • in nonNullableType mode ? string is equivalent to string | null | undefined and null or undefined are not assignable to any other type than null or undefined

Where is the incompatibility with a flag-switched feature ?

@RyanCavanaugh
Member

Flags that change the semantics of a language are a dangerous thing. One problem is that the effects are potentially very non-local:

function fn(x: string): number;
function fn(x: number|null): string;

function foo() {
    return fn(null);
}

var x = foo(); // x: number or x: string?

It's important that someone looking at a piece of code can "follow along" with the type system and understand the inferences that are being made. If we starting having a bunch of flags that change the rules of the language, this becomes impossible.

The only safe sort of thing to do is to keep the semantics of assignability the same and change what's an error vs what isn't depending on a flag, much like how noImplicitAny works today.

@fdecampredon

I know it would break retro-compatibility, an I understand @RyanCavanaugh point of view, but after tasting that with flowtype it is honestly really an invaluable feature, I hope it will ends up being a part of typescript

@fletchsod-developer

In addition to RyanCavanaugh's comment --> From what I read somewhere, the ES7 specification / proposal mention the use of function overloading (Same function name but different input parameter datatype). That is a very sorely needed feature for Javascript.

@NoelAbrahams

From the flow docs:

Flow considers null to be a distinct value that is not part of any other type

var o = null;
print(o.x); // Error: Property cannot be accessed on possibly null value

Any type T can be made to include null (and the related value undefined) by writing ?T

var o: ?string = null;
print(o.length); // Error: Property cannot be accessed on possibly null or undefined value

[Flow] understands the effects of some dynamic type tests

(i.e. in TS lingo understands type guards)

var o: ?string = null;
if (o == null) {
  o = 'hello';
}
print(o.length); // Okay, because of the null check

Limitations

  • Checks on object properties are limited because of the possibility of aliasing:

    In addition to being able to adjust types of local variables, Flow can sometimes also adjust types of object properties, especially when there are no intermediate operations between a check and a use. In general, though, aliasing of objects limits the scope of this form of reasoning, since a check on an object property may be invalidated by a write to that property through an alias, and it is difficult for a static analysis to track aliases precisely

  • Type guard-style checks can be redundant for object properties.

[D]on't expect a nullable field to be recognized as non-null in some method because a null check is performed in some other method in your code, even when it is clear to you that the null check is sufficient for safety at run time (say, because you know that calls to the former method always follow calls to the latter method).

  • undefined is not checked.

Undefined values, just like null, can cause issues too. Unfortunately, undefined values are ubiquitous in JavaScript and it is hard to avoid them without severely affecting the usability of the language. For example, arrays can have holes for elements; object properties can be dynamically added and removed. Flow makes a tradeoff in this case: it detects undefined local variables and return values, but ignores the possibility of undefined resulting from object property and array element accesses

@spion
spion commented Nov 25, 2014

What if the option is added at the same time when introducing the null type (and the questionmark shorthand)? The presence of a null type in a file would force the compiler into non-nullable mode for that file even if the flag is not present at the command line. Or is that a bit too magical?

@fdecampredon

@jbondc seems good. however the problem with that is that it will ends up with ! everywhere :p

@misfo
misfo commented Dec 10, 2014
It's tempting to want to change JavaScript but the reality is a 'string' is nullable or can be undefined.

What does this mean? There are no static types in js. So, yes, strings are "nullable", but let's not forget that they are also numberable and objectable and fooable, etc. Any value can have any type.

So when layering a static type system on top of javascript, choosing whether static types are nullable or not is just a design decision. It seems to me non-nullable types are a better default, because it's usually only in special cases that you want a function signature, for instance, to accept a null value in addition to the type specified.

@metaweta

Directives like "use strict" that cause scoped changes to semantics are already a part of the language; I think it would be reasonable to have a "use nonnullable types" directive in TypeScript.

@fdecampredon

@metaweta I don't think it's enough, for example what happens if a non null module consume a nullable one :

//module A
export function getData(): string[] {
  return null;
}
//module B
'use nonnull'
import A = require('./A');

var data: string[] = A.getData();

data in module B is in fact nullable, but since 'use nonnull' was not used in module A should we report an error ?
I don't see a way to solve that problem with directive based feature.

@metaweta

Yes,

var data: string[] = A.getData();

would cause an error. Instead, you'd have to provide a default value for when getData() returns null:

var data: string[] = A.getData() || [];
@fdecampredon

@metaweta ok but how do you know that it's an error ? :)
type of getData is still '() => string[]' would you automaticly treat everything that comes from a 'nullable module' as 'nullable ' ?

@metaweta

Yes, exactly (unless a type from the nullable module is explicitly marked otherwise).

@Griffork

That sounds like you now want a per file flag that dictates whether of not that file defaults to nullable or not.

Personally I think it's a bit late to introduce this change, and @RyanCavanaugh is right, the change would make Typescript less predictable as you would not be able to determine what was going on just by looking at a file.

Do projects start with this compiler flag on or off by default? If someone is working on a no default nullable project and create / switch to a default nullable one will that cause confusion?
I currently work with No Implicit Any in most of my projects, and whenever I come across a project that doesn't have that option turned on it takes me by surprise.

The no impicit any is good, but in terms of flags that change the way the language behaves, I think that should be the line. Any more than that and people who are working on multiple projects started by different people with different rules are going to lose a lot of productivity due to false assumptions and slip ups.

@metaweta

@RyanCavanaugh was concerned about non-locality, and directives are lexically scoped. You can't get any more local unless you annotate each site. I'm not particularly in favor of the directive; I was just pointing out that the option exists and that it's at least as reasonable as "use strict" in ES5. I'm personally in favor of non-nullable types by default, but practically, it's too late for that. Given those constraints, I'm in favor of using ! somehow. @jbondc 's proposal lets you distinguish null from undefined; given that Java backends continue to make people use both values, it seems the most useful to me.

@Griffork

I'm sorry if I wasn't clear, I was both agreeing with Ryan and adding my own concerns.

@fdecampredon

Honestly if adding use not-null is the price for avoiding all the null pointer exception I would pay it without any problem, considering null or undefined as assignable to any type is the worse error that typescript made in my opinion.

@Griffork

@jbondc I have not used 'use strict' and am therefore making some assumptions, please correct me if my assumptions are wrong:

Not null does not affect the syntax that the programmer writes, but the capabilities of the next programmer that tries to use that code (assuming that creator and user are separate people).

So the code:

function myfoo (mynumber: number) {
    return !!mynumber;
} 

(typing on a phone so may be wrong)
Is valid code in both a normal project and a notnull project. The only way that the coder would know whether or not the code is working is if they look at the command line arguments.

At work we have a testing project (which includes prototyping new features) and a main project (with our actual code). When the prototypes are ready to be moved from one project to another (typically with large refractors), there would be no errors in the code, but errors in the use of the code. This behaviour is different to no implicit any and use strict which both would error immediately.

Now I have a fair amount of sway in these projects, so I can warn the people in charge to not use this new 'feature' because it wouldn't save time, but I don't have that capacity over all of the projects at work.
If we want to enable this feature in even one project, then we have to enable it in all of our other projects because we have a very significant amount of code sharing and code migration between projects, and this 'feature' would cause us a lot of time going back and 'fixing' functions that were already finished.

@metaweta

Right @jbondc. @Griffork: Sorry I didn't catch that misunderstanding; a directive is a literal string expression that appears as the first line of a program production or function production and its effects are scoped to that production.

"use not-null";
// All types in this program production (essentially a single file) are not null

versus

function f(n: number) {
  "use not-null";
  // n is not null and local variables are not null
  function g(s: string) {
    // s is not null because g is defined in the scope of f
    return s.length;
  }
  return n.toFixed(2);
}

function h(n: number) {
  // n may be null
  if (n) { return n.toFixed(3); }
  else { return null; }
}
@aleksey-bykov

Non-nullable types are useless. Non-nullable types are useles. They are useless. Useless! You don't realize it, but you don't really need them. There is very little sense in restricting yourself by proclaiming that from now on we are not going to be using NULL's. How would you represent a missing value, for example in a situation when you are trying to find a substring that is not there? Not being able to express a missing value (what NULL does now for now) isn't going to solve your problem. You will trade a harsh world with NULL's everywhere for equally harsh one with no missing values at all. What you really need is called algebraic data types that (among many other cool things) feature the ability to represent a missing value (what you are looking for in the first place and what is represented by NULL in imperative world). I am strongly against adding non-nullables to the language, because it looks like useless syntactic/semantic trash that is a naive and awkward solution to a well-known problem. Please read about Optionals in F# and Maybe in Haskell as well as variants (aka tagged unions, discriminated unions) and pattern matching.

@metaweta

@aleksey-bykov It sounds like you're unaware that JavaScript has two nullish values, undefined and null. The null value in JavaScript is only returned on a non-matching regexp and when serializing a date in JSON. The only reason it's in the language at all was for interaction with Java applets. Variables that have been declared but not initialized are undefined, not null. Missing properties on an object return undefined, not null. If you explicitly want to have undefined be a valid value, then you can test propName in obj. If you want to check whether a property exists on the object itself rather than if it's inherited, use obj.hasOwnProperty(propName). Missing substrings return -1: 'abc'.indexOf('d') === -1.

In Haskell, Maybe is useful precisely because there's no universal subtype. Haskell's bottom type represents non-termination, not a universal subtype. I agree that algebraic data types are needed, but if I want a tree labeled by integers, I want every node to have an integer, not null or undefined. If I want those, I'll use a tree labeled by Maybe int or a zipper.

If we adopt a "use not-null" directive, I'd also like "use not-void" (neither null nor undefined).

@aleksey-bykov

If you want to guarantee your own code from nulls just prohibit the null
literals. It's way easier than developing non-nullable types. Undefined is a
little bit more complicated, but if you know what they are coming from then
you know how to avoid them. Bottom in Haskell is invaluable! I wish
JavaScript (TypeScript) had a global super type without a value. I miss it
badly when I need to throw in an expression. I've been using TypeScript
since v 0.8 and never used nulls let alone had a need for them. Just ignore
them like you do with any other useless language feature like with
statement.

@metaweta

@aleksey-bykov If I'm writing a library and want to guarantee that inputs are not null, I have to do runtime tests for it everywhere. I want compile-time tests for it, it's not hard to provide, and both Closure and Flow provide support for non-null/undefined types.

@aleksey-bykov

@metaweta, you cannot guarantee yourself from nulls. Before your code is compiled there is a gazillion ways to make your lib cry: pleaseNonNullablesNumbersOnly(<any> null). After compiled to js there are no rules at all. Secondly, why would you care? Say it loud and clear upfront nulls are not supported, you put a null you will get a crash, like a disclaimer, you cannot guarantee yourself from all sort of people out there, but you can outline of your scope of responsibilities. Thirdly, I can hardly think of a major mainstream lib that is bulletproof to whatever user may put as input, yet it is still crazy popular. So is your effort worth troubles?

@metaweta

@aleksey-bykov If my library's clients are also type-checked, then I certainly can guarantee I won't get a null. That's the whole point of TypeScript. By your reasoning, there's no need for types at all: just "say loud and clear" in your documentation what the expected type is.

@Griffork

Off topic, nulls are extremely valuable for us because checking them is faster than checking against undefined.
While we don't use them everywhere, we try to use them where possible to represent uninitialised values and missing numbers.

On topic:
We've never had an issue with nulls 'escaping' into other code, but we have had issues with random undefinedes or NaNs appearing. I believe that careful code management is better than a flag in this scenario.

However, for library typings it would be nice to have the redundant type null so that we can choose to annotate functions that can return null (this should not be enforced by the compiler, but by coding practices).

@aleksey-bykov

@metaweta, by my reasoning your clients should not use nulls in their code base, it's not that hard, do a full search for null (case sensitive, whole word) and delete all of them. Too clunky? Add a compiler switch --noNullLiteral for fanciness. Everything else stays intact, same code, no worries, way lighter solution with minimal footprint. Back to my point, suppose your non-nullable types found their way to TS and avialable in 2 different flavors:

  • one can use ! syntax to denote a type that cannot take a null, for example string! cannot take a null
  • noNullsAllowed switch is on

then you get a piece of json from your server over ajax with nulls everywhere, moral: the dynamic nature of javascript cannot be fixed by a type annotation on top of it

@metaweta

@aleksey-bykov By the same token, if I'm expecting an object with a numeric property x and I get {"x":"foo"} from the server, the type system won't be able to prevent it. That's necessarily a runtime error and an inescapable problem when using something other than TypeScript on the server.

If, however, the server is written in TypeScript and running on node, then it can be transpiled in the presence of a .d.ts file for my front end code and the type checking will guarantee that the server will never send JSON with nulls in it or an object whose x property is not a number.

@aleksey-bykov

@metaweta, non-nullable types sure would be another type safety measure, I am no questioning that, I am saying that by imposing some very basic discipline (avoiding null literals in your code) you can eliminate 90% of your problems without asking for any help from the compiler. Well, even if you have enough resources to enforce this measure, then you still won't be able to eliminate the rest 10% of the problems. So what is the question after all? I ask: do we really need it that bad? I don't, I learned how to live without nulls successfully (ask me how), I don't remember when I got a null reference exception last time (besides in data from our server). There are way cooler stuff I wish we had. This particular one is so insignificant.

@spion
spion commented Dec 19, 2014

Yes we do need this badly. See The billion dollar mistake by Hoare. The null pointer exception (NPE) is the most common error encountered in typed programming languages that don't discriminate nullable from non-nullable types. Its so common that Java 8 added Optional in a desperate attempt to battle it.

Modelling nullables in the type system is not just a theoretical concern, its a huge improvement. Even if you take great care to avoid nulls in your code, the libraries you use might not and therefore its useful to be able to model their data properly with the type system.

Now that there are unions and type guards in TS, the type system is powerful enough to do this. The question is whether it can be done in a backward-compatible way. Personally I feel that this feature is important enough for TypeScript 2.0 to be backwards-incompatible in this regard.

Implementing this feature properly is likely to point to code that is already broken rather than break existing code: it will simply point to the functions that leak nulls outside them (most likely unintentionally) or classes that don't properly initialize their members (this part is harder as the type system may need to make allowance for member values to be initialized in the constructors).

This is not about not using nulls. Its about properly modelling all the types involved. Infact this feature would allow the use of nulls in a safe way - there would be no reason to avoid them anymore! The end result would be very similar to pattern matching on an algebraic Maybe type (except it would be done with an if check rather than a case expression)

And this isn't just about null literals. null and undefined are structurally the same (afaik there are no functions/operators that work on one but not the other) therefore they could be modelled sufficiently well with a single null type in TS.

@NoelAbrahams

@metaweta,

The null value in JavaScript is only returned on a non-matching regexp and when serializing a date in JSON.

Not true at all.

  • Interaction with the DOM produces null:
  console.log(window.document.getElementById('nonExistentElement')); // null
  • As @aleksey-bykov pointed out above, ajax operations can return null. In fact undefined is not a valid JSON value:
 JSON.parse(undefined); // error
 JSON.parse(null); // okay
 JSON.stringify({ "foo" : undefined}); // "{}"
 JSON.stringify({ "foo" : null}); // '{"foo":null}'

NB: We can pretend that undefined is returned via ajax, because accessing a non-existent property will result in undefined - which is why undefined is not serialised.

If, however, the server is written in TypeScript and running on node, then it can be transpiled in the presence of a .d.ts file for my front end code and the type checking will guarantee that the server will never send JSON with nulls in it or an object whose x property is not a number.

This is not entirely correct. Even if the server is written in TypeScipt, one can in no way guarantee nulls from being introduced without checking every single property of every single object obtained from persistent storage.

I kind of agree with @aleksey-bykov on this. While it would be absolutely brilliant if we can have TypeScript alert us at compile time about errors introduced by null and undefined, I fear it will only induce a false sense of confidence and end up catching trivia while the real sources of null go undetected.

@Arnavion
Contributor

Even if the server is written in TypeScipt, one can in no way guarantee nulls from being introduced without checking every single property of every single object obtained from persistent storage.

This is in fact an argument for non-nullable types. If your storage can return null Foo's, then the type of the object retrieved from that storage is Nullable<Foo>, not Foo. If you then have a function that returns that is meant to return Foo, then you have to take responsibility by handling the null (either you cast it because you know better or you check for null).

If you didn't have non-nullable types you would not necessarily think to check for null when returning the stored object.

I fear it will only induce a false sense of confidence and end up catching trivia while the real sources of null go undetected.

What sort of non-trivia do you think non-nullable types will miss?

@spion
spion commented Dec 19, 2014

This is not entirely correct. Even if the server is written in TypeScipt, one can in no way guarantee nulls from being introduced without checking every single property of every single object obtained from persistent storage.

If the persistent storage supports typed data, then there would be no need. But even if that weren't the case, you'd have checks only at the data fetching points and then have a guarantee throughout all of your other code.

I kind of agree with @aleksey-bykov on this. While it would be absolutely brilliant if we can have TypeScript alert us at compile time about errors introduced by null and undefined, I fear it will only induce a false sense of confidence and end up catching trivia while the real sources of null go undetected.

Using nullable types wouldn't be an absolute requirement. If you feel that its unnecessary to model the cases where a method returns null as they're "insignificant", you could just not use a nullable type in that type definition (and get the same unsafety as always). But there is no reason to think that this approach will fail - there are examples of languages that have successfully implemented it already (e.g. Kotlin by JetBrains)

@fdecampredon

@aleksey-bykov Honestly you got it completely wrong, one of the best thing about non-nullable types is the possibility to express a type as nullable.
With your strategy of never using null to prevent null pointer error you completely loose the possibility of using null out of the fear of introducing error, that's completely silly.

Another thing please in a discussion about a language feature don't go with stupid comments like :

Non-nullable types are useless. Non-nullable types are useles. They are useless. Useless! You don't realize it, but you don't really need them.

That just make me feel like I should ignore whatever you will ever post anywhere on the web, we are here to discuss about a feature, I can understand and gladly accept that your point of view is not mine, but don't behave like a kid.

@NoelAbrahams

I am not against introducing non-null type annotation at all. It has been shown to be useful in C# and other languages.

The OP has changed the course of the discussion with the following:

Honestly if adding use not-null is the price for avoiding all the null pointer exception I would pay it without any problem, considering null or undefined as assignable to any type is the worse error that typescript made in my opinion.

I was merely pointing out the prevalence of null and undefined in the wild.

@NoelAbrahams

I should also add that one of the things that I truly appreciate about TypeScript is the laissez-faire attitude of the language. It has been a breath of fresh air.

Insisting that types are non-nullable by default goes against the grain of that spirit.

@Griffork

I've been seeing a number of arguments floating around as to why we do / don't need this and I want to see if I understand all of the underlying cases that have been discussed such far:

  1. want to know whether or not a function can return null (caused by it's execution pattern, not it's typing).
  2. want to know if a value can be null.
  3. want to know if a data object can contain null values.

Now, there are only two cases in which situation 2 can occur: of you're using nulls or if a function returns a null. If you eliminate all nulls from your code (assuming that you don't want them) then really situation 2 can only occur as a result of situation 1.

Situation 1 I think is best solved by annotating the function's return type to show presence of a null value. This does not mean that you need a non null type. You can annotate the function (for example by using union types) and not have non null types, it's just like documentation, but probably clearer in this case.

Solution 2 is also solved by this.

This allows programmers working at their company to use processes and standards to enforce that null types are marked up, and not the Typescript team (exactly the same way that the whole typing system is an opt-in so would the explicit nullable types be an opt in).

As for scenario 3, the contract between the server and the client is not for Typescript to enforce, being able to mark-up the affected values as possibly null might be an improvement, but eventually you'll get the same garentee from that as typescript's tooling gives you on every other value (which is to say, none unless you have good coding standards or practices!

(posting from phone, sorry for errors)

@aleksey-bykov

@fdecampredon, it's not the fear in the first place, it's that using null is unnecessary. I don't need them. As a nice bonus I got a problem of null reference exceptions eliminated. How is it all possible? By employing a sum-type with an empty case. Sum-types are a native feature of all FP languages like F#, Scala, Haskell and together with product types called algebraic data types. Standard examples of sum types with an empty case would be Optional from F# and Maybe from Haskell. TypeScript doesn't have ADT's, but instead it has a discussion in progress about adding non-nullables that would model one special case of what ADT's would have covered. So my message is, ditch the non-nullables for ADT's.

@spion, Bad news: F# got nulls (as legacy of .NET). Good news: no one uses them. How can you not use null when it's there? They have Optionals (just like the most recent Java as you mentioned). So you don't need null if you have a better choice at your disposal. This is what I am ultimately suggesting: leave nulls (non-nulls) alone, just forget they exist, and implement ADT's as a language feature.

@fletchsod-developer

We use null but not the same way you people use. My company source code comes in 2 parts.

  1. When it comes to data (like database data), we replace null with blank data at the time of variable declaration.

  2. All others, like programmable objects, we use null so we know when there's a bug and loophole in source code where objects aren't created or assignable that aren't necessary javascript objects. The undefined is javascript object issues where there's a bug or loophole in the source code.

The data we don't want to be nullable cuz it is customer data and they'll see null wordings.

@fdecampredon

@aleksey-bykov typescript has adt with union type, the only thing missing is pattern matching but that's a feature that just cannot be implemented with the typescript philosophy of generating javascript close to the original source.

On the other end it's impossible to define whith union type the exclusion of null value, that's why we need non-null type.

@aleksey-bykov

@fdecampredon, TS doesn't have ADT's, it has unions which are not sum types, because, as you said, 1. they cannot model an empty case correctly since there is no unit type, 2. there is no reliable way to destructure them

pattern matching for ADT's can be implemented in a way that is aligned closely with generated JavaScript, anyway I hope that this argument isn't a turning point

@spion
spion commented Dec 19, 2014

@aleksey-bykov this is not F#. Its a language that aims to model the types of JavaScript. JavaScript libraries use null and undefined values. Therefore, those values should be modeled with the appropriate types. Since not even ES6 supports algebraic data types, it doesn't make sense to use that solution given TypeScript's design goals

Additionally, JavaScript programmers typically use if checks (in conjuction with typeof and equality tests), instead of pattern matching. These can already narrow TypeScript union types. From this point its only a tiny step to support non-nullable types with benefits comparable to algebraic Maybe etc.

I'm surprised that nobody actually mentioned the huge changes that lib.d.ts may need to introduce and potential problems of the transient null state of class fields during constructor invocation. Those are some real, actual potential issues to implement non-nullable types...

@Griffork the idea is to avoid having null checks everywhere in your code. Say you have the following function

declare function getName(personId:number):string|null;

The idea is that you check whether the name is null only once, and execute all the rest of the code free from worries that you have to add null checks.

function doSomethingWithPersonsName(personId:number) {
  var name = getName(personId);
  if (name != null) return doThingsWith(name); // type guard narrows string|null to just string
  else { return handleNullCase(); }
}

And now you're great! The type system guarantees that doThingsWith will be called with a name that is not null

function doThingsWith(name:string) {
  // Lets create some funny versions of the name
  return [uppercasedName(name), fullyLowercased(name), funnyCased(name)]
}

None of these functions need to check for a null, and the code will still work without throwing. And, as soon as you try to pass a nullable string to one of these functions, the type system will tell you immediately that you've made an error:

function justUppercased(personId:number) {
  var name = getName(personId);
  return uppercasedName(name); // error, passing nullable to a function that doesn't check for nulls.
}

This is a huge benefit: now the type system tracks whether functions can handle nullable values or not, and furthermore, it tracks whether they actually need to or not. Much cleaner code, less checks, more safety. And this is not just about strings - with a library like runtime-type-checks you could also build type guards for much more complex data

And if you don't like the tracking because you feel that its not worth modeling the possibility of a null value, you can revert back to the good old unsafe behavior:

declare function getName(personId:number):string;

and in those cases, typescript will only warn you if you do something that is obviously wrong

uppercasedName(null);

I frankly don't see downsides, except for backward-compatibility.

@aleksey-bykov

@jbondc there is one #186

@metaweta

@fdecampredon Union types are just that, unions. They are not disjoint unions a.k.a. sums. See #186.

@Arnavion
Contributor

@aleksey-bykov

Note that adding an option type is still going to be a breaking change.

// lib.d.ts
interface Document {
    getElementById(id: string): Maybe<Element>;
}

...

// Code that worked with 1.3
var myCanvas = <HTMLCanvasElement>document.getElementById("myCanvas");
// ... now throws the error that Maybe<Element> can't be cast to an <HTMLCanvasElement>

After all, you can get a poor man's option types right now with destructuring

class Option<T> {
    hasValue: boolean;
    value: T;
}

var { hasValue, myCanvas: value } = <Option<HTMLCanvasElement>> $("myCanvas");
if (!hasValue) {
    throw new Error("Canvas not found");
}
// Use myCanvas here

but the only value from this is if lib.d.ts (and any other .d.ts, and your whole codebase but we'll assume we can fix that) also use it, otherwise you're back to not knowing whether a function that doesn't use Option can return null or not unless you look at its code.

Note that I also am not in favor of types being non-null by default (not for TS 1.x anyway). It is too big a breaking change.

But let's say we're talking about 2.0. If we're going to have a breaking change anyway (adding option types), why not make types non-nullable by default as well? Making types non-nullable by default and adding option types is not exclusive. The latter can be standalone (eg. in F# as you point out) but the former requires the latter.

@aleksey-bykov

@Arnavion, there is some misunderstanding, I didn't say we need to replace the signatures, all existing signatures stay intact, it's for the new developments you are free to go either with ADT or whatever else you want. So no breaking changes. Nothing is being made non-null by default.

If ATDs are here, it's up to a developer to wrap all places where nulls can leak into the application by transforming then into optionals. This can be an idea for a standalone project.

@Arnavion
Contributor

@aleksey-bykov

I didn't say we need to replace the signatures, all existing signatures stay intact

I said that the signatures need to be replaced, and I gave the reason already:

but the only value from this is if lib.d.ts (and any other .d.ts, and your whole codebase but we'll assume we can fix that) also use it, otherwise you're back to not knowing whether a function that doesn't use Option can return null or not unless you look at its code.


Nothing is being made non-null by default. Ever.

For TS 1.x, I agree, only because it is too big a breaking change. For 2.0, using option type in the default signatures (lib.d.ts etc.) would already be a breaking change. Making types non-nullable by default in addition to that becomes worth it and carries no downsides.

@aleksey-bykov

I disagree, introducing optionals should not break anything, it's not like we either use optionals or nullables or non-nullables. Everyone uses whatever they want. Old way of doing things should not depend on new features. It's up to a developer to use an appropriate tool for his immediate needs.

@Arnavion
Contributor

So you're saying that if I have a function foo that returns Option<number> and a function bar that returns number, I'm not allowed to be confident that bar cannot return null unless I look at the implementation of bar or maintain documentation "This function never returns null."? Don't you think this punishes functions which will never return null?

@aleksey-bykov

function bar from your example was known as nullable from the begging of time, it was used in some 100500 applications all around and everyone treated the result as nullable, now you came around and looked inside of it and discovered that null is impossible, does it mean that you should go ahead and change the signature from nullable to non-nullable? I think you should not. Because this knowledge, although valuable, isn't worth braking 100500 applications. What you should do is to come up with a new lib with revised signatures that does it like this:

old_lib.d.ts

...
declare function bar(): number; // looks like can return a potentially nullable number
...

revised_lib.d.ts

declare function bar(): !number; // now thank to the knowledge we are 100% certain it cannot return null

now the legacy apps keep using the old_lib.d.ts, for new apps a developer is free to choose revised_libs.d.ts

@Arnavion
Contributor

Unfortunately revised_libs.d.ts has 50 other functions that I haven't looked at yet, all of which return number (but I don't know whether it's really number or nullable number). What now?

@aleksey-bykov

Well, take your time, ask for help, use versioning (depending on the level of knowledge you've gained so far you may want to release it graduately with ever increasing version number: revised_lib.v-0.1.12.d.ts)

@Arnavion
Contributor

It's not necessary actually. A function that returns nullable type in the signature but non-nullable type in the implementation only results in redundant error checking by the caller. It doesn't compromise safety. Gradually annotating more and more functions with ! as you discover them will work just fine, as you said.

I'm not a fan of ! only because it's more baggage to type (both in terms of keystrokes and needing to remember to use it). If we want non-nullability in 1.x then ! is one of the options already discussed above, but I would still say that having an eventual breaking change with 2.0 and making non-nullability the default is worth it.

On the other hand, maybe it'll lead to a Python 2/3-esque situation, where nobody upgrades to TS 2.0 for years because they can't afford to go through their million-line codebase making sure that every variable declaration and class member and function parameter and... is annotated with ? if it can be null. Even 2to3 (the Python 2 to 3 migration tool) doesn't have to deal with wide-ranging changes like that.

Whether 2.0 can afford to be a breaking change depends on the TS team. I would vote for yes, but then I don't have a million-line codebase that will need fixing for it, so maybe my vote doesn't count.

@Arnavion
Contributor

Perhaps we should ask the Funscript folks how they reconcile DOM API returning nulls with F# (Funscript uses TypeScript's lib.d.ts and other .d.ts for use from F# code). I've never used it, but looking at http://funscript.info/samples/canvas/index.html for example it seems the type provider does not think that document.getElementsByTagName("canvas")[0] can ever be undefined.

Edit: Here it seems document.getElementById() is not expected to return null. At the very least it doesn't seem to be returning Option<Element> seeing it is accessing .onlick on the result.

@Griffork

@spion
Thanks, I hadn't thought of that.

At work our codebase is not small, and this breaking change that people want would set us back a lot of time with very little gain. Through good standards and clear communication between our developers we've not had problems with nulls appearing where they shouldn't.
I am honestly surprised that some people are pushing for it so badly.
Needless to say this would give us very little benefit and would cost us a lot of time.

@spion
spion commented Dec 20, 2014

@Griffork have look at this filtered list of issues in the typescript compiler for an estimate on how big of a benefit this change could make. All of the "crash" bugs listed at that link could be avoided by using non-nullable types. And we're talking about the awesome Microsoft level of standards, communication and code-review here.

Regarding breakage, I think that if you continue using existing type definitions its possible that you wont get any errors at all, except the compiler pointing out potentially uninitialized variables and fields leaking out. On the other hand, you might get a lot of errors, particularly e.g. if class fields are often left uninitialized in constructors in your code (to be initialized later). Therefore I understand your concerns, and I'm not pushing for a backward-incompatible change for TS 1.x. I still hope that I've managed to persuade you that if any change to the language was worthy of breaking backward compatibility, its this one.

In any case, Facebook's Flow does have non-nullable types. Once its more mature, it might be worth investigating as a replacement of TS for those of us who care about this issue.

@aleksey-bykov

@spion, the only number your list gives us is how many times null or undefined were mentioned for any reason out there, basically it only says that null and undefined have been talked about, I hope it's clear that you cannot use it as an argument

@spion
spion commented Dec 20, 2014

@aleksey-bykov I am not - I looked at all the issues on that filtered list and every single one that had the word "crash" in it was related to a stack trace which shows that a function attempted to access a property or a method of an undefined or null value.

I tried to narrow the filter with different keywords and I (think I) managed to get all of them

@Griffork

@spion
Question: how many of those errors are caused in locations that they would need to mark the variable as nullable or undefinable though?

E. G. If an object can have a parent, and you initialise parent to null, and you're always going to have one object with a parent that is null, you will still have to declare the parent as possibly null.
The problem here is if a programmer writes some code with the assumption that a loop will always break before it reaches the null parent. That's not a problem with null being in the language, the exact same thing would happen with undefined.

Reasons for keeping null as easy to use as it is now:
β€’ It's a better default value than undefined.
. 1) it's faster to check in some cases (our code must be very performant)
. 2) it makes for in loops work more predictably on objects.
. 3) it makes array usage make more sense when using nulls for blank values (as opposed to missing values). Note that delete array[i] and array[i] = undefined have different behaviour when using indexOf (and probably other popular array methods).

What I feel the result of making nulls require extra mark up to use in the language:
β€’ I got an undefined error instead of a null error (which is what would happen in most of the typescript scenarios).

When I said we don't have a problem with nulls escaping, I meant that variables that are not initialised to null never become null, we still get null exception errors in exactly the same place we would get undefined errors (as does the Typescript team). Making null harder to use (by requiring extra syntax) and leaving undefined the same will actually cause more problems to some developers (e. g. us).

Adding extra syntax to use null means that for several weeks / months developers who use a lot of null will be making errors left, right and center while they try to remember the new syntax. And it will be another way to slip up in the future (by annotating something slightly incorrectly). [Time to point out that I hate the idea of using symbols to represent types, it makes the language less clear]

Until you can explain to me a situation in which null causes an error or problem that undefined would not, then I won't agree with making null significantly harder to use than undefined. It has it's use case, and just because it's not helping you, doesn't mean that the breaking change that you want (that will hurt the workflow of other developers) should go ahead.

Conclusion:
There is no point in being able to declare non-nullable types without being able to define non-undefined types. And non-undefined types are not possible due to the way javascript works.

@spion
spion commented Dec 21, 2014

@Griffork when I say non-nullable, I mean non-nullable-or-undefined. And its not true that its not possible due to the way JS works. With the new type guard features, once you use a type guard you know that the value flowing from there cannot be null or undefined anymore. There is a tradeoff involved there too, and I submit Facebook Flow's implementation as proof that its quite doable.

The only case that will become slightly harder to use is this one: I will temporarily assign null to this variable then I will use it in this other method as if it not null, but I know that I'll never call this other method before initializing the variable first, so there is no need to check for nulls. This is very brittle code, and I would definitely welcome the compiler warning me about it: its a refactor away from being a bug anyway.

@Griffork

@spion
I do believe that I'm finally understanding where you're coming from.

I believe that you're wanting type guards to help determine when a value cannot be null, and allow you to call functions that don't check for null within. And if the guarding if-statement is removed, then that becomes an error.

I can see that being useful.

I also believe that this isn't really going to be the silver bullet you're hoping for.

A compiler that does not run your code and test every facet of it is not going to be better at determining where undefineds/nulls are than the programmer who wrote the code. I am concerned that this change would lull people into a false sense of security, and actually make null/undefined errors more difficult to track when they do occur.
Really, I think the solution that you need is a good set of testing tools that support Typescript, that can reproduce these sort of bugs in Javascript, rather than implementing a type in a compiler that can not deliver on it's promise.

You mention Flow as having a solution to this problem, but when reading your link I saw some concerning things:

"Flow can sometimes also adjust types of object properties, especially when there are no intermediate operations between a check and a use. In general, though, aliasing of objects limits the scope of this form of reasoning, since a check on an object property may be invalidated by a write to that property through an alias, and it is difficult for a static analysis to track aliases precisely."

"Undefined values, just like null, can cause issues too. Unfortunately, undefined values are ubiquitous in JavaScript and it is hard to avoid them without severely affecting the usability of the language[...] Flow makes a tradeoff in this case: it detects undefined local variables and return values, but ignores the possibility of undefined resulting from object property and array element accesses."

Now undefined and null work differently, undefined errors can still show up everywhere, the question mark cannot guarantee that the value is not null, and the language behaves more differently to Javascript (which is what TS is trying to avoid from what I've seen).

p.s.

foo(thing: whatiknowaboutmyobject) {
    if (thing.hidden) {
        delete thing.description;
    }
}

if (typeof thing.description === "string") {
    //thing.description is non-nullable now, right?
    foo(thing);
    //What is thing.description?
    console.log(thing.description.length);
}
@Arnavion
Contributor

TS is already vulnerable to aliasing effects (as is any language that allows mutable values). This will compile without any errors:

function foo(obj: { bar: string|number }) {
    obj.bar = 5;
}

var baz: { bar: string } = { bar: "5" };

foo(baz);

console.log(baz.bar.charAt(0)); // Runtime error - Number doesn't have a charAt method

The Flow docs are stating this only for completeness.

Edit: Better example.

@Griffork

Old:

Yes, but I argue that the means of setting something to undefined is much greater than the means of setting something to another value.

I mean,

mything.mystring = 5; // is clearly wrong.
delete mything.mystring; //is not clearly wrong - this is not quite the equivalent of setting mystring to >undefined.

Edit:
Meh, at this point it's pretty much personal preference. After using javascript for ages, I do not think this suggestion is going to help the language. I think it's going to lull people into a false sense of security, and I think that it will drive Typescript (as a language) away from Javascript.

@spion
spion commented Dec 21, 2014

@Griffork For an example of how the current typescript luls you into a false sense of security, try the example you presented in the playground:

var mything = {mystring: "5"}; 
delete mything.mystring;
console.log(mything.mystring.charAt(1));

By the way, the delete operator could be treated the same way as assigning a value of type null and that would be sufficient to cover your case too.

The claim that the language will behave differently than JavaScript is true, but meaningless. TypeScript already has behavior different than JavaScript. The point of a type system has always been to disallow programs that don't make sense. Modelling non-nullable types simply adds a couple of extra restrictions. Disallowing the assignment of null or undefined values to a variable of non-nullable type is precisely the same as disallowing the assignment of a number to a variable of type string. JS allows both, TS could allow neither

@NoelAbrahams

@spion

the idea is to avoid having null checks everywhere in your code

If I understand what you are advocating:

A. Make all types non-null by default.
B. Mark fields and variables that are nullable.
C. Ensure the application/library developer checks all entry points into the application.

But doesn't that mean the onus for ensuring one's code is free of nulls is on the person writing the code, and not on the compiler? We are effectively telling the compiler "dw, I'm not letting any nulls into the system."

The alternative is to say, nulls are everywhere, so don't bother, but if something is non-nullable then I'll let you know.

The fact is the latter approach is prone to null reference exceptions, but it's more truthful. Pretending that a field on an object obtained over the wire (i.e. ajax) is non-null implies having faith in God πŸ˜ƒ .

I believe there is strong disagreement on this issue because, depending on what one is working on, item C above could either be trivial or infeasible.

@spion
spion commented Dec 21, 2014

@jbondc I'm glad you asked that. Indeed CallExpression is marked as undefined or nullable. However, the type system currently does not take any advantage of that - it still allows all the operations on typeArguments as if it isn't null or undefined.

However, when using the new union types in combination with non-nullable types, the type could be expressed as NodeArray<TypeNode>|null. Then the type system will not allow any operations on that field unless a null check is applied:

if (ce.typeArguments != null) {
  callSomethingOn(ce.typeArguments)
}

// callSomethingOn doesn't need to perform any checks

function callSomethingOn(na:NodeArray<TypeNode>) {
...
}

With the help of TS 1.4 type guards, inside the if block the type of the expression will be narrowed to NodeArray<TypeNode> which in turn will allow all NodeArray operations on that type; additionally, all functions called within that check will be able to specify their argument type to be NodeArray<TypeNode> without performing any more checks, ever.

But if you try to write

function someOtherFunction(ce: CallExpression) {
  callSomethingOn(ce.typeArguments)
}

the compiler will warn you about it at compile time, and the bug simply wouldnt've happened.

So no, @NoelAbrahams, this is not about knowing everything for certain. Its about the compiler helping you tell what value a variable or field can contain, just like with all other types.

Of course, with external values, as always, its up to you to specify what their types are. You can always say that external data contains a string instead of a number, and the compiler wont complain yet your program will crash when trying to do string operations on the number.

But without non-nullable types, you don't even have the ability to say that a value can't be null. The null value is assignable to each and every type and you can't make any restrictions about it. Variables can be left uninitialized and you wont get any warnings since undefined is a valid value for any type. Therefore, the compiler is unable to help you catch null and undefined-related errors at compile time.

I find it surprising that there are so many misconceptions about non-nullable types. They're just types that can't be left uninitialized or can't be assigned the values null or undefined. This isn't that dissimilar to being unable to assign a string to a number. This is my last post on this issue, as I feel that I'm not really getting anywhere. If anyone is interested in finding out more, I recommend starting with the video "The billion dollar mistake" I mentioned above. The issue is well known and tackled by many modern languages and compilers successfully.

@NoelAbrahams

@spion, I do entirely agree about all the benefits of being able to state whether a type can or cannot be null. But the question is do you want types to be non-null _by default_?

@Arnavion
Contributor

Pretending that a field on an object obtained over the wire (i.e. ajax) is non-null implies having faith in God

So don't pretend. Mark it as nullable and you'll be forced to test it before using it as non-nullable.

@NoelAbrahams

Sure. It boils down to whether we want to mark a field as nullable (in a system where fields are non-null by default) or whether we want to mark a field as non-nullable (in a system where fields are nullable by default).

The argument is that the former is a breaking change (which may or may not be of significance) and also untenable because it requires the application developer to check and guarantee all entry points.

@fdecampredon

@NoelAbrahams I don't see why it's 'intenable' basically most of the time you don't want null, also when an entry point can return null you will have to check it, in the end a type system with non-null type as default will allows you to make less null check because you will be able to trust some api/library/entry point in your application.

When you think a bit about it marking a type as non null in a nullable type system has limited value, you will still be able to consume nullable typed variable/return type without being forced to test it.
It will also force definition author to write more code, since most of the time well designed library never return null nor undefined value.
Finally even the concept is strange, in a type system with non nullable type, a nullable type is perfectly expressible as an union type: ?string is the equivalent of string | null | undefined. In a type system with nullable type as default where you can mark type as non nullable how would you express !string ? string - null - undefined ?

In the end I don't really understand the concern of people here, null is not a string, in the same way than 5 is not a string, both value won't be able to be used where a string is expected, and letting slip var myString: string = null is as error prone as: var myString: string = 5.
Having null or undefined assignable to any type is perhaps a concept that developer are familiar with, but it is still a bad one.

@NoelAbrahams

I don't think I was entirely correct in my previous post: I'll blame it on the hour.

I've just looked through some of our code to see how things would work and it would certainly help to mark certain fields as nullable, for example:

interface Foo {
        name: string;
        address: string|null; /* Nullable */
}

var foo:Foo = new FooClass();
foo.name.toString(); // Okay
foo.address.toString(); // Error: do not use without null check

But what I do object to is the following:

foo.name = undefined; // Error: non-nullable

I feel this will interfere with the natural way of working with JavaScript.

@fdecampredon

The exact same could apply with number :

interface Foo {
        name: string;
        address: string|number; 
}
var foo:Foo = new FooClass();
foo.name.toString(); // Okay
foo.address.slice() // error

foo.name  = 5 // error

And it's still valid in JavaScript

Reasonably how many time do you willingly assign null to a property of an object ?

@Griffork

I think that most things would be marked as null but you'd be relying on type guards to declare that the field is now non nullable.

@Griffork

@fdecampredon
Quite a lot actually.

@NoelAbrahams

@Griffork,

I think that most things would be marked as null

That was my initial thought. But after going through some sections of our code I found comments such as the following:

interface MyType {

     name: string;

     /** The date the entry was updated from Wikipedia or undefined for user-submitted content. */
     wikiDate: Date; /* Nullable */
}

The idea that a field is nullable is often used to provide information. And TypeScript will catch errors if it requires a type guard when accessing wikiDate.

@fdecampredon

foo.name = 5 // error
And it's still valid in JavaScript

True, but that is an error because TypeScript knows with 100% certainty that it was not intentional.

whereas

foo.name = undefined; // Do not send name to server

is perfectly intentional.

I think the implementation that would most closely fit our requirements is to not use union types, but go with the original suggestion:

 wikiDate: ?Date;
@Griffork

I agree with @NoelAbrahams

@fdecampredon

foo.name = 5 // error
And it's still valid in JavaScript
True, but that is an error because TypeScript knows with 100% certainty that it was not intentional.

The compiler just know that you marked name as string and not string | number if you want a nullable value you would just mark it as ?string or string | null ( which is pretty much equivalent )

I think the implementation that would most closely fit our requirements is to not use union types, but go with the original suggestion:

wikiDate: ?Date;

So we are agree type are non null by default and you would mark nullable with ? :) .
Note that it would be an union type since ?Date would be the equivalent of DateΒ | null | undefined :)

@Griffork

Oh sorry, I was trying to agree to nullable by default and not null with special typing (the symbols are confusing).

@NoelAbrahams

@fdecampredon, actually what it means is when a field or variable is marked as nullable then a type guard is required for access:

var wikiDate: ?Date;

wikiDate.toString(); // error
wikiDate && wikiDate.toString(); // okay

This is not a breaking change, because we should still be able to do this:

 var name: string;   // okay
 name.toString();  // if you think that's fine then by all means

Perhaps you believe that we can't have this without introducing null into union types?

@fdecampredon

Your first example is absolutely right when you do :

wikiDate && wikiDate.toString(); // okay

you use a typeguard and the compiler should not warn anything.

However your second example is not good

var name: string;   // okay
name.toString();  // if you think that's fine then by all means

the compiler should have an error here, a simple algorithm could just error on the first line (unitialized variable not marked as nullable), a more complex one could try to detect assignation before first usage :

var name: string;   // okay
name.toString();  // error because not initialized
var name: string;
if (something) {
  name = "Hello World";
}Β else {
  name = "Foo bar";
}
name.toString();  // no error since name will always be initialized.

I don't know exactly where to put the barrier but it would sure need some kind of subtile tunning to not get in the way of the developper.

It's a breaking change and cannot be introduced before 2.0, except perhaps with the 'use nonnull' directive proposed by @metaweta

@Griffork

Why not have:

var string1: string; //this works like typescript does currently, doesn't need type-guarding before use, null and undefined can be assigned to it.
string1.length; //works
var string2: !string; //this doesn't work because the string must be assigned to a non-null and non-undefined value, doesn't need type-guarding before use.
var string3: ?string; //this must be type guarded to non-null, non-undefined before use.
var string4: !string|null = null; //This may be null, but should never be undefined (and must be type-guarded before use).
var string5: !string|undefined; //this may never be null, but can be undefined (and must be type-guarded before use).

And have a compiler flag (that only works if -noimplicitany is on) which says -noinferrednulls, which disables the normal syntax for types (like string, and int) and you have to supply a ? or ! with them (null, undefined and any types being exceptions).

In this manner, non-nullables are an opt-in, and you can use the compiler flag to force a project to be explicity nulled.
The compiler flag errors at the assignment of types, not after (like the previous proposal).

Thoughts?

Edit: I wrote this because it forces the idea of using non-null to be explicit in every action. Anyone who reads the code who comes from any other TS project will know exactly what's happening. Also the compiler flag becomes very obvious if it's on (as blah: string is an error, but blah:!string isn't, similar to the way -noimplicitany works).

Edit2:
DefinatelyTyped could then be upgraded to support noninferrednulls, and they won't change the use of the libraries if people choose to not opt-in to the ? and ! feature.

@metaweta

I don't care whether non-null and non-undefined are opt-in, opt-out, with
type modifiers (!?), a directive, or a compiler flag; I'll do whatever it
takes to get them so long as they are possible to express, which is not
currently the case.

On Mon, Dec 22, 2014 at 2:35 PM, Griffork notifications@github.com wrote:

Why not have:

var string1: string; //this works like typescript does currently., doesn't need type-guarding before use, null and undefined can be assigned to it.
string1.length; //worksvar string2: !string; //this doesn't work because the string must be assigned to a non-null and non-undefined value, doesn't need type-guarding before use.var string3: ?string; //this must be type guarded to non-null, non-undefined before use.var string4: !string|null = null; //This may be null, but should never be undefined (and must be type-guarded before use).var string5: !string|undefined; //this may never be null, but can be undefined (and must be type-guarded before use).

And have a compiler flag (that only works if -noimplicitany is on) which
says -noinferrednulls, which disables the normal syntax for types (like
string, and int) and you have to supply a ? or ! with them (null, undefined
and any being exceptions).

In this manner, non-nullables are an opt-in, and you can use the compiler
flag to force a project to be explicity nulled.
The compiler flag errors at the assignment of types, not after (like
the previous proposal).

Thoughts?

Reply to this email directly or view it on GitHub
#185 (comment)
.

Mike Stay - metaweta@gmail.com
http://www.cs.auckland.ac.nz/~mike
http://reperiendi.wordpress.com

@jesseschalken
Contributor

@jods4 The noImplicitAny flag does not change the meaning of existing code, it only requires the code to be explicit about something that would otherwise be implicit.

Code Flag Meaning
interface Foo { blah; } interface Foo { blah:any; }
interface Foo { blah; } noImplicitAny error, explicit type required
var foo = 'blah' var foo:string = 'blah'
var foo = 'blah' noImplicitNull var foo:!string = 'blah'

With noImplicitNull, before you had a variable which null could be written to. Now you have a variable which null cannot be written to. That's an entirely different beast to noImplicitAny.

@jesseschalken
Contributor

@RyanCavanaugh has already ruled out flags which change the semantics of existing code. If you're going to flatly ignore the express requirements of the TS team then this ticket is going to hang around for another year.

@jods4
jods4 commented Aug 29, 2015

@jesseschalken Sorry but I fail to see the difference.
Before noImplicitAny you may have this in your code:
let double = x => x*2;
It compiles and works fine. But once you turn on noImplicitAny, then the compiler throws an error at you saying that x is implicitly any. You have to modify your code to make it work with the new flag:
let double = (x: any) => x*2 or better yet let double = (x: number) => x*2.
Note that although the compiler raised an error, it would still emit perfectly working JS code (unless you turn off emit on errors).

The situation with nulls is pretty much the same in my opinion. Let's assume for discussion that with the new flag, T is non-nullable and T? or T | null denotes the union of type T and null.
Before you might have had:
let foo: string; foo = null; or even just let foo = "X"; foo = null which would be inferred to string just the same.
It compiles and works fine. Now turn on the new noImplicitNull flag. Suddenly TS throws an error indicating that you can't assign null to something which was not explicitely declared as such. But except for the typing error your code still emits the same, correct JS code.
With the flag you need to state your intention explicitely and modify the code:
string? foo; foo = null;

So what is the difference, really? Code is always emitting fine and its runtime behavior has not changed at all. In both cases you get errors from the typing system and you have to modify your code to be more explicit in your type declarations to get rid of them.

Also, in both cases, it is possible to take code written under the strict flag and compile it with the flag turned off and it still works the same and without error.

@jods4
jods4 commented Aug 29, 2015

@robertknight Very close to my current thinking.

For modules / definitions that have not opted in the strict non-null types, T should basically mean: turn off all kind of null errors on this type. Trying to coerce it to T? can still create compatibility problems.

The problem is that today some T are actually non-nullable and some are nullable. Consider:

// In a strict module, function len does not accept nulls
function len(x: string): number { return x.length; }
// In a legacy module, some calls to len
let abc: string = "abc";
len(abc);

If you alias string to string? in the legacy module, then the call becomes an error because you pass a possibly null variable into a non-nullable parameter.

@jesseschalken
Contributor

@jods4 Read my comment again. Look at the table. I don't know how to express it any more clearly.

Your example was explicitly crafted to arrive at your conclusion by putting the definition of foo and the assignment to foo next to each other. With noImplicitAny, the only errors that result are specifically from the code that needs to change (because it hasn't changed its meaning, it has only required it to be expressed more explicitly). With noImplicitNull, the code that caused the error was the assignment to foo but the code that needed to change to fix it (to have the old meaning) was the definition of foo. This is critically important, because the flag has changed the meaning of the definition of foo. The assignment and the definition can obviously be on different sides of a library boundary, for which the noImplicitNull flag has changed the meaning of that library's public interface!

@jods4
jods4 commented Aug 29, 2015

the flag changed the meaning of the definition of foo.

Yes, that is true. It changed from "I don't have the slightest idea whether the variable can hold null or not -- and I just don't care" to "This variable is null-free". There's a 50/50 chance that it was right and if it's not, you must precise your intent in the declaration. In the end, the result is just the same as with noImplicitAny: you must make your intent more explicit in the declaration.

The assignment and the definition can obviously be on different sides of a library boundary

Indeed, typically the declaration in the library and the use in my code. Now:

  1. If the library has opted in to strict nulls then it must declare its types correctly. If the library says it has strict null types and x is non nullable, then me trying to assign a null is indeed an error that ought to be reported.
  2. If the library has not opted in to strict nulls then the compiler should not raise any error for its usage.

This flag (just like noImplicitAny) can not be turned on without adjusting your code.

I see your point, but I would say that we do not change the meaning code; rather we express meaning that is not catured by the type system today.

Not only is this good because it will catch errors in today code, but I'd say that without taking such a step there will never be usable non-null types in TS.

@Griffork
Griffork commented Sep 2, 2015

Good news for non nullable types! It looks like the TS team are alright with introducing breaking changes in TS updates!
If you see this:
http://blogs.msdn.com/b/typescript/archive/2015/09/02/announcing-typescript-1-6-beta-react-jsx-better-error-checking-and-more.aspx
They introduce a breaking change (mutually exclusive optional syntax) with a new file type, and they introduce a breaking change without the new file type (affects everyone).
That is a precedent we can argue non nullable types with (e. g. a .sts or strict Typescript extension and corresponding .sdts).

Now we just need to figure out if we want the compiler to attempt to check for undefined types or just null types (and what syntax) and we have a solid proposal.

@spion
spion commented Sep 3, 2015

@jbondc Very interesting read. Happy to see that my intuition about migration to NNBD (non-nullable by default) being easier than migration to optional non-nullable has been confirmed by studies (an order of magnitude less changes to migrate, and in the case of Dart, 1-2 annotations per 1000 lines of code needed nullity changes, not more than 10 even in null-heavy code)

I'm not sure if the complexity of the document really reflects the complexity of non-nullable types. For example, in the generics section they discuss 3 kinds of formal type parameters, then show that you don't actually need those. In TS, null would simply be the type of the completely empty record (no properties, no methods) while {} would be the root non-null type, and then non-nullable generics are simply G<T extends {}> - no need to discuss multiple kinds of formal type parameters at all.

Additionally it seems that they propose a lot of non-essential sugar, like var !x

The survey of existing languages that have dealt with the same problem is the real gem though.

Reading the document I realized that Optional / Maybe types are more powerful than nullable types, especially in a generic context - mostly because of the ability to encode Just(Nothing). For example, if we have a generic Map interface that contains values of type T and supports get which may or may not return a value depending on the presence of a key:

interface Map<T> {
  get(s:string):Maybe<T>
}

there is nothing preventing T from being of type Maybe<U>; the code will work perfectly well and return Just(Nothing) if a key is present but contains a Nothing value, and will simply return Nothing if the key is not present at all.

In contrast, if we use nullable types

interface Map<T> {
  get(s:string):T?
}

then its impossible to distinguish between a missing key and a null value when T is nullable.

Either way, the ability to differentiate nullable from non-nullable values and model the available methods and properties accordingly is a prerequisite for any kind of type safety.

@jods4
jods4 commented Sep 3, 2015

@jbondc This is a very intersting find. They obviously did a lot of work and study on this.

I find comforting that studies show that 80% of declarations are actually meant non-null or that there's only 20 nullity annotations per KLOC (p. 21). As noted in the document this is a strong argument for non-null by default, which was also my feeling.

Another argument in favor of non-null is that it creates a cleaner type system: null is its own type, T is non-null and T? is a synonym for T | null. Because TS already has union type all is nice, clean and works well.

Seeing the list of recent languages that tackled that problem, I really think that a new modern programming language should handle this long-standing issue and reduce null bugs in code bases. This is a far too common problem for something that ought to be modelled in the type system. I still hope TS will get it some day.

I found the idea of operator T! intriguing and possibly useful. I was thinking of a system where T is a non-null type, T? is T | null. But it bothered me that you couldn't really create a generic API that guarantees a non-null result even in the face of a null input. I don't have good use-cases, but in theory I couldn't model this faithfully: function defined(x) { return x || false; }.
Using the non-null reversal operator, one could do: function defined<T>(x: T): T! | boolean. Meaning that if defined returns a T it is guaranteed to be non-null, even if the generic T constraint was nullable, say string?. And I don't think it's hard to model in TS: given T!, if T is a union type that includes null type, return the type resulting from removing null from the union.

@spion

Reading the document I realized that Optional / Maybe types are more powerful than nullable types

You can nest Maybe structures, you cannot nest null, indeed.

This is an interesting discussion in the context of defining new apis, but when mapping existing apis there's little choice. Making the language map Nulls to Maybe will not take advantage of that benefit, unless the function is rewritten entirely.

Maybe encodes two distinct pieces of information: whether there is a value and what the value is. Taking your Map example and looking at C# this is obvious, from Dictionary<T,K>:
bool TryGet(K key, out T value).
Notice that if C# had tuples (maybe C# 7), this is basically the same as:
(bool hasKey, T value) TryGet(K key)
Which is basically a Maybe and allows storing null.

Note that JS has its own way of dealing with this issue and it creates a whole lot of new interesting problems: undefined. A typical JS Map would return undefined if the key is not found, its value otherwise, including null.

@robertknight

Related proposal for C# 7 - dotnet/roslyn#5032

@Griffork
Griffork commented Sep 7, 2015

You guys do realise that the problem isn't solved unless you model undefined in the same manner?
Otherwise all your null problems will just be replaced with undefined problems (which imo are more prevalent anyway).

@jods4
jods4 commented Sep 8, 2015

@Griffork

all your null problems will just be replaced with undefined problems

No, why would they?
My null problems will go away and my undefined problems will remain.

True, undefined is still an issue. But that depends a lot on your coding style. I code with almost no undefined except where the browser forces them on me, which means 90% of my code would be safer with null checks.

@NoelAbrahams

I code with almost no undefined except where the browser forces them on me

I would have thought that JavaScript forces undefined on one at every turn.

  • Uninitialised variables. let x; alert(x);
  • Omitted function arguments. let foo = (a?) => alert(a); foo();
  • Accessing non-existent array elements. let x = []; alert(x[0]);
  • Accessing non-existent object properties. let x = {}; alert(x['foo']);

Null, on the other hand, occurs in fewer and more predictable situations:

  • DOM access. alert(document.getElementById('nonExistent'));
  • Third-party web service responses (since JSON.stringify strips undefined) . { name: "Joe", address: null }
  • Regex.exec

For this reason, we prohibit the use of null, convert all null received over the wire to undefined, and always use strict equality checking for undefined. This has worked well for us in practice.

Consequently I do agree that the undefined problem is the more prevalent one.

@jods4
jods4 commented Sep 8, 2015

@NoelAbrahams Coding style, I tell you :)

Uninitialised variables

I always initialize variables and I have noImplicitAny turned on, so let x would be an error anyway. The closer I would use in my project is let x: any = null, although that's code I wouldn't write often.

Optional function parameters

I use default parameter values for optional parameters, it seems to me that makes more sense (your code will read and use the parameter somehow, doesn't it?). So for me: function f(name?: string = null, options?: any = {}).
Accessing the raw undefined parameter value would be an exceptional case for me.

Accessing non-existent array elements

This is something that I strive not to do in my code. I check my arrays length to not go out of bounds and I don't use sparse arrays (or try to fill empty slots with default values such as null, 0, ...).
Again, you may come up with a special case where I would do that, but that would be an exception, not the rule.

Accessing non-existent object properties.

Pretty much the same thing as for arrays. My objects are typed, if a value is not available I set it to null, not undefined. Again you may find edge cases (like doing a dictionary probe) but they are edge cases and not representative of my coding style.

In all exceptional cases where I get an undefined back, I immediately take action on the undefined result and do not propagate it further or "work" with it. Typical real-world example in a fictional TS compiler with null checks:

let cats: Cat[];
// Note that find returns undefined if there's no cat named Kitty
let myFavoriteCat = cats.find(c => c.name === 'Kitty'); 
if (myFavoriteCat === undefined) {
  // Immediately do something to compensate here:
  // return false; or 
  // myFavoriteCat = new Cat('Kitty'); or
  // whatever makes sense.
}
// Continue with assurance that myFavoriteCat is not null (it was an array of non-nullable cats after all).

For this reason, we prohibit the use of null , convert all null received over the wire to undefined , and always use strict equality checking for undefined . This has worked well for us in practice.

From this I understand that you use a very different coding style than I do. If you basically use undefined everywhere then yes, you will benefit from statically checked null types a lot less than I
would.

Yes, the thing is not 100% watertight because of undefined and I don't believe one can create a reasonably usable language that is 100% correct in this respect. JS introduces undefined in too many places.

But as I hope you can tell from my answers above, with appropriate coding style there is a lot to benefit from null checks. At least my opinion is that in my code base it would help find and prevent many stupid bugs and be a productivity enhancer in my team environment.

@NoelAbrahams

@jods4, it was interesting to read your approach.

I think the objection that I have to that approach is there appear to be a lot of rules that need to be adhered to - it's rather like communism vs. the free market πŸ˜„

The TS team internally have a rule similar to ours for their own style.

@jods4
jods4 commented Sep 8, 2015

@NoelAbrahams "Use undefined" is as much a rule as "Use null".

In any case, consistency is key and I wouldn't like a project where I am not sure if things are supposed to be null or undefined (or an empty string or zero). Especially since TS currently does not help with this issue...

I know TS has a rule to favor undefined over null, I am curious if this is an arbitrary "for the sake of consistency" choice or if there are more arguments behind the choice.

Why I like to use null rather than undefined:

  1. It works in a familiar way for our devs, many come from static OO languages such as C#.
  2. Uninitialized variables are usually regarded as a code smell in many languages, not sure why it should be different in JS. Make your intention clear.
  3. Although JS is a dynamic language, performance is better with static types that do not change. It is more efficient to set a property to null than delete it.
  4. It supports the clean difference between null which is defined but signifies the absence of value, and undefined, which is... undefined. A place where the difference between the two is obvious: optional parameters. Not passing a parameter results in undefined. How do you pass the empty value if you use undefined for that in your code base? Using null there's no problem here.
  5. It lays off a clean path to null checking, as discussed in this thread, which is not really practical with undefined. Although maybe I'm day-dreaming on this one.
  6. You have to make a choice for consistency, IMHO null is as good as undefined.
@isiahmeadows
Contributor

I think the reason for preferring undefined to null is because of
default arguments and consistency with obj.nonexistentProp returning
undefined.

Other than that, I don't get the bikeshedding over what counts as null
enough to require the variable to be nullable.

On Tue, Sep 8, 2015, 06:48 jods notifications@github.com wrote:

@NoelAbrahams https://github.com/NoelAbrahams "Use undefined" is as
much a rule as "Use null".

In any case, consistency is key and I wouldn't like a project where I am
not sure if things are supposed to be null or undefined (or an empty
string or zero). Especially since TS currently does not help with this
issue...

I know TS has a rule to favor undefined over null, I am curious if this
is an arbitrary "for the sake of consistency" choice or if there are more
arguments behind the choice.

Why I like to use null rather than undefined:

  1. It works in a familiar way for our devs, many come from static OO
    languages such as C#.
  2. Uninitialized variables are usually regarded as a code smell in
    many languages, not sure why it should be different in JS. Make your
    intention clear.
  3. Although JS is a dynamic language, performance is better with
    static types that do not change. It is more efficient to set a property to
    null than delete it.
  4. It supports the clean difference between null which is defined but
    signifies the absence of value, and undefined, which is... undefined.
    A place where the difference between the two is obvious: optional
    parameters. Not passing a parameter results in undefined. How do you
    pass the empty value if you use undefined for that in your code
    base? Using null there's no problem here.
  5. It lays off a clean path to null checking, as discussed in this
    thread, which is not really practical with undefined. Although maybe
    I'm day-dreaming on this one.
  6. You have to make a choice for consistency, IMHO null is as good as
    undefined.

β€”
Reply to this email directly or view it on GitHub
#185 (comment)
.

@jods4
jods4 commented Sep 8, 2015

@impinball
Can we stop using "bikeshedding" in every github discussion? We can safely say that using undefined or null is a team preference; but the issue whether to try to include undefined in the null checks (or not) and how it would work is not trivial. So I don't get how that's bikeshedding in the first place?

I have built a fork of TS 1.5 with non-nullable types and it was surprisingly easy. But I think that there are two difficult issues that need a consensus to have non-nullable types in the official TS compiler, both have been discussed at length above without a clear conclusion:

  1. What do we do with undefined? (my opinion: it's still everywhere and unchecked)
  2. How do we handle compatibility with existing code, in particular definitions? (my opinion: opt-in flag, at least per definition file. Turning the flag on is "breaking" because you may have to add null annotations.)
@isiahmeadows
Contributor

My personal opinion is that null and undefined should be treated
equivalently for purposes of nullability. Both are used for that use case,
representing the absence of a value. One is that the value never existed,
and the other is that the value once existed and no longer does. Both
should count for nullability. Most JS functions return undefined, but many
DOM and library functions return null. Both serve the same use case. Thus,
they should be treated equivalently.

The bikeshedding reference was about the code style arguments over which
should be used to represent the absence of a value. Some are arguing for
just null, some are arguing for just undefined, and some are arguing for a
mixture. This proposal shouldn't limit itself to just one of those.

On Tue, Sep 8, 2015, 13:28 jods notifications@github.com wrote:

@impinball https://github.com/impinball
Can we stop using "bikeshedding" in every github discussion? We can safely
say that using undefined or null is a team preference; but the issue
whether to try to include undefined in the null checks (or not) and how
it would work is not trivial. So I don't get how that's bikeshedding in the
first place?

I have built a fork of TS 1.5 with non-nullable types and it was
surprisingly easy. But I think that there are two difficult issues that
need a consensus to have non-nullable types in the official TS compiler,
both have been discussed at length above without a clear conclusion:

  1. What do we do with undefined? (my opinion: it's still everywhere
    and unchecked)
  2. How do we handle compatibility with existing code, in particular
    definitions? (my opinion: opt-in flag, at least per definition file.
    Turning the flag on is "breaking" because you may have to add null
    annotations.)

β€”
Reply to this email directly or view it on GitHub
#185 (comment)
.

@Arnavion
Contributor
Arnavion commented Sep 8, 2015

@jods4 I take it from your point 2 that your fork also infers the non-nullable type by default instead of requiring an explicit non-nullable type annotation?

Can you link it please? I would like to try it out.

@jods4
jods4 commented Sep 8, 2015

@impinball
I would love to see (some) safety against undefined as well, but it is quite pervasive.

In particular, can we define an array of non-nullable types?
Given that an out-of-bounds (or sparse) array access returns undefined, can we conveniently define and use arrays?
I think that requiring all arrays to be nullable is too much of a burden in practice.

Should we differentiate null and undefined types? That's not difficult: T | null, T | undefined, T | null | undefined and may provide an easy answer to the question above. But then what about shorthand syntax: what does T? stand for? Both null and undefined? Do we need two different shorthands?

@Arnavion
Null and Undefined types already exist in TS.
My take was to:

  1. Make all types non-nullable (including inferred types);
  2. Give the null type a name (null) that you can use in type declarations;
  3. Remove the widening from null to any;
  4. Introduce syntax shorthand T? which is the same as T | null;
  5. Remove implicit conversions from null to any other type.

Without access to my sources I think that's the gist of it. Existing Null type and the wonderful TS type system (especially union types and type guards) do the rest.

I have not committed my work on github yet, so I can't share a link for now. I wanted to code the compatibility switch first but I have been very busy with other things :(
The compatibility switch is a lot more involved <_<. But it's important because right now the TS compiler compiles itself with a lot of errors and lots of existing tests fail.
But it seems to actually work nicely on brand new code.

@Griffork
Griffork commented Sep 9, 2015

Let me summarise what I've seen from some commenters so far in the hope that I can illustrate how this conversation is going around in circles:
Problem: Certain 'special case' values are finding their was into code that's not designed to deal with them because they're treated differently to every other type in the language (i.e. null and undefined).
Me: why don't you just lay out programming standards so those problems don't happen?
Other: because it would be nice if the intention could be reflected in the typing, because then it won't be documented differently by every team working in type script, and less false assumptions about 3rd party libraries will happen.
Everone: How are we going to deal with this problem?
Other: let's just make nulls more strict and use standards to deal with undefined!

I don't see how this can possibly be considered a solution when undefined is far more a problem than null is!

Disclaimer: this does not reflect the attitudes of everyone here, it's just that it's come up enough that I wanted to highlight this.
The only way this will be accepted it's if it's a solution to a problem, not a little bit of a solution to the smaller part of a problem.

@Griffork
Griffork commented Sep 9, 2015

Dang phone!
*everyone

*it's come up enough that I wanted to highlight it.

Also the last paragraph should read "the only way this proposal"

@isiahmeadows
Contributor

@jods4 I would say that it would depend on certain constructs. In some cases, you can guarantee non-nullability in those cases, such as the following:

declare const list: T![];

for (const entry of list) {
  // `entry` is clearly immutable here.
}

list.forEach(entry => {
  // `entry` is clearly immutable here.
})

list.map(entry => {
  // `entry` is clearly immutable here.
})

In this case, the compiler would have to have a ton of logic to ensure that the array is checked in bounds:

declare const list: T![]

for (let i = 0; i < list.length; i++) {
  // This could potentially fail if the compiler doesn't correctly do the static bounds check.
  const entry: T![] = list[i];
}

And in this case, there is no way you could guarantee that the compiler could verify the access to get entry is in bounds without actually evaluating parts of the code:

declare const list: T![]

const end = round(max, list.length);

for (let i = 0; i < end; i++) {
  const entry: T![] = list[i];
}

There are some easy, obvious ones, but there are some harder ones.

@jods4
jods4 commented Sep 9, 2015

@impinball Indeed, modern API such as map, forEach or for..of are ok because they skip elements that were never initialized or deleted. (They do include elements that have been set to undefined, but our hypothetical null-safe TS would forbid that.)

But classical array access is an important scenario and I would like to see a good solution for that. Doing complex analysis as you suggested is clearly not possible except in trivial cases (yet important cases since they are common). But note that even if you could prove that i < array.length it doesn't prove that the element is initialized.

Consider the following example, what do you think TS should do?

let array: T![] = [];  // an empty array of non-null, non-undefined T
// blah blah
array[4] = new T();  // Fine for array[4], but it means array[0..3] are undefined, is that OK?
// blah blah
let i = 2;
// Note that we could have an array bounds guard
if (i < array.length) {
  let t = array[i];  // Inferred type would be T!, but this is actually undefined :(
}
@isiahmeadows
Contributor

There's also the problem with Object.defineProperty.

let array = new Array(5)
Object.defineProperty(array, "length", 2, {
  get() { return 10 },
})

On Wed, Sep 9, 2015, 17:49 jods notifications@github.com wrote:

@impinball https://github.com/impinball Indeed, modern API such as map,
forEach or for..of are ok because they skip elements that were never
initialized or deleted. (They do include elements that have been set to
undefined, but our hypothetical null-safe TS would forbid that.)

But classical array access is an important scenario and I would like to
see a good solution for that. Doing complex analysis as you suggested is
clearly not possible except in trivial cases (yet important cases since
they are common). But note that even if you could prove that i <
array.length it doesn't prove that the element is initialized.

Consider the following example, what do you think TS should do?

let array: T![] = []; // an empty array of non-null, non-undefined T// blah blah
array[4] = new T(); // Fine for array[4], but it means array[0..3] are undefined, is that OK?// blah blahlet i = 2;// Note that we could have an array bounds guardif (i < array.length) {
let t = array[i]; // Inferred type would be T!, but this is actually undefined :(
}

β€”
Reply to this email directly or view it on GitHub
#185 (comment)
.

@aleksey-bykov

All you need to do is to:

  1. Make the Null and Undefined types referenceable.
  2. Prevent null and undefined values to be assignable to anything.
  3. Use a union type with Null and/or Undefined where nullability us implied.

It indeed is a bold breaking change and looks more suitable for a language
extension.
On Sep 10, 2015 9:13 AM, "Isiah Meadows" notifications@github.com wrote:

There's also the problem with Object.defineProperty.

let array = new Array(5)
Object.defineProperty(array, "length", 2, {
get() { return 10 },
})

On Wed, Sep 9, 2015, 17:49 jods notifications@github.com wrote:

@impinball https://github.com/impinball Indeed, modern API such as
map,
forEach or for..of are ok because they skip elements that were never
initialized or deleted. (They do include elements that have been set to
undefined, but our hypothetical null-safe TS would forbid that.)

But classical array access is an important scenario and I would like to
see a good solution for that. Doing complex analysis as you suggested is
clearly not possible except in trivial cases (yet important cases since
they are common). But note that even if you could prove that i <
array.length it doesn't prove that the element is initialized.

Consider the following example, what do you think TS should do?

let array: T![] = []; // an empty array of non-null, non-undefined T//
blah blah
array[4] = new T(); // Fine for array[4], but it means array[0..3] are
undefined, is that OK?// blah blahlet i = 2;// Note that we could have an
array bounds guardif (i < array.length) {
let t = array[i]; // Inferred type would be T!, but this is actually
undefined :(
}

β€”
Reply to this email directly or view it on GitHub
<
https://github.com/Microsoft/TypeScript/issues/185#issuecomment-139055786>
.

β€”
Reply to this email directly or view it on GitHub
#185 (comment)
.

@jods4
jods4 commented Sep 10, 2015

@impinball
I feel OK regarding your example. Using defineProperty in this way is stepping outside of the TS safety box and into the dynamic JS realm, don't you think? I don't think I ever called defineProperty directly in TS code.

@aleksey-bykov

looks more suitable for a language extension.

Another issue with this proposal is that unless it gets widely accepted it has lessen value.
Eventually, we need updated TS definitions and I don't think it will happen if it's a seldom used, incompatible, extension or fork of TS.

@isiahmeadows
Contributor

I do know that typed arrays do have a guarantee, because IIRC they throw a
ReferenceError on out of bounds array load and stores. Regular arrays and
arguments objects return undefined when the index is out of bounds. In my
opinion, that's a JS language flaw, but fixing it would most definitely
break the Web.

The only way to "fix" this is in ES6, via a constructor returning a proxy,
and its instance prototype and self prototype set to the original Array
constructor. Something like this:

Array = (function (A) {
  "use strict";
  function check(target, prop) {
    const i = +prop;
    if (prop != i) return target[prop];
    if (i >= target.length) {
      throw new ReferenceError();
    }
    return i;
  }

  function Array(...args) {
    return new Proxy(new Array(...args), {
      get(target, prop) {
        return target[check(target, prop)];
      },
      set(target, prop, value) {
        return target[check(target, prop)] = value;
      },
    });
  }

  Array.prototype = A.prototype;
  Array.prototype.constructor = Array
  Object.setPrototypeOf(Array, A);
  return Array;
})(Array);

(note: it's untested, typed on a phone...)

On Thu, Sep 10, 2015, 10:09 jods notifications@github.com wrote:

@impinball https://github.com/impinball
I feel OK regarding your example. Using defineProperty in this way is
stepping outside of the TS safety box and into the dynamic JS realm, don't
you think? I don't think I ever called defineProperty directly in TS code.

@aleksey-bykov https://github.com/aleksey-bykov

looks more suitable for a language extension.

Another issue with this proposal is that unless it gets widely accepted it
has lessen value.
Eventually, we need updated TS definitions and I don't think it will
happen if it's a seldom used, incompatible, extension or fork of TS.

β€”
Reply to this email directly or view it on GitHub
#185 (comment)
.

@jods4
jods4 commented Sep 10, 2015

@impinball Not sure that there is something to "fix" in the first place...
Those are the semantics of JS and TS must accomodate them somehow.

@Griffork

What if we just have a (slightly different) mark up for declaring a sparse array vs a non-sparse array, and make the non-sparse one automatically initialise (maybe the programmers can supply a value or equation for the initialisation). That way we can force sparse arrays to be type T|undefined (which would change to type T using for... of and other 'safe' operations) and leave non-sparse array's types alone.

//not-sparse
var array = [arrlength] => index*3;
var array = <number[]>[3];
//sparse
var array = [];

Obviously this is not the final syntax.
The second example would initialise every value to the compiler-default for that type.
This does mean that for non-sparse arrays you have to type them, otherwise I suspect you'd have to cast them to something after they were fully initialised.
Also we should need a non-sparse type for arrays so programmers can cast their arrays to non-sparse.

@jods4
jods4 commented Sep 10, 2015

@Griffork
I don't know... it gets confusing.

That way we can force sparse arrays to be type T|undefined (which would change to type T using for... of and other 'safe' operations)

Because of a quirk in JS it doesn't work that way. Assume let arr: (T|undefined)[].
So I am free to do: arr[0] = undefined.
If I do that then using those "safe" functions will return undefined for the first slot. So in arr.forEach(x => ...) you cannot say that x: T. It still has to be x: T|undefined.

The second example would initialise every value to the compiler-default for that type.

This is not very TS-like in spirit. Maybe I'm wrong but it seems to me that TS philosophy is that types are only an additional layer on top of JS and they don't impact codegen. This has perf implications for the sake of partial type-correctness that I don't quite like.

TS can obviously not protect you from everything in JS and there are several functions / constructs that you may call from valid TS, which have dynamic effects on the runtime type of your objects and break static TS type analysis.

Would it be bad if this was a hole in the type system? I mean, this code is really not common: let x: number[] = []; x[3] = 0; and if that's the kind of things you want to do then maybe you should declare your array let x: number?[].

It's not perfect but I think it's good enough for most real-world usage. If you're a purist who wants a sound type system, then you should certainly look at another language, because TS type system is not sound anyway. What do you think?

@Griffork

That's why I said you also need the ability to cast to a non-sparse array
type, so you can initialise an array yourself without the performance
impact.
I'd settle for just a differentiation between (what are meant to be) sparse
arrays and non-sparse arrays by type.

For those who don't know why this is important, it's the same reason you
would want a difference between T and T|null.

On 9:11AM, Fri, Sep 11, 2015 jods notifications@github.com wrote:

@Griffork https://github.com/Griffork
I don't know... it gets confusing.

That way we can force sparse arrays to be type T|undefined (which would
change to type T using for... of and other 'safe' operations)

Because of a quirk in JS it doesn't work that way. Assume let arr:
(T|undefined)[].
So I am free to do: arr[0] = undefined.
If I do that then using those "safe" functions will return undefined
for the first slot. So in arr.forEach(x => ...) you cannot say that x: T.
It still has to be x: T|undefined.

The second example would initialise every value to the compiler-default
for that type.

This is not very TS-like in spirit. Maybe I'm wrong but it seems to me
that TS philosophy is that types are only an additional layer on top of JS
and they don't impact codegen. This has perf implications for the sake of
partial type-correctness that I don't quite like.

TS can obviously not protect you from everything in JS and there are
several functions / constructs that you may call from valid TS, which have
dynamic effects on the runtime type of your objects and break static TS
type analysis.

Would it be bad if this was a hole in the type system? I mean, this code
is really not common: let x: number[] = []; x[3] = 0; and if that's the
kind of things you want to do then maybe you should declare your array let
x: number?[].

It's not perfect but I think it's good enough for most real-world usage.
If you're a purist who wants a sound type system, then you should certainly
look at another language, because TS type system is not sound anyway. What
do you think?

β€”
Reply to this email directly or view it on GitHub
#185 (comment)
.

@isiahmeadows
Contributor

@jods4 What I meant by "fixing" was "fixing" what's IMHO a JS language design flaw. Not TypeScript, but in JavaScript itself.

@isiahmeadows
Contributor

@jods I'm complaining about JS, not TS. I'll admit it's off topic.

On Thu, Sep 10, 2015, 19:19 Griffork notifications@github.com wrote:

That's why I said you also need the ability to cast to a non-sparse array
type, so you can initialise an array yourself without the performance
impact.
I'd settle for just a differentiation between (what are meant to be) sparse
arrays and non-sparse arrays by type.

For those who don't know why this is important, it's the same reason you
would want a difference between T and T|null.

On 9:11AM, Fri, Sep 11, 2015 jods notifications@github.com wrote:

@Griffork https://github.com/Griffork
I don't know... it gets confusing.

That way we can force sparse arrays to be type T|undefined (which would
change to type T using for... of and other 'safe' operations)

Because of a quirk in JS it doesn't work that way. Assume let arr:
(T|undefined)[].
So I am free to do: arr[0] = undefined.
If I do that then using those "safe" functions will return undefined
for the first slot. So in arr.forEach(x => ...) you cannot say that x: T.
It still has to be x: T|undefined.

The second example would initialise every value to the compiler-default
for that type.

This is not very TS-like in spirit. Maybe I'm wrong but it seems to me
that TS philosophy is that types are only an additional layer on top of
JS
and they don't impact codegen. This has perf implications for the sake of
partial type-correctness that I don't quite like.

TS can obviously not protect you from everything in JS and there are
several functions / constructs that you may call from valid TS, which
have
dynamic effects on the runtime type of your objects and break static TS
type analysis.

Would it be bad if this was a hole in the type system? I mean, this code
is really not common: let x: number[] = []; x[3] = 0; and if that's the
kind of things you want to do then maybe you should declare your array
let
x: number?[].

It's not perfect but I think it's good enough for most real-world usage.
If you're a purist who wants a sound type system, then you should
certainly
look at another language, because TS type system is not sound anyway.
What
do you think?

β€”
Reply to this email directly or view it on GitHub
<
https://github.com/Microsoft/TypeScript/issues/185#issuecomment-139408240>
.

β€”
Reply to this email directly or view it on GitHub
#185 (comment)
.

@isiahmeadows
Contributor

And as for my statement with Array lengths, we could operate on the assumption that all array accesses are in bounds, and that out-of-bounds access is undefined unless explicitly specified in the interface. That's very much like what C/C++ does, and it would allow both better typing, and potentially a whole load of compiler optimizations, if someone decides to eventually write a third party compiler that uses the language spec, but isn't as concerned about matching the emit.

I know supporting matching C/C++ undefined behavior sounds very stupid on the surface, but I think that in this case, it could be worth it. It's rare to see something that is actually done better by making an out of bounds access. 99.99% of uses I've seen for that are just extremely pungent code smells, almost always done by people who have almost no familiarity with JavaScript.

(Most of these people, in my experience, haven't even heard of CoffeeScript, much less TypeScript. Many of them are even unaware of the new version of JS that was just finalized and standardized, ES2015.)

@aaronshaf

Is there an emerging resolution to this?

@aaronshaf

Short of having a non-nullable type, it still seems useful for TypeScript to fail if one is trying to access a property on a variable that is assuredly null.

var o = null;
console.log(o.x);

... should fail.

@reissbaker

Ensuring through the type system that all array access is bounds-checked seems like drifting into the realm of dependent types. While dependent types are pretty neat, that seems like a much bigger feature than non-nullable types.

It seems like there are three options assuming that bounds-checking isn't enforced on arrays at compile time:

  1. Array indexing (and any arbitrary array element access by index) is considered to return a nullable type, even on arrays of non-nullable types. Essentially, the [] "method" has a type signature of T?. If you know that you're only doing bounds-checked indexing, you can cast the T? to a T! in your application code.
  2. Array indexing returns exactly the same type (with the same nullability) as the array's generic type parameter, and it's assumed that all array access is bounds-checked by the application. Out-of-bounds access will return undefined, and won't be caught by the type-checker.
  3. The nuclear option: all arrays are hardcoded into the language as being nullable, and attempts to use non-nullable arrays fail typechecks.

These all apply to index-based access of properties on objects, too, e.g. object['property'] where object is of type { [ key: string ]: T! }.

Personally I prefer the first option, where indexing into an array or object returns a nullable type. But even the second option seems better than everything being nullable, which is the current state of affairs. The nuclear option is gross but honestly also still better than everything being nullable.

There's a second question of whether types should be by default non-nullable, or by default nullable. It seems like in either case it would be useful to have syntax both for explicitly nullable and explicitly non-nullable types in order to handle generics; e.g. imagine a get method on a container class (e.g. a Map) that took some value and possibly returned a type, even if the container only contained non-nullable values:

class Container<K,V> {
  get(key: K): V? {
    // fetch from some internal data structure and return the value, if it exists
    // return null otherwise
  }
}

// only non-nullable values allowed in the container
const container = new Container<SomeKeyClass!, SomeValueClass!>();
const val: SomeValueClass!;
// ... later, we attempt to read from the container with a get() call
// even though only non-nullables are allowed in the container, the following should fail:
// get() explicitly returns null when the item can't be found
val = container.get(someKey);

Similarly, we might (this is less of a strong argument) want to ensure our container class only accepted non-null keys on inserts, even when using a nullable key type:

class Container<K, V> {
  insert(key: K!, val: V): void {
    // put the val in the data structure
    // the key must not be null here, even if K is elsewhere a nullable type
  }
}

const container = new Container<SomeKeyClass?, SomeValueClass>();
container.insert(null, new SomeValueClass()); // fails

So regardless of whether the default changes, it seems like it would be useful to have explicit syntax for both nullable types and non-nullable types. Unless I'm missing something?

At the point where there's syntax for both, the default seems like it could be a compiler flag similar to --noImplicitAny. Personally I'd vote for the default staying the same until a 2.0 release, but either seems fine as long as there's an escape hatch (at least temporarily).

@isiahmeadows
Contributor

I would prefer the second option, as even though it would make out-of-bounds access undefined behavior (in the realm of TS typing), I think that's a good compromise for this. It can greatly increase speed, and it's simpler to deal with. If you are really expecting an out-of-bounds access is possible, you should either use a nullable type explicitly, or cast the result to a nullable type (which is always possible). And generally, if the array is non-nullable, any out-of-bounds access is almost always a bug, one that should erupt violently at some point (It's a JS flaw).

It pretty much requires the programmer to be explicit in what they're expecting. It's less type-safe, but in this case, I think type safety may end up getting in the way.

Here's a comparison with each option, using a summation function as an example (primitives are the most problematic):

// Option 1
function sum(numbers: !number[]) {
  let res = 0
  for (let i = 0; i < numbers.length; i++) {
    res += <!number> numbers[i]
  }
  return res
}

// Option 2
function sum(numbers: !number[]) {
  let res = 0
  for (let i = 0; i < numbers.length; i++) {
    res += numbers[i]
  }
  return res
}

// Option 3
function sum(numbers: number[]) {
  let res = 0
  for (let i = 0; i < numbers.length; i++) {
    res += <!number> numbers[i]
  }
  return res
}

Another example: a map function.

// Option 1
function map<T>(list: !T[], f: (value: !T, index: !number) => !T): !T[] {
  let res: !T[] = []
  for (let i = 0; i < list.length; i++) {
    res.push(f(<!T> list[i], i));
  }
  return res
}

// Option 2
function map<T>(list: !T[], f: (value: !T, index: !number) => !T): !T[] {
  let res: !T[] = []
  for (let i = 0; i < list.length; i++) {
    res.push(f(list[i], i));
  }
  return res
}

// Option 3
function map<T>(list: T[], f: (value: !T, index: !number) => !T): T[] {
  let res: T[] = []
  for (let i = 0; i < list.length; i++) {
    const entry = list[i]
    if (entry !== undefined) {
      res.push(f(<!T> entry, i));
    }
  }
  return res
}
@isiahmeadows
Contributor

Another question: what is the type of entry in each of these? !string, ?string, or string?

declare const regularStrings: string[];
declare const nullableStrings: ?string[];
declare const nonnullableStrings: !string[];

for (const entry of regularStrings) { /* ... */  }
for (const entry of nullableStrings) { /* ... */  }
for (const entry of nonnullableStrings) { /* ... */  }
@reissbaker

Option three was a bit of a tongue-in-cheek suggestion πŸ˜›

Re: your last question:

declare const regularStrings: string[];
declare const nullableStrings: string?[];
declare const nonNullableStrings: string![]; // fails typecheck in option three

for(const entry of regularStrings) {
  // option 1: entry is of type string?
  // option 2: depends on default nullability
}

for(const entry of nullableStrings) {
  // option 1 and 2: entry is of type string?
}

for(const entry of nonNullableStrings) {
  // option 1: entry is of type string?
  // option 2: entry is of type string!
}

In some cases β€” where you want to return a non-nullable type and you're getting it from an array, for example β€” you'll have to do an extra cast with option one assuming you've elsewhere guaranteed there are no undefined values in the array (the requirement of this guarantee doesn't change regardless of which approach is taken, only the need to type as string!). Personally I still prefer it because it's both more explicit (you have to specify when you're taking possibly-dangerous behavior, as opposed to it happening implicitly) and more consistent to how most container classes work: for example, a Map's get function clearly returns nullable types (it returns the object if it exists under the key, or null if it doesn't), and if Map.prototype.get returns a nullable then object['property'] should probably do the same, since they make similar guarantees about nullability and are used similarly. Which leaves arrays as the odd ones out where null reference errors can creep back in, and where random access is allowed to be non-nullable by the type system.

There are definitely other approaches; for example, currently Flow uses option two, and last I checked SoundScript made sparse arrays explicitly illegal in their spec (well, strong mode/"SaneScript" makes them illegal, and SoundScript is a superset of the new rules), which to some extent sidesteps the problem although they'll still need to figure out how to deal with manual length changes and with initial allocation. I suspect they'll come closer to option one in the convenience vs. safety tradeoff β€” that is, it'll be less convenient to write but more safe β€” due to their emphasis on type system soundness, but it'll probably look somewhat different than either of these approaches due to the ban on sparse arrays.

I think the performance aspect is extremely theoretical at this point, since AFAIK TypeScript will continue emitting the same JS regardless of casts for any choice here and the underlying VMs will continue bounds-checking arrays under the hood regardless. So I'm not too swayed by that argument. The question to my mind is mostly around convenience vs. safety; to me, the safety win here seems worth the convenience tradeoff. Of course, either is an improvement over having all types be nullable.

@isiahmeadows
Contributor

I agree that the performance part is mostly theoretical, but I still would
like the convenience of assuming. Most arrays are dense, and nullability by
default doesn't make sense for boolean and numeric arrays. If it's not
intended to be a dense array, it should be marked as explicitly nullable so
the intent is clear.

TypeScript really needs a way of asserting things, since asserts are often used to assist in static type checking in other languages. I've seen in the V8 code base an `UNREACHABLE();` macro that allows for assumptions to be a little more safely, crashing the program if the invariant is violated. C++ has `static_assert` for static assertions to aid in type checking.

On Tue, Oct 20, 2015 at 4:01 AM, Matt Baker notifications@github.com
wrote:

Option three was a bit of a tongue-in-cheek suggestion [image:
πŸ˜›]

Re: your last question:

declare const regularStrings: string[];declare const nullableStrings: string?[];declare const nonNullableStrings: string![]; // fails typecheck in option three
for(const entry of regularStrings) {
// option 1: entry is of type string?
// option 2: depends on default nullability
}
for(const entry of nullableStrings) {
// option 1 and 2: entry is of type string?
}
for(const entry of nonNullableStrings) {
// option 1: entry is of type string?
// option 2: entry is of type string!
}

In some cases β€” where you want to return a non-nullable type and you're
getting it from an array, for example β€” you'll have to do an extra cast
with option one assuming you've elsewhere guaranteed there are no undefined
values in the array (the requirement of this guarantee doesn't change
regardless of which approach is taken, only the need to type as string!).
Personally I still prefer it because it's both more explicit (you have to
specify when you're taking possibly-dangerous behavior, as opposed to it
happening implicitly) and more consistent to how most container classes
work: for example, a Map's get function clearly returns nullable types
(it returns the object if it exists under the key, or null if it doesn't),
and if Map.prototype.get returns a nullable then object['property']
should probably do the same, since they make similar guarantees about
nullability and are used similarly. Which leaves arrays as the odd ones out
where null reference errors can creep back in, and where random access is
allowed to be non-nullable by the type system.

There are definitely other approaches; for example, currently Flow uses
option two http://flowtype.org/docs/nullable-types.html, and last I
checked SoundScript made sparse arrays explicitly illegal in their spec
https://github.com/rwaldron/tc39-notes/blob/master/es6/2015-01/JSExperimentalDirections.pdf
(well, strong mode/"SaneScript" makes them illegal, and SoundScript is a
superset of the new rules), which to some extent sidesteps the problem
although they'll still need to figure out how to deal with manual length
changes and with initial allocation.

I think the performance aspect is extremely theoretical at this point,
since AFAIK TypeScript will continue emitting the same JS regardless of
casts for any choice here and the underlying VMs will continue
bounds-checking arrays under the hood regardless. So I'm not too swayed by
that argument. The question to my mind is mostly around convenience vs.
safety; to me, the safety win here seems worth the convenience tradeoff. Of
course, either is an improvement over having all types be nullable.

β€”
Reply to this email directly or view it on GitHub
#185 (comment)
.

Isiah Meadows

@tejacques

Should we just start calling them non-void types? In any case, I think explicitly defining non-void with T! or !T is a mistake. It's difficult to read as a human, and also difficult to deal with all of the cases for the TypeScript compiler.

The main issue I see with non-void types by default is that it is a breaking change. Well what if we just add in some more static analysis, similar to Flow, that doesn't change the behavior at all, but will catch more bugs? Then we can catch most many of the bugs of this class now, but don't change the syntax, and in the future, it will be much easier to introduce a compiler flag or default behavior that is less of a breaking change.

// function compiles happily
function len(x: string): number {
    return x.length;
}

len("works"); // 5
len(null); // error, no property length of null
function len(x: string): number {
    if (x === null) {
        return -1;
    }
    return x.length;
}

len("works"); // 5
len(null); // null

What would really be going on here is modeling the input data as non-void, but adding void implicitly when it is handled in the function. Similarly, the return type is non-void unless it can explicitly return null or undefined

We can also add the ?T or T? type, which forces the null (and/or undefined) check before use. Personally I like T?, but there is precedent to use ?T with Flow.

function len(x: ?string): number {
    return x.length; // error: no length property on type ?string, you must use a type guard
}

One more example -- what about using function results?

function len(x: string): number {
    return x.length;
}

function identity(f: string): string {
    return f;
}

function unknown(): string {
    if (Math.random() > 0.5) {
        return null;
    }
    return "maybe";
}

len("works"); // 5
len(null); // error, no property length of null

identity("works"); // "works": string
identity(null); // null: void
unknown(); // ?string

len(identity("works")); // 5
len(identity(null)); // error, no property length of null
len(unknown()); // error: no length property on type ?string, you must use a type guard

Under the hood, what's really going on here is that TypeScript is inferring whether or not a type can be null by seeing whether it handles null, and is given a possibly null value.

The only tricky part here is how to interface with definition files. I think this can be solved by having the default be to assume that a definition file declaring a function(t: T) does null/void checking, much like the second example. This means those functions will be able to take null values without the compiler generating an error.

This now allows two things:

  1. Gradual adoption of the ?T type syntax, of which optional parameters would already be swapped to.
  2. In the future a compiler flag --noImplicitVoid, could be added which would treat declaration files the same as compiled code files. This would be "breaking", but if done far down the road, the majority of libraries will adopt the best practice of using ?T when the type can be void and T when it cannot. It would also be opt-in, so only those who choose to use it would be affected. This could also require the ?T syntax be used in the case where an object could be void.

I think this is a realistic approach, as it gives much improved safety when the source is available in TypeScript, finding those tricky issues, while still allowing easy, fairly intuitive, and backwards compatible integration with definition files.

@isiahmeadows
Contributor

The prefix ? variant is also used in Closure Compiler annotations, IIRC.

On Tue, Nov 17, 2015, 13:37 Tom Jacques notifications@github.com wrote:

Should we just start calling them non-void types? In any case, I think
explicitly defining non-void with T! or !T is a mistake. It's difficult
to read as a human, and also difficult to deal with all of the cases for
the TypeScript compiler.

The main issue I see with non-void types by default is that it is a
breaking change. Well what if we just add in some more static analysis,
similar to Flow, that doesn't change the behavior at all, but will catch
more bugs? Then we can catch most many of the bugs of this class now, but
don't change the syntax, and in the future, it will be much easier to
introduce a compiler flag or default behavior that is less of a breaking
change.

// function compiles happilyfunction len(x: string): number {
return x.length;
}

len("works"); // 5
len(null); // error, no property length of null

function len(x: string): number {
if (x === null) {
return -1;
}
return x.length;
}

len("works"); // 5
len(null); // null

What would really be going on here is modeling the input data as non-void,
but adding void implicitly when it is handled in the function. Similarly,
the return type is non-void unless it can explicitly return null or
undefined

We can also add the ?T or T? type, which forces the null (and/or
undefined) check before use. Personally I like T?, but there is precedent
to use ?T with Flow.

function len(x: ?string): number {
return x.length; // error: no length property on type ?string, you must use a type guard
}

One more example -- what about using function results?

function len(x: string): number {
return x.length;
}
function identity(f: string): string {
return f;
}
function unknown(): string {
if (Math.random() > 0.5) {
return null;
}
return "maybe";
}

len("works"); // 5
len(null); // error, no property length of null

identity("works"); // "works": string
identity(null); // null: void
unknown(); // ?string

len(identity("works")); // 5
len(identity(null)); // error, no property length of null
len(unknown()); // error: no length property on type ?string, you must use a type guard

Under the hood, what's really going on here is that TypeScript is
inferring whether or not a type can be null by seeing whether it handles
null, and is given a possibly null value.

The only tricky part here is how to interface with definition files. I
think this can be solved by having the default be to assume that a
definition file declaring a function(t: T) does null/void checking, much
like the second example. This means those functions will be able to take
null values without the compiler generating an error.

This now allows two things:

  1. Gradual adoption of the ?T type syntax, of which optional
    parameters would already be swapped to.
  2. In the future a compiler flag --noImplicitVoid, could be added
    which would treat declaration files the same as compiled code files. This
    would be "breaking", but if done far down the road, the majority of
    libraries will adopt the best practice of using ?T when the type can
    be void and T when it cannot. It would also be opt-in, so only those
    who choose to use it would be affected.

I think this is a realistic approach, as it gives much improved safety
when the source is available in TypeScript, finding those tricky issues,
while still allowing easy, fairly intuitive, and backwards compatible
integration with definition files.

β€”
Reply to this email directly or view it on GitHub
#185 (comment)
.

@tejacques

Good point. There's also a lot of similarities with the existing optional parameter declarations:

interface withOptionalProperty {
    o?: string
}
interface withVoidableProperty {
    o: ?string
}

function withOptionalParam(o?: string) { }
function withVoidableParam(o: ?string) { }
@isiahmeadows
Contributor

Actually, they do use prefix. They use ? for explicit nullable types and ! for non-nullable types, with nullable being the default.

@reissbaker

The voidable vs nullable distinction makes a lot of sense. πŸ‘

@Griffork

I feel like this is going around in circles.

Have you all read above why a voidable/non-voidable definition is not going to solve the underlying problem?

@tejacques

@Griffork I've read every comment. I'll admit that what I said is somewhat of a rehashing / combination of what others have said, but I think it is the most realistic path forward. I don't know what you see as the underlying problem, but to me the underlying problem is that null and undefined are part of every type, and the compiler does not currently ensure safety when trying to use an argument of type T. As per my example:

function len(x: string): number {
    return x.length;
}

len("works");
// No error -- 5

len(null);
// Compiler allows this but should error here with something like
// error: no property 'length' of null

len(undefined);
// Compiler allows this but should error here with something like
// error: no property 'length' of undefined

That's my take on the fundamental problem with the language -- the lack of safety. A huge amount of this can be fixed by looking at the flow of arguments into functions using static and type analysis, and giving an error when something could be done unsafely. There doesn't need to be any ?T type at all when the code is available because that analysis can be performed (albeit not in every case perfectly accurately all of the time). The reason for adding a ?T type is because it forces the safety check and makes the programmer's intention very clear.

The problem with the implementation is backwards compatibility. If the TS team were to tomorrow release a change that now a type T is non-void by default, it would break existing code which currently accept and handle void inputs with those type signatures. TS team members have stated in this very issue they are not willing to make such a large breaking change. They are willing to break some things, if the effect is small enough and the benefit is large enough, but this would have too large an impact.

My proposal is in two separate parts, one of which I think would be a great addition to the language which changes none of the existing syntax/semantics except for finding real true bugs, and the other is a possible way to reduce the impact of the breaking change to get the stronger guarantees of non-void types in the future. It is still a breaking change, but of a hopefully more acceptable size and nature.

Part One:

  • Add analysis to identify when null/undefined/void types could be passed to functions which do not handle them (only works when TS code is present, not in definition files).
  • Add ?T type which forces void check before using the argument. This is really just syntactic sugar around a language-level option type.
  • These two features can be implemented independently as each have their own individual merit

Part Two:

  • Wait. Later down the line after Part One is introduced, and the community and users of TS have had time to use those features, the standard will be to use T when the type is not null, and ?T when the type could be null. This isn't guaranteed, but I think this would be a clear obvious best practice.
  • As a separate feature, add a compiler option --noImplicitVoid which requires types to be ?T if they can be void. This is just the compiler enforcing the already existing best-practice. If there is a definition file that doesn't adhere to the best practice it would be incorrect, but that's why it is opt-in.
  • If you really wanted to be strict, the flag could accept arguments specifying which directories/files it should apply to. Then you could apply the change only to your code, and exclude node_modules.

I think this is the most realistic option because Part One can be done even without the second part. It's still a good feature that will largely mitigate this issue. Sure it's not perfect, but I'll take good enough if it means it can happen. It also leaves the option on the table for true non-null types in the future. A problem right now is the longer this problem persists, the more of a breaking change it is to fix it because there is more code being written in TS. With Part One, it at worst should dramatically slow that down, and at best reduce the impact over time.

@dallonf
dallonf commented Nov 18, 2015

@tejacques I, for one, am totally on board with this.

@spion
spion commented Nov 18, 2015

@tejacques - FWIW, I completely agree with your assessment and proposal. Lets hope the TS team agrees :)

@dallonf
dallonf commented Nov 18, 2015

Actually, two things:

One, I'm not sure the Flow-like analysis mentioned Part One is necessary. While it's very cool and useful, I certainly wouldn't want it to hold up voidable/?T types, which seem much more feasible in the current design of the language and provide much more long-term value.

I would also phrase --noImplicitVoid a bit differently - let's say it disallows assigning null and undefined to non-voidable (that is, default) types rather than "requiring types to be ?T if they can be void". I'm pretty sure we mean the same thing, just semantics; focuses on usage rather than definition, which, if I understand how TS works, is the only things that it actually can enforce.

And something just now came to mind: we would have then four levels of voidability (this is also true of Flow, which I think is inspiring a good deal of this conversation):

interface Foo {
  w: string;
  x?: string;
  y: ?string;
  z?: ?string;
}

Under --noImplicitVoid, w can only be a valid string. This is a huge win for type safety. Goodbye, billion dollar mistake! Without --noImplicitVoid, of course, the only constraint is that it must be specified, but it can be null or undefined. This is a rather dangerous behavior of the language, I think, because it looks like it's guaranteeing more than it really is.

x is completely lenient under current settings. It can be a string, null, undefined, and it need not even be present in objects that implement it. Under --noImplicitVoid, things get a little more complex... you might not define it, but if you do set something to it, can it not be void? I think the way Flow handles this is that you could set x to undefined (imitating non-existence), but not null. This might be a little bit too opinionated for TypeScript, though.

Also, it would make logical sense to require void-checking x before using it, but that again would be a breaking change. Could this behavior be part of the --noImplicitVoid flag?

y can be set to any string or void value, but must be present in some form, even if void. And, of course, accessing it requires a void-check. This behavior might be a bit surprising. Should we consider not setting it to be the same as setting it to undefined? If so, what would make it different from x, except requiring a void check?

And finally, z need not be specified, can be set to absolutely anything (well, except a non-string of course), and (for good reason) requires a void check before accessing it. Everything makes sense here!

There's a bit of overlap between x and y, and I suspect that x would eventually become deprecated by the community, preferring the z form for maximum safety.

@jods4
jods4 commented Nov 19, 2015

@tejacques

There doesn't need to be any ?T type at all when the code is available because that analysis can be performed (albeit not in every case perfectly accurately all of the time).

This statement is incorrect. Lots of aspects of nullability can't be accounted for even with full code source available.

For instance: when calling into an interface (statically), you can't know if passing null is alright or not if the interface is not annotated accordingly. In a structural type system such as TS, it's not even easy to know which objects are implementations of the interface and which are not. In general you don't care, but if you want to deduce nullability from the source code, you would.

Another example: if I have an array list and a function unsafe(x) whose source code shows that it doesn't accept a null argument, the compiler can't say if that line is safe or not: list.filter(unsafe). And in fact, unless you can statically know what all possible contents of list will be, this can't be done.

Other cases are linked to inheritance and more.

I'm not saying that a code analysis tool that would flag blatant violation of null contracts has no value (it does). I'm just pointing out that down-playing the usefulness of null annotations when source code is available is IMHO an error.

I had said somewhere in this discussion that I think inference of nullability could help reduce backward compatibility in many simple cases. But it can't completely replace source annotations.

@Griffork

@tejacques my bad, I misread your comment (for some reason my brain decided void===null :( I blame just waking up).
Thanks for the extra post though, it made far more sense to me than your original post. I actually quite like that idea.

@tejacques

I'm going to reply to the three of you separately because writing it out on one posts is an unreadable blob.

@tejacques

@Griffork No problem. Sometimes it's hard to convey everything properly through text and I think it was worth clarifying anyway.

@tejacques

@dallonf Your rephrasing is exactly what I mean -- we're on the same page.

I think the difference between x?: T and y: ?T would be in the tooltips/function usage, and in the typing and guard used.

Declaring as an optional argument changes the tooltips/function usage so it's clear it is optional:
a?: T does not need to be passed as an argument in the function call, can be left blank
If a declaration does not have ?: it must be passed as an argument in the function call, and cannot be left blank

w: T is a required non-void argument (with --noImplicitVoid)
does not require a guard

x?: T is an optional argument, so the type is really T | undefined
requires an if (typeof x !== 'undefined') guard.
Note the triple glyph !== for exact checking on undefined.

y: ?T is a required argument, and the type is really T | void
requires an if (y == null) guard.
Note the double glyph == which matches both null and undefined i.e. void

z?: ?T is an optional argument, and the type is really T | undefined | void which is T | void
requires an if (z == null) guard.
Note again the double glyph == which matches both null and undefined i.e. void

As with all optional arguments, you can't have required arguments that follow optional ones. So that's what the difference would be. Now, you could also just do a null guard on the optional argument, and that would work, too, but a key difference is that you could not pass the null value to the function if you called it; however, you could pass undefined.

I think all of these actually have a place, so it wouldn't necessarily deprecate the current optional argument syntax, but I agree that the z form is the safest.

Edit: Updated wording and fixed some typos.

@tejacques

@jods4 I agree with pretty much everything you said. I'm not trying to downplay the importance of non-void typing. I'm just trying to push it one phase at a time. If the TS team can't do it later, at least we're better off, and if they can do it down the road after implementing more checks and ?T then that's mission accomplished.

I think that the case with Arrays is a really tricky one indeed. You could always do something like this:

function numToString(x: number) {
    return x.toString();
}
var nums: number[] = Array(100);
numToString(nums[0]); // You are screwed!

You can try to do something specifically for uninitialized arrays, like typing the Array function as Array<?T> / ?T[] and upgrading it to T[] after a for-loop initializing it, but I agree that you can't catch everything. That said, that's already a problem anyway, and arrays typically don't even send uninitialized values to map/filter/forEach.

Here's an example -- the output is the same on Node/Chrome/IE/FF/Safari.

function timesTwo(x: number) {
    return x * 2;
}
function all(x) {
    return true;
}
var nums: number[] = Array(100);
nums.map(timesTwo);
// [undefined x 100]
nums.filter(all);
// []
nums.forEach(function(x) { console.log(x); })
// No output

That's really not helping you that much, since it is unexpected, but it's not an error in real JavaScript today.

The only other thing I want to stress is that you can make progress even with interfaces, it's just a lot more work and effort via static analysis than via type system, but it's not too dissimilar to what already happens now.

Here's an example. Let's assume that --noImplicitVoid is off

interface ITransform<T, U> {
    (x: T): U;
}

interface IHaveName {
    name: string;
}

function transform<T, U>(x: T, fn: ITransform<T, U>) {
    return fn(x);
}

var named = {
    name: "Foo"
};

var wrongName = {
    name: 1234
};

var namedNull = {
    name: null
};


var someFun = (x: IHaveName) => x.name;
var someFunHandlesVoid = (x: IHaveName) => {
    if (x != null && x.name != null) {
        return x.name;
    }
    return "No Name";
};

All of the above code compiles just fine -- no issues. Now let's try using it

someFun(named);
// "Foo"
someFun(wrongName);
// error TS2345: Argument of type '{ name: number; }' is not assignable to parameter
// of type 'IHaveName'.
//   Types of property 'name' are incompatible.
//     Type 'number' is not assignable to type 'string'.
someFun(null);
// Not currently an error, but would be something like this:
// error TS#: Argument of type 'null' is not assignale to parameter of type 'IHaveName'.
someFun(namedNull);
// Not currently an error, but would be something like this:
// error TS#: Argument of type '{ name: null; }' is not assignable to parameter of
// type 'IHaveName'.
//   Types of property 'name' are incompatible.
//     Type 'null' is not assignable to type 'string'.

someFunHandlesVoid(named);
// "Foo"
someFunHandlesVoid(wrongName);
// error TS2345: Argument of type '{ name: number; }' is not assignable to parameter
// of type 'IHaveName'.
someFunHandlesVoid(null);
// "No Name"
someFunHandlesVoid(namedNull);
// "No Name"

transform(named, someFun);
// "Foo"
transform(wrongName, someFun);
// error TS2453: The type argument for type parameter 'T' cannot be inferred from the usage.
// Consider specifying the type arguments explicitly.
//   Type argument candidate '{ name: number; }' is not a valid type argument because it
//   is not a supertype of candidate 'IHaveName'.
//     Types of property 'name' are incompatible.
//       Type 'string' is not assignable to type 'number'.
transform(null, someFun);
// Not currently an error, but would be something like this:
// error TS#: The type argument for type parameter 'T' cannot be inferred from the usage.
// Consider specifying the type arguments explicitly.
//   Type argument candidate 'null' is not a valid type argument because it
//   is not a supertype of candidate 'IHaveName'.
transform(namedNull, someFun);
// Not currently an error, but would be something like this:
// error TS#: The type argument for type parameter 'T' cannot be inferred from the usage.
// Consider specifying the type arguments explicitly.
//   Type argument candidate '{ name: null; }' is not a valid type argument because it
//   is not a supertype of candidate 'IHaveName'.
//     Types of property 'name' are incompatible.
//       Type 'string' is not assignable to type 'null'.

transform(named, someFunHandlesVoid);
// "Foo"
transform(wrongName, someFunHandlesVoid);
// error TS2453: The type argument for type parameter 'T' cannot be inferred from the usage.
// Consider specifying the type arguments explicitly.
//   Type argument candidate '{ name: number; }' is not a valid type argument because it
//   is not a supertype of candidate 'IHaveName'.
transform(null, someFunHandlesVoid);
// "No Name"
transform(namedNull, someFunHandlesVoid);
// "No Name"

You're right that you can't catch everything, but you can catch a lot of stuff.

Final note -- what should the behavior of the above be when --noImplicitVoid is on?

Now someFun and someFunHandlesVoid are both typechecked the same and produce the same error messages that someFun produced. Even though someFunHandlesVoid does handle void, calling it with a null or undefined is an error because the signature states it takes non void. It would need to be typed as (x: ?IHaveName) : string to accept null or undefined. If we change it's type, then it continues to work as it did before.

This is the part that is a breaking change, but all we have to do to fix it was add a single character ? to the type signature. We can even have another flag --warnImplicitVoid which does the same thing as a warning so we can slowly migrate over.

@tejacques

I feel like a total jerk for doing this, but I'm going to make one more post.

At this point I'm not sure what to do to proceed. Is there a better idea? Should we:

  • keep discussing/speccing out how this should behave?
  • turn this into three new feature proposals?
    • Enhanced Analysis
    • Maybe/Option type ?T
    • --noImplicitVoid compiler option
  • ping TypeScript team members for input?

I'm leaning towards new proposals and continuing discussion there since it's almost inhumane to ask the TypeScript team to catch up on this thread considering how long it is.

@Griffork

@tejacques

  • You're missing a typeof in the triple equals example to dallonf.
  • You appear to be missing some ? in the example to jods4.

Much as I think the stuff should stay in this thread, I think this thread isn't really being "watched" any more (maybe more like glanced at occasionally). So creating some new threads would definitely build traction.
But wait a few days/a week for people to pop their heads up and supply feedback first. You will want your proposal to be pretty solid.

Edit: remembered markdown exists.

@jesseschalken
Contributor

Commenting on this thread is pretty futile at this stage. Even if a proposal is made which the TypeScript team considers acceptable (I attempted this above in August) there's no way they'll find it among the noise.

The best you can hope is that the level of attention is enough of a prompt for the TypeScript team to come up with their own proposal and implement it. Otherwise, just forget about it and use Flow.

@isiahmeadows
Contributor

+1 for splitting this up, but for now, the --noImplicitVoid option can
wait for the nullable type to be implemented.

So far, we've mostly come to agreement on the syntax and semantics of
nullable types, so if someone could write out a proposal and implementation
of it, that would be golden. I've got a proposal from a similar process
regarding enums of other types, but I just haven't had the time to
implement it due to other projects.

On Wed, Nov 18, 2015, 21:24 Tom Jacques notifications@github.com wrote:

I feel like a total jerk for doing this, but I'm going to make one more
post.

At this point I'm not sure what to do to proceed. Is there a better idea?
Should we:

  • keep discussing/speccing out how this should behave?
  • turn this into three new feature proposals?
    • Enhanced Analysis
    • Maybe/Option type ?T
    • --noImplicitVoid compiler option
  • ping TypeScript team members for input?

I'm leaning towards new proposals and continuing discussion there since
it's almost inhumane to ask the TypeScript team to catch up on this thread
considering how long it is.

β€”
Reply to this email directly or view it on GitHub
#185 (comment)
.

@falsandtru
Contributor

+1 for --noImplicitNull option(disallow void and null assignment).

@andy-hanson

I attempted to mitigate this problem with a special type Op<A> = A | NullType. It seems to work pretty well. See here.

@ahejlsberg ahejlsberg was assigned by mhegazy Feb 20, 2016
@pietro909

+1 for --noImplicitNull as well PLEASE πŸ‘

@lqbweb
lqbweb commented Apr 1, 2016

+1 for --noImplicitNull

@Gaelan
Gaelan commented Apr 3, 2016

Should this be closed?

@isiahmeadows
Contributor

@Gaelan Given #7140 is merged, if you would like to file a new, dedicated issue for --noImplicitNull as suggested by a few people here, then it's probably safe to do so now.

@Gaelan
Gaelan commented Apr 4, 2016

@isiahmeadows It would probably be better to leave this open then.

@mhegazy
Contributor
mhegazy commented Apr 5, 2016

Should this be closed?

We think #2388 is the renaming part of this work. This is why we have not declared this feature complete yet.

if you would like to file a new, dedicated issue for --noImplicitNull as suggested by a few people here, then it's probably safe to do so now.

I am not sure i understand what is requested semantics of this new flag. i would recommend opening a new issue with a clear proposal.

@isiahmeadows
Contributor

@mhegazy The idea posited earlier in this issue for --noImplicitNull was that everything has to be explicitly ?Type or !Type. IMHO I don't feel it's worth the boilerplate when there's another flag that infers non-nullable by default that IIRC was already implemented when nullable types themselves were.

@ahejlsberg
Member

Closing now that #7140 and #8010 are both merged.

@ahejlsberg ahejlsberg closed this Apr 26, 2016
@ahejlsberg ahejlsberg added this to the TypeScript 2.0 milestone Apr 26, 2016
@mhegazy mhegazy added the Fixed label Apr 26, 2016
@massimiliano-mantione

Sorry if I comment on a closed issue but I don't know a better place where to ask and I don't think this is worth a new issue if there's no interest.
Would it be feasible to handle implicit null on a per-file basis?
Like, handle a bunch of td files with noImplicitNull (because they come from definitelytyped and were conceived that way) but handle my source as implicitNull?
Would anybody find this useful?

@mhegazy
Contributor
mhegazy commented May 17, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment