Type Tracking

[TwitchBronBron]

This is the first of several type tracking tickets.

Tasks:

• 1. Add a new AstNode class called TypeExpression.

  It will be the base AstNode for all future type expressions. The following existing code will be parsed in this way:

  the as type in function paramenters

  'the `string`, `integer`, and `SomeInterface` parts should be TypeExpression. The `as` belongs to the 
  `FunctionParameterExpression`
  sub doSomething(p1 as string, p2 as integer, p3 as SomeInterface)
  end sub

  as ReturnType for function return types

  'the `boolean` should be a TypeExpression. the `as` belongs to the FunctionExpression
  function doSomething() as boolean

  as MemberType for class and interface types

  class Video
      ' the `boolean` should be a TypeExpression, the `as` belongs to the FieldStatement
      public isPlaying as boolean
  end class
  
  interface Metadata
      ' the `string` should be a TypeExpression, the `as` belongs to the FieldStatement
      url as string
  end interface

  This is a breaking change to the AST. For example, FunctionParameterExpression will no longer have a typeToken, but will instead have a typeExpression.

  Here's a rough idea of the structure of the TypeExpression class

  class TypeExpression extends Expression
      constructor(
          public expression:Expression
      ){}
  
      public getType(){
          if (isVariableExpression(this.expression) ){
              return this.getSymbolTable().getType(this.expression.name.text);
          } else if (isDottedGetExpression(this.expression)) { 
              //look up the leftmost variable (i.e. `ns1.ns2.ns3.Iface` would get us `ns1`
              //then, use the BscType interfaces to dig down into the type to get the type
          } else {
              //maybe other stuff?
          }
      }

• 2. Update the parser to properly create subclasses that extend TypeExpression for (no longer necessary)

  Since all AstNode instances are equipped with a .getSymbolTable() function and BrighterScript already links the symbol tables to the correct node level, the TypeExpression nodes will not need any type of linking phase. These will be automatically resolved and avaiable to the SymbolTable

• 3. Add a getType() method to the base AstNode, which can then be overridden by all AstNode descendents.

  Moving forward, every AstNode will have the potential to have a type. To improve stability of the project, the nodes should return "DynamicType" when the type is not known.

  abstract class AstNode
      public abstract getType(): BscType {
          //TODO implement for every node. 
     }
      //...all the other props
  end class

• 4. Enhance the SymbolTable BscType logic
  The current SymbolTable implementation only adds names, but no types (they're all added as dynamic). This needs to be enhanced to actually add symbol types.

  Consider the following code example.

  function main()
      name1 = "John" 
      name2 = name1 '
      name3 = name2 '
  end function

  The symbol table logic might look something like this:

  const table = new SymbolTable();
  table.AddSymbol("name1", new StringType());
  table.addSymbol("name2", new ReferenceType("name1"));
  table.addSymbol("name3", new ReferenceType("name2"));

  This would happen at the start of onProgramValidate for every file. Then, for every scope, the scope symbol tables would be linked to the file, and these Types would be able to derive their information from the parent symbol table accordingly.

  We should probably cache the type lookups during every scope validation, and clear it at the end of each scope validation. Otherwise we'd spend many cycles reevaluating the same types.

• 5. Validate all TypeExpression AstNodes to ensure they reference valid types that can be found in the symbol table

• 6. Modify SymbolTable to always return a single type. That means if there are multiple different symbol types represented, they should be merged into a UnionType.

• 6b. Add way to actually declare a Union Type in code:

function intValue(num as string or integer) as integer
   if type(num) = "Integer"
     return num
   end if
   return Val(num, 10)
end function 

• 6c. Make UnionType.getMemberTypes look at the intersection of member types of all the UnionTypes' types. And since this will require coding, I will probably also need to type it (with a keyboard!)

• 7. Type chains - For DottedGets, IndexedGets, etc. how do we let the developer know where the chain failed its understanding the current type.

• 8. Remove NamespacedVariableExpression in favour of TypeExpression except NamespaceStatement.name which should be DottedGetExpression

• 9. Add type caching, at the appropriate (scope?) level. We must make sure that non-resolved types don't get cached

• 10. Simplify SymbolTable.getSymbolType to use type (if available), first from cache, then from table, otherwise create ReferenceType, if TableProvider is given, and cache result.

Future Work (Non-Breaking, Eg. Phase 2 or 3)

• Allow TypeExpressions in more locations (eg. AssignmentStatements, etc)

• BinaryExpressions and UnaryExpressions should be able to infer resultant types (eg. Integer * Integer => Integer, String + String => String, etc.)

• Change Hover & Completion to use SymbolTable instead of variableDeclarations/callables
  This will complete the refactor to move away from using these lookup tables to unify how a symbol can be discovered/known. When this is done, the concept of VariableDeclarations can be removed from the code.

• Validate various expressions for type validity

  AssignmentStatement type equality

  sub main()
      name = "bob"
      name = 1
  end sub
  '      ~ type 'Integer' cannot be assigned to type 'String'

  Function argument equality

  sub main()
      printMessage(1)
      '            ~ Argument of type 'number' is not assignable to parameter of type 'string'
  end sub
  function printMessage(message as string)
      print message
  end function

• Add ComponentType that are generated from XML or future Component creation. Members (and member types) could be inferred from the component's fields

• CreateObject() should infer correct type (using data scraped from Roku SceneGraph docs for built in types, or from known components)

• Update ArrayType to allow indexedGet, etc. to infer type. Allow code & validation like:

  function sumArray(numbers as integer[]) as integer
       sum = 0
       for each num in numbers
           sum = sum + num ' num is known to be an integer
       end for
       return sum
   end function
   
   sub addName(nameList as string[])
        nameList.push(3.14) ' validation error --- nameList.push expects string
    end sub

   ```