Breaking changes in Haxe 4.0.0

Important and breaking changes in Haxe 4

abstract Null<T>

The Null<T> type is now @:coreType abstract instead of a typedef to T. It still supports implicit conversion from and to T, as well as forwarding any field access to T, so its basic usage shouldn't be affected at all.

However, this change affects macros. Becasue it's not a typedef now, it won't be automatically followed by Context.follow and it won't be matched by the TType constructor. Instead it's now represented as a TAbstract instance, so if a macro is handling Null specifically, it needs to be changed to match TAbstract instead of TType. Also if the macro is handling TAbstract, e.g. by recursing into its underlying type, an additional match for the Null case might be needed, unless it's already handling abstracts with the @:coreType metadata.

The reason for this change is mainly to cleanup the inconsistency and confusion about Null within compiler and macro source. That is, on most static platforms, the nullable type is often different than a non-nullable type, for e.g. basic types (Int,Bool,etc), so having Null<T> represented as a mere type alias (which is what typedef is) leads to bugs and general confusion. Moreover, having an abstract Null<T> type sets the ground for possible future exploration regarding null-safety features, because with abstract types we can control implicit casting and variance easier.

Related PR: https://github.com/HaxeFoundation/haxe/pull/6380

New typed AST node: TEnumIndex

A new constructor were added to the haxe.macro.Type.TypedExprDef enum: TEnumIndex(e1:TypedExpr). It represents access to a enum index value and is generated by the pattern matcher.

Before this node was introduced, the pattern matcher was generating a (potentially inlined) Type.enumIndex call that was hard to distinguish later in the optimization filters and generators.

Related PR: https://github.com/HaxeFoundation/haxe/pull/6364

New Typed AST node: TIdent (TODO: TUnbound?)

Identifiers that are not bound to a known field or variable are now represented by a special haxe.macro.Type.TypedExprDef constructor: TIdent(s:String). These identifiers are most often introduced by the use of untyped (e.g. untyped __js__).

Before this node, such identifiers were represented by a TLocal node pointing a fake local variable with @:unbound metadata. This solution wasn't perfect, because compiler/macro developer would expect TLocal to point an actually declared variable, so having to check whether a given local variable is unbound every time is error-prone.

Related PR: https://github.com/HaxeFoundation/haxe/pull/6424

Unicode-aware lexer

Haxe now expects the source files (.hx) to be UTF-8 encoded and positions reported and expected by the compiler now contain character offsets instead of byte offsets.

This is an important change for IDE support maintainers: all display query offsets and all reported positions must be treated as character-based rather than byte-based.

This makes haxe report proper positions for source files containing multi-byte characters, such as Cyrillic, CJK and others. This also should make it easier for editors to integrate with haxe ide services, since they mostly work with character offsets as well. If the editor has the handling position conversion by e.g. encoding/decoding the file on-request, this should be disabled for Haxe 4.

Related issue: https://github.com/HaxeFoundation/haxe/issues/5163 Related PR: https://github.com/HaxeFoundation/haxe/pull/6172