swf output not compatible with flash player 27

[andyli]

Tested with Haxe 3.4.3 and a0577b7.

A hello world program:

class HelloWorld {
	static function main():Void {
		trace("Hello world!");
	}
}

Compiled with

-main HelloWorld
-cp src
-swf bin/HelloWorld.swf

Run with Flash Player Debugger 27.0.0.130 (on a Mac)

Warning: Failed to parse corrupt data.
VerifyError: Error #1001: The method MethodInfo-49() is not implemented.

        at HelloWorld$/main()
        at boot_1ee6/init()
        at flash::Boot/start()
        at boot_1ee6()

If run with Flash Player (not Debugger), the Flash Player will simply crash.
Quick link for FP download: https://www.adobe.com/support/flashplayer/debug_downloads.html

[Simn]

Argh, so that was a real issue after all... It came up on our CI, but I was assuming that it's just some stupid setup issue because it only happened on OS X.

No idea right now what could be causing this...

[Simn]

Could you check if this is a regression from 3.4.2? There are no changes to the SWF generator between 3.4.2 and 3.4.3, so I'm very puzzled about this.

[andyli]

Just checked. The swf produced with Haxe 3.4.2 runs fine...

[andyli]

Looks like it's an OCaml issue...

• Haxe 3.4.3 compiled with OCaml 4.02.3 didn't reproduce this issue.
• Haxe 3.4.3 compiled with OCaml 4.05.0 reproduced this issue.

[Simn]

Whaaaat...

[andyli]

I've just pushed to the three_four_three branch to use OCaml 4.02.3 to build Haxe on Mac, and replaced the 3.4.3 Mac binaries and installer to use the new build artifacts.

We still need to investigate OCaml 4.05's issue since we can't use 4.02.3 forever.

[Simn]

What I don't understand (besides everything else) is why it only fails on 3.4.3 and not on development.

[andyli]

What I don't understand (besides everything else) is why it only fails on 3.4.3 and not on development It also fails on development (at least for the HelloWorld program).

[Simn]

But our CI doesn't fail on development, whereas it was failing on the 3.4.3 branch. Where's the difference?

[andyli]

But our CI doesn't fail on development, whereas it was failing on the 3.4.3 branch. Where's the difference? I think it didnt fail on the 3.4.3 branch either. The flash test still ran, or did you commented it out? My HaxeCI repo has the Haxe dev branch failed as expected. Maybe it only affect small programs?

[Simn]

It was failing: https://travis-ci.org/HaxeFoundation/haxe/jobs/274558244#L4379

I disabled the tests because I thought it was some setup issue, and I really needed the builds before the conference.

[andyli]

Some more testing:

• Haxe 3.4.3 compiled with OCaml 4.02.3 to 4.04.2 (inclusive) didn't reproduce this issue. The produced HelloWorld.swf files are exactly the same byte by byte according to cmp.
• Haxe 3.4.3 compiled with OCaml 4.05.0 reproduced this issue. The HelloWorld.swf file produced is largely different from the correct one.

The hex dump of the two swf files:
HelloWorld_4.04.2.swf.hex.txt
HelloWorld_4.05.0.swf.hex.txt

[andyli]

The current development branch passes because flash target is not tested in mac builds since b4221c3.

[Simn]

Haha... I'm so stupid. x)

[andyli]

Turn out the issue isn't Mac specific - I successfully reproduced it on both Windows and Linux. It's just that we have been using OCaml 4.02.3 in CIs (except for Mac, in which the homebrew ocaml package is 4.05.0).

It also means that this is an serious issue because there is no way for the Linux package to build with an older OCaml version - they will just use the one available in the Linux distro. i.e. The flash target is broken in most recent Linux distro releases.

@ncannasse @Simn Would you try to find out what's happen here?

Quick reminder on how to use OCaml 4.05.0 to compile Haxe 3.4.3 in Windows:

# in a cygwin terminal
opam switch 4.05.0+mingw32c
eval `opam config env`
opam install camlp4
cd path/to/haxe
# clean everything and checkout 3.4.3
make clean && git clean -fdx && git reset --hard HEAD
git checkout 3.4.3
git submodule update --force
# compile
make

[Simn]

Oh man... that could be some 10 year old Obj.magic coming back to haunt us.

[Simn]

Compare -D dump output

Identical

This is good because it means the rest of the compiler works fine. This is bad because it means the problem is the SWF generator itself.

Compare in decompiler

header: identical (except file size)
frames: identical
others: identical
scripts: structure is identical:

scripts
  flash
    Boot
    Lib
  haxe
    Log
  Dynamic
  Main
  Std
  boot_54a1

Compare scripts

Boot: identical
Lib: identical
Log: different

4.02:

package haxe
{
   public class Log
   {
      
      public static function trace(v:*, infos:Object = undefined):void
      {
         Boot.__trace(v,infos);
      } 
      public function Log()
      {
      }
   }
}

4.05:

package haxe
{
   public class Log
   {
      
      public static function trace(param1:*, param2:Object = undefined):void
      {
      } 
      public function Log()
      {
      }
   }
}

Main: identical
Std: identical
boot_54a1: different

4.02:

package
{
   import flash.Boot;
   import flash.Lib;
   import flash.display.MovieClip;
   
   public dynamic class boot_54a1 extends Boot
   {
       
      public function boot_54a1()
      {
         super();
         if(Lib.current == null)
         {
            Lib.current = this;
         }
         start();
      }
      
      override public function init() : void
      {
         Math.NaN = Number.NaN;
         Math.NEGATIVE_INFINITY = Number.NEGATIVE_INFINITY;
         Math.POSITIVE_INFINITY = Number.POSITIVE_INFINITY;
         Math.isFinite = function(i:Number):Boolean
         {
            return isFinite(i);
         };
         Math.isNaN = function(i1:Number):Boolean
         {
            return isNaN(i1);
         };
         Main.main();
      }
   }
}

4.05:

package
{
   import flash.Boot;
   import flash.Lib;
   import flash.display.MovieClip;
   
   public dynamic class boot_54a1 extends Boot
   {
       
      public function boot_54a1()
      {
         super();
         if(Lib.current == null)
         {
            Lib.current = this;
         }
         start();
      }
      
      override public function init() : void
      {
         Math.NaN = Number.NaN;
         Math.NEGATIVE_INFINITY = Number.NEGATIVE_INFINITY;
         Math.POSITIVE_INFINITY = Number.POSITIVE_INFINITY;
         Math.isFinite = function(param1:Number):Boolean
         {
         };
         Math.isNaN = function(param1:Number):Boolean
         {
         };
         Main.main();
      }
   }
}

First thoughts

Some problem with function argument names, but why...

[Simn]

Update: It's not the MultiArray implementation. I reverted that change back to Array/DynArray and still get the same behavior.

[Simn]

Someone on #ocaml mentioned that they had an issue with changed evaluation order which they addressed in this commit: facebook/infer@5b3c2c0

I have confirmed that a bytecode Haxe compiled from 4.04.2 also reproduces the issue, so it's almost definitely an evaluation order problem. Now I just have to find out where it comes from...

[Simn]

Tag IDs + length:

Working:

69: 6
9: 5
86: 16
72: 6119
76: 16
1: 2

Broken:

69: 6
9: 5
86: 16
72: 5436
76: 16
1: 2

72 is TActionScript3, so it's what genswf9 produces.

[Simn]

I finally found it and fixed it in HaxeFoundation/ocamllibs@b2d729b

Took me 5 hours to track this down... Please confirm that it works now.

[andyli]

Took me 5 hours to track this down...

I thought it could take even longer given how tricky it is, I'm so glad you fixed it in just 5 hours :)
I confirm the development branch compiles and works nicely using OCaml 4.05.

I think we need a 3.4.4 release soonish.

[Simn]

I think we need a 3.4.4 release soonish.

Agreed... can you start a branch? I made a bit of a mess in the 3.4.3 branch when I was in a hurry to get the release out.

Maybe we should try to get the RunCi changes onto that branch too so it's a bit cleaner. There was also a hardcoded "three_four_three" somewhere which shouldn't be there.

[andyli]

No problem, I will sort this out in this weekend. Getting the RunCi split may be a bit too complex for the 3.4 branch (e.g. php/php7 change), but I will see what I can do there.

[andyli]

Closing this since the fix was applied to both development and 3.4_bugfix.