Haxe compiler is much slower, particularly on Windows #5928

Open
jgranick opened this Issue Jan 6, 2017 · 18 comments

Projects

None yet

6 participants

@jgranick
jgranick commented Jan 6, 2017

I mentioned this issue on the mailing list:

https://groups.google.com/forum/#!topic/haxelang/nTrTBVQhSvQ

On the same machine, Haxe on Windows compiles approximately twice as slow as a Linux virtual machine. This may be caused by how Haxe is compiled, has anyone compared the runtime performance of OCPWin with other versions of OCaml?

Anecdotally, Haxe 2 compilation was so fast it was "non-existant", while build times have climbed to 7 seconds for Neko builds and 14 (!) seconds for CPP generation for our simple projects. The Haxe 3.4 compiler continues the trend with longer build times than 3.2.1

There are probably many factors involved, more advanced features, OS differences, and different code. If there is anything we can do to remedy this on our end (such as avoiding use of some features, or avoiding type inference, fewer files in the class path, etc) it would be very valuable to know.

I hope that we can find some solutions to accelerate the build time, I hope this does not sound critical, because I love the improvements that have come into the compiler, but I also hope there are things we can do (outside of compiler cache) to speed things up. The "haxe.exe" process is becoming the biggest bottleneck in performance for our whole build chain.

Thank you 😄

@jgranick
jgranick commented Jan 6, 2017

I am sure that this has been seen:

https://janestreet.github.io/ocaml-perf-notes.html

Literally, I know nothing about optimized OCaml code (and likely, the compiler is already optimized in many ways) but the note about using compare with floats, at least, seemed relevant. I don't know how expensive "expensive" is, but it may be valuable.

On Windows (compared to Unix platforms) I suspect system calls might be the suspect. @ncannasse, how has Haxe 2 built on Windows? Was it a different implementation of OCaml? Is the issue cross-compilation from a Linux server?

Thanks guys

@RealyUniqueName
Member

In our team we have a linux and a win guys with the same hardware configurations. And build times differs by a factor of 1.5 - 2.
I think this difference can be caused by different target architectures. On linux Haxe is built for 64bit while on windows Haxe is built for 32bit.

@jgranick
jgranick commented Jan 6, 2017

I heard that you can build Haxe for Windows in 64-bit, is there a way to compare the performance there?

@RealyUniqueName
Member

I'll do it today.
However i'm not an experienced windows or ocaml user, so i don't know how to make sure i've built 64 version :)
I'm building with cygwin and my builds are labeled as haxe in task manager, while builds downloaded from haxe.org are labeld as haxe (32bit). So i guess it's safe to assume i'm building 64bit version.

@Simn
Member
Simn commented Jan 6, 2017

https://technet.microsoft.com/en-us/sysinternals/processexplorer.aspx

Right-click a column label -> Select Columns -> Image Type (64 vs 32-bit)

@RealyUniqueName
Member

Here is results for two machines with the same hardware (i5-4670 3.4Hz, 16Gb RAM, same SSD):

Ubuntu 16.04 (my build):

name     | time(s) |   % |      # | info
-----------------------------------
analyzer |  12.470 |  13 | 718632 | 
filters  |  17.850 |  19 |      1 | 
generate |  32.310 |  33 |      1 | 
  swf    |  32.310 |  33 |      1 | 
haxelib  |   0.000 |   0 |      2 | 
macro    |  15.160 |  16 |  33013 | 
other    |   4.060 |   4 |      1 | 
parsing  |   2.590 |   3 |   7821 | 
typing   |   9.930 |  10 |  10155 | 
write    |   2.100 |   2 |      1 | 
  swf    |   2.100 |   2 |      1 | 
-----------------------------------
total    |  96.470 | 100 |      0 | 

Windows 10 (haxe.org nightly build):

name     | time(s) |   % |      # | info
-----------------------------------     
analyzer |  28.073 |  13 | 718602 |     
filters  |  20.960 |  10 |      1 |     
generate |  71.014 |  33 |      1 |     
  swf    |  71.014 |  33 |      1 |     
haxelib  |   0.180 |   0 |      2 |     
macro    |  22.707 |  10 |  33013 |     
other    |  34.215 |  16 |      1 |     
parsing  |   5.058 |   2 |   7821 |     
typing   |  33.309 |  15 |  10155 |     
write    |   2.760 |   1 |      1 |     
  swf    |   2.760 |   1 |      1 |     
-----------------------------------     
total    | 218.275 | 100 |      0 |

Windows 10 (my build):

name     | time(s) |   % |      # | info
-----------------------------------
analyzer |  10.054 |  11 | 718562 |
filters  |  22.019 |  23 |      1 |
generate |  27.947 |  29 |      1 |
  swf    |  27.947 |  29 |      1 |
haxelib  |   0.162 |   0 |      2 |
macro    |  13.654 |  14 |  33013 |
other    |   6.559 |   7 |      1 |
parsing  |   2.389 |   2 |   7821 |
typing   |  10.620 |  11 |  10155 |
write    |   2.255 |   2 |      1 |
  swf    |   2.255 |   2 |      1 |
-----------------------------------
total    |  95.662 | 100 |      0 |

So my (probably) 64bit build is 2.3 times faster than "official" 32bit build.

@Simn
Member
Simn commented Jan 6, 2017

That seems way too much for just 32 vs. 64 bit, so I suspect there might be another problem.

You could try grabbing https://github.com/waneck/hxbuilds and retrace the build server's steps. That should allow you to compare the builds properly.

@Simn
Member
Simn commented Jan 6, 2017

Paging @waneck because this appears to be a real problem.

@Simn Simn added the priority-urgent label Jan 6, 2017
@Simn Simn assigned waneck and Simn Jan 6, 2017
@RealyUniqueName
Member

You could try grabbing https://github.com/waneck/hxbuilds and retrace the build server's steps. That should allow you to compare the builds properly.

I will be able to do it on Monday at the best.

@RealyUniqueName
Member
RealyUniqueName commented Jan 6, 2017 edited

Here are results from another Win10 machine with the same hardware to be sure it's not a single machine specific:

my build:

name     | time(s) |   % |      # | info
-----------------------------------
analyzer |   9.939 |  10 | 718498 |
filters  |  22.221 |  23 |      1 |
generate |  26.198 |  27 |      1 |
  swf    |  26.198 |  27 |      1 |
haxelib  |   0.203 |   0 |      2 |
macro    |  15.898 |  16 |  69781 |
other    |   6.774 |   7 |      1 |
parsing  |   2.200 |   2 |   7818 |
typing   |  11.087 |  11 |  16023 |
write    |   2.356 |   2 |      1 |
  swf    |   2.356 |   2 |      1 |
-----------------------------------
total    |  96.876 | 100 |      0 |

haxe.org build:

name     | time(s) |   % |      # | info
-----------------------------------
analyzer |  27.134 |  14 | 718498 |
filters  |  29.519 |  16 |      1 |
generate |  75.797 |  40 |      1 |
  swf    |  75.797 |  40 |      1 |
haxelib  |   0.165 |   0 |      2 |
macro    |  26.216 |  14 |  69781 |
other    |   8.070 |   4 |      1 |
parsing  |   4.484 |   2 |   7818 |
typing   |  14.344 |   8 |  16023 |
write    |   2.396 |   1 |      1 |
  swf    |   2.396 |   1 |      1 |
-----------------------------------
total    | 188.124 | 100 |      0 |
@andyli
Member
andyli commented Jan 6, 2017 edited

@RealyUniqueName Would you also give the AppVeyor build artifacts a try? They are also 32-bit. The hxbuilds server's are cross-compiled from linux, but AppVeyor's are not.
Here are the artifacts of a recent AppVeyor build: https://ci.appveyor.com/project/HaxeFoundation/haxe/build/job/dw65rlk2phvuwps0/artifacts

@Simn
Member
Simn commented Jan 6, 2017

Interesting, I didn't even realize we had these.

I don't have a large enough project to properly test this at the moment, but my initial attempts suggest that the AppVeyor build is indeed superior:

  • Nightly:
C:\GitHub\ds>C:\HaxeToolkit\nightly\haxe-nightly.exe build.hxml --times
name     | time(s) |   % |     # | info
----------------------------------
analyzer |   0.877 |  39 | 26334 |
filters  |   0.395 |  18 |     1 |
generate |   0.302 |  13 |     1 |
  js     |   0.302 |  13 |     1 |
haxelib  |   0.072 |   3 |     1 |
macro    |   0.079 |   3 |  1069 |
other    |   0.001 |   0 |     1 |
parsing  |   0.160 |   7 |   254 |
typing   |   0.360 |  16 |     1 |
----------------------------------
total    |   2.246 | 100 |     0 |
C:\GitHub\ds>C:\HaxeToolkit\nightly\haxe-realyunique.exe build.hxml --times
name     | time(s) |   % |     # | info
----------------------------------
analyzer |   0.815 |  38 | 26334 |
filters  |   0.364 |  17 |     1 |
generate |   0.281 |  13 |     1 |
  js     |   0.281 |  13 |     1 |
haxelib  |   0.067 |   3 |     1 |
macro    |   0.071 |   3 |  1069 |
other    |   0.001 |   0 |     1 |
parsing  |   0.183 |   9 |   254 |
typing   |   0.364 |  17 |     1 |
----------------------------------
total    |   2.145 | 100 |     0 |
  • My build:
C:\GitHub\ds>C:\HaxeToolkit\nightly\haxe-simn.exe build.hxml --times
name     | time(s) |   % |     # | info
----------------------------------
analyzer |   0.659 |  36 | 26334 |
filters  |   0.295 |  16 |     1 |
generate |   0.279 |  15 |     1 |
  js     |   0.279 |  15 |     1 |
haxelib  |   0.069 |   4 |     1 |
macro    |   0.069 |   4 |  1069 |
other    |   0.001 |   0 |     1 |
parsing  |   0.159 |   9 |   254 |
typing   |   0.321 |  17 |     1 |
----------------------------------
total    |   1.852 | 100 |     0 |
  • AppVeyor:
C:\GitHub\ds>C:\HaxeToolkit\nightly\haxe-appveyor.exe build.hxml --times
name     | time(s) |   % |     # | info
----------------------------------
analyzer |   0.677 |  36 | 26334 |
filters  |   0.296 |  16 |     1 |
generate |   0.282 |  15 |     1 |
  js     |   0.282 |  15 |     1 |
haxelib  |   0.069 |   4 |     1 |
macro    |   0.068 |   4 |  1069 |
other    |   0.001 |   0 |     1 |
parsing  |   0.161 |   9 |   254 |
typing   |   0.320 |  17 |     1 |
----------------------------------
total    |   1.872 | 100 |     0 |
@RealyUniqueName
Member

Looks like appveyor build gives the same compilation time as my build or even better.

name     | time(s) |   % |      # | info
-----------------------------------
analyzer |  10.304 |  12 | 718628 |
filters  |  11.033 |  12 |      1 |
generate |  19.735 |  22 |      1 |
  swf    |  19.735 |  22 |      1 |
haxelib  |   0.140 |   0 |      2 |
macro    |  15.393 |  17 |  69785 |
other    |   6.502 |   7 |      1 |
parsing  |  14.088 |  16 |   7818 |
typing   |   9.187 |  10 |  16023 |
write    |   1.943 |   2 |      1 |
  swf    |   1.943 |   2 |      1 |
-----------------------------------
total    |  88.326 | 100 |      0 |

However it failed to run on some machines with this error:

The application was unable to start correctly (0xc000007b)

http://image.prntscr.com/image/c1618abee9204d6d8db58324ed42f16c.png
All tested machines run Win10 x64.

@Justinfront
Contributor

Haxe 2 was using older ocaml 3.12 and now it uses 4.3 or something? Can the build servers build with different ocaml versions and or c compilers, do the ocamllibs added make an difference?

@jgranick
jgranick commented Jan 6, 2017

Are Neko builds also done on the same Linux server, or are they compiled on a Windows machine?

@jgranick
jgranick commented Jan 6, 2017

I'm also seeing faster performance from the AppVeyor build:

lime build neko --times

Before

name     | time(s) |   % |      # | info
-----------------------------------
analyzer |   0.577 |   9 | 100700 |
filters  |   0.397 |   6 |      1 |
generate |   0.609 |   9 |      1 |
  neko   |   0.609 |   9 |      1 |
macro    |   1.623 |  24 |    190 |
other    |   0.005 |   0 |      1 |
parsing  |   0.642 |  10 |   1038 |
typing   |   2.902 |  43 |   3195 |
-----------------------------------
total    |   6.756 | 100 |      0 |

After

name     | time(s) |   % |      # | info
-----------------------------------
analyzer |   0.542 |  11 | 100700 |
filters  |   0.391 |   8 |      1 |
generate |   0.429 |   9 |      1 |
  neko   |   0.429 |   9 |      1 |
macro    |   1.131 |  24 |    190 |
other    |   0.002 |   0 |      1 |
parsing  |   0.485 |  10 |   1038 |
typing   |   1.782 |  37 |   3195 |
-----------------------------------
total    |   4.763 | 100 |      0 |
@Simn Simn added this to the 3.4 milestone Jan 9, 2017
@Simn
Member
Simn commented Jan 10, 2017

With latest Haxe we can now use 64bit builds properly. As expected it doesn't give any significant performance improvements, but it was a necessary step regardless.

Now we just have to figure out what to do with the build servers @waneck.

@andyli
Member
andyli commented Jan 18, 2017

@RealyUniqueName The error you got (0xc000007b) is probably due to missing dll files. I've now configured AppVeyor to include the necessary dll files in the zip, so it should work now. Try the latest one here: https://ci.appveyor.com/project/HaxeFoundation/haxe/build/job/16gpv5cabio3jinb/artifacts

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment