New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compare test speed for gabriel/bagpass/Travis/AppVeyor #940

Closed
jskeet opened this Issue Aug 17, 2017 · 2 comments

Comments

Projects
None yet
1 participant
@jskeet
Member

jskeet commented Aug 17, 2017

As well as #932, there's an odd discrepancy between test times, all netcoreapp1.0, without the Slow tests:

  • Gabriel: 77s
  • Bagpass: 29s
  • AppVeyor: 15s
  • Travis: 133s

Gabriel/Bagpuss are running the same hardware, so that difference is going to be explained by #932, but why is AppVeyor twice as fast as Bagpuss (both Windows) and Travis twice as slow as Gabriel (both Linux)?

More investigation required.

@jskeet

This comment has been minimized.

Show comment
Hide comment
@jskeet

jskeet Aug 17, 2017

Member

After merging #942, running on netcoreapp1.0:

  • Gabriel: 30s
  • Bagpuss: 15.7s
  • AppVeyor: 15s
  • Travis: 57s

So it looks like Travis is running on slower hardware and the .NET Core tests take longer on Linux than on Windows. Running just GuessZoneIdByTransitionsUncached takes 19 seconds on Gabriel and 4 seconds on Bagpuss, which is basically the entire difference. That's partly due to Linux having more time zones than Windows, but we should probably investigate where the time is going, too.

Member

jskeet commented Aug 17, 2017

After merging #942, running on netcoreapp1.0:

  • Gabriel: 30s
  • Bagpuss: 15.7s
  • AppVeyor: 15s
  • Travis: 57s

So it looks like Travis is running on slower hardware and the .NET Core tests take longer on Linux than on Windows. Running just GuessZoneIdByTransitionsUncached takes 19 seconds on Gabriel and 4 seconds on Bagpuss, which is basically the entire difference. That's partly due to Linux having more time zones than Windows, but we should probably investigate where the time is going, too.

@jskeet

This comment has been minimized.

Show comment
Hide comment
@jskeet

jskeet Oct 15, 2017

Member

Current test times (now that we've sped up the tests for GuessZoneIdByTransitionsUncached by fetching all the time zones just once):

netcoreapp1.0

  • Travis: 33.5s
  • AppVeyor: 13.7s
  • Gabriel: 17.2s
  • Bagpuss: 20.1s

netcoreapp2.0

  • Travis: 25.0s
  • AppVeyor: 10.7s
  • Gabriel: 16.7s
  • Bagpuss: 17.2s

Gabriel and Bagpuss are now close enough that I think the remaining difference between AppVeyor and Travis is due to hardware. Closing.

Member

jskeet commented Oct 15, 2017

Current test times (now that we've sped up the tests for GuessZoneIdByTransitionsUncached by fetching all the time zones just once):

netcoreapp1.0

  • Travis: 33.5s
  • AppVeyor: 13.7s
  • Gabriel: 17.2s
  • Bagpuss: 20.1s

netcoreapp2.0

  • Travis: 25.0s
  • AppVeyor: 10.7s
  • Gabriel: 16.7s
  • Bagpuss: 17.2s

Gabriel and Bagpuss are now close enough that I think the remaining difference between AppVeyor and Travis is due to hardware. Closing.

@jskeet jskeet closed this Oct 15, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment