Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Some tests on travis are running too long #12691

Closed
sgiehl opened this issue Apr 1, 2018 · 7 comments

Comments

@sgiehl
Copy link
Member

commented Apr 1, 2018

In most cases the build is currently failing as the system tests for mysqli get terminated after 40min.
For testing purpose I added a small script to check the running times of tests > 5 sec:

https://travis-ci.org/matomo-org/matomo/jobs/360648037

Tests running very long are:

  • Test Piwik\Tests\System\ArchiveCronTest::testArchivePhpCron took 438.5 seconds
  • ​​​​​​​​Test Piwik\Tests\System\ArchiveWebTest::test_WebArchiveScriptCanBeRun_WithPhpCgi_AndWithoutTokenAuth took 466.8 seconds

Setup of the fixture MaySitesImportedLogs only takes ~30 sec, but contains a lot of data. Maybe we should use a smaller fixture for those tests to run faster?

And some of the API tests Piwik\Tests\System\AutoSuggestAPITest::testAnotherApi take alomst 60sec

@sgiehl

This comment has been minimized.

Copy link
Member Author

commented Apr 1, 2018

ArchiveWebTest moved to Integration tests for now, so tests are passing again. Nevertheless we should check if we can improve the Archiving tests to run faster...

@mattab

This comment has been minimized.

Copy link
Member

commented Jun 26, 2018

As found by Phpunit speedtrap in #12763 (comment) the current slowest tests are:

You should really fix these slow tests (>500ms)...
 1. 215204ms to run Piwik\Tests\System\ArchiveCronTest:testArchivePhpCron
 2. 59605ms to run Piwik\Tests\System\TwoVisitorsTwoWebsitesDifferentDaysTest:testApi with data set #7
 3. 54062ms to run Piwik\Tests\System\AutoSuggestAPITest:testAnotherApi with data set #92
 4. 51416ms to run Piwik\Tests\System\AutoSuggestAPITest:testAnotherApi with data set #96
 5. 51372ms to run Piwik\Tests\System\AutoSuggestAPITest:testAnotherApi with data set #90
 6. 50215ms to run Piwik\Tests\System\AutoSuggestAPITest:testAnotherApi with data set #74
 7. 48881ms to run Piwik\Tests\System\AutoSuggestAPITest:testAnotherApi with data set #72
 8. 48671ms to run Piwik\Tests\System\AutoSuggestAPITest:testAnotherApi with data set #94
 9. 47777ms to run Piwik\Tests\System\AutoSuggestAPITest:testAnotherApi with data set #66
 10. 47513ms to run Piwik\Tests\System\AutoSuggestAPITest:testAnotherApi with data set #58
...and there are 202 more above your threshold hidden from view
Time: 32.84 minutes, Memory: 368.00MB

@mattab

This comment has been minimized.

Copy link
Member

commented Jun 18, 2019

@sgiehl this issue was probably fixed a while ago, but noticed now the IntegrationTest suite is timing out (running through 99% but timing out at 40min) eg. https://travis-ci.org/matomo-org/matomo/jobs/547052031

@mattab mattab changed the title System tests on travis with MYSQLI are running too long Some tests on travis are running too long Jun 18, 2019

@mattab mattab added this to the 3.11.0 milestone Jun 18, 2019

@sgiehl

This comment has been minimized.

Copy link
Member Author

commented Jul 1, 2019

I have checked for long running tests on travis. Actually there are currently only 2 tests that take more than 10 seconds:

  • 13856ms to run Piwik\Plugins\PrivacyManager\tests\Integration\DataPurgingTest:test_LogDataPurging_WorksWhenVisitsInPastTracked
  • 65079ms to run Piwik\Tests\Integration\ServeStaticFileTest:test_nonReadableFile
@tsteur

This comment has been minimized.

Copy link
Member

commented Sep 3, 2019

Moving this for now to priority backlog. I reckon the tests aren't really a problem now. Since we made a lot of changes recently would eventually need to look again at which tests take a long time and then see what we can do. Had a look at the 2 tests that take long. The static file that is not readable not sure why it takes so long. I wonder if it was maybe only once taking long by "accident". The other one is fetching quite a few reports over quite a few days which explains why it takes maybe 13s (which maybe isn't all that long). It's possible it's faster now.
Need to see again later when it becomes an issue again.

@sgiehl

This comment has been minimized.

Copy link
Member Author

commented Sep 4, 2019

@tsteur the static file test should already be fast now with this change: https://github.com/matomo-org/matomo/pull/14605/files

@tsteur

This comment has been minimized.

Copy link
Member

commented Sep 4, 2019

Closing the issue for now 👍

@tsteur tsteur closed this Sep 4, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.