[META] Fix random test failures

[jenlampton]

There are several tests that fail randomly, usually only on the Travis CI infrastructure. Let's fix them, or replace them with others so that they don't do more harm than good.

Still giving us trouble:

• AJAX: AJAX forms on cached pages
• Form API: Forms using form storage on cached page
• SimpleTest: Installation profile module tests
• Update: Update core functionality Random test failure: Update: Update core functionality (UpdateCoreTestCase) #1965
• Image: Image style flood protection (ImageStyleFloodProtection):
  fail Correct number of images (1) return 403 access denied. image.test:1792
• Random failure: ModuleUninstallTestCase->testUninstallLocale (module.test) #5226 | Random failure: ModuleUninstallTestCase->testUninstallLocale (module.test)
• Fixed: Functional tests: LayoutUpgradePathTest fails too often in GHA. #6293 | Functional tests: LayoutUpgradePathTest fails too often in GHA

Fixed:

• Path pattern bulk updating (PathPatternBulkUpdateTestCase) - Fixed Random test failures with PathPatternBulkUpdateTestCase #4862
• Random failures or timeouts in BatchProcessingTestCase (race condition in all batch processing) - Random failures or timeouts in BatchProcessingTestCase (race condition in all batch processing) #5205
• Comment: Link with label 1 new comment not found. (CommentInterfaceTest) Fixed: Random failures in CommentInterfaceTest - Link with label 1 new comment not found. #5189
• Locale: Translation import (LocaleImportFunctionalTest) - Fixed: Random test failure: Locale: Translation import (LocaleImportFunctionalTest) #2611
• Bootstrap: Page cache test (BootstrapPageCacheTestCase) - Issue #4418: Avoid race-condition in page header test. Fixing regression from #4306. backdrop#3173
• Comment: Comment interface (CommentInterfaceTest) - Issue #1478: Avoid race-condition in comment-new test checking. backdrop#2601
• Image: Image style flood protection - Issue #1478: Fix random failures in image style flood test. backdrop#1201
• Language settings file not valid JSON - Fixed in Random failure in "Field translations tests": The configuration file "language.settings" could not be read.  #511
• Path: Path pattern basic tests - Issue #1478: Fix random failures in user tests and Module API. backdrop#1200
• User admin test case - Issue #1478: Fix random failures in user tests and Module API. backdrop#1200
• Module: Module API - Issue #1478: Fix random failures in user tests and Module API. backdrop#1200
• Bootstrap: Boot and exit hook invocation - Issue #1478: Fix random failures in user tests and Module API. backdrop#1200
• Bootstrap: Page cache test - Issue #1478: Fix random failures in user tests and Module API. backdrop#1200
• Bootstrap: Page cache test (still) - Issue #1478: Fixing failing page compression test. backdrop#1383
• Batch API: Batch processing (BatchProcessingTestCase) - Issue #1478: Remove random values from Batch processing. backdrop#1983
• Layout: Layout Interface Test - Issue #1478: Fixing PHP 5.3 encoding of JSON backslashes. backdrop#1450
• Layout: Layout BlockText Test (LayoutBlockTextTest) - Issue #1478: Avoid random content in layout block content. backdrop#1942
• Layout: Fix Random LayoutInterfaceTest Failures in PHP 5.3 - Fix Random LayoutInterfaceTest Failures in PHP 5.3 #3080
• Image: Image styles path and URL functions (ImageStylesPathAndUrlUnitTest) - Issue #1478: Remove random string from image style test. backdrop#1984
• Date: Date Theme (DateThemeTestCase) (was meant to be fixed in 1.13.2 with DateThemeTestCase randomly(?) fails on testDateDisplayCombination #3794; same failure noticed with 1.14.1) follow-up issue: DateThemeTestCase fails at certain times of day #4180:
  Start and end date on same day with timezone, with remaining days renders correctly. date_themes.test:137

[quicksketch]

I got a couple more randoms today. I think some of these new ones might be related to Background Fetch if the server isn't running fast enough. This was on PHP 5.4 and the tests ran for 30 minutes, compared to the normally speedy 10-15 minutes.

---- Bootstrap: Boot and exit hook invocation (HookBootExitTestCase) ----


Status    Group      Filename          Line Function                            
--------------------------------------------------------------------------------
Fail      Other      bootstrap.test     377 HookBootExitTestCase->testHookBootE
    Page cache cleared.
Fail      Other      bootstrap.test     380 HookBootExitTestCase->testHookBootE
    hook_boot called with aggressive cache and no cached page.
Fail      Other      bootstrap.test     381 HookBootExitTestCase->testHookBootE
    hook_exit called with aggressive cache and no cached page.


---- Layout: Layout Interface Test (LayoutInterfaceTest) ----


Status    Group      Filename          Line Function                            
--------------------------------------------------------------------------------
Fail      Other      layout.test        608 LayoutInterfaceTest->testMainConten
    "The "Main page content" block may only be added once to this
    layout." found
Exception Notice     layout.test        614 LayoutInterfaceTest->testMainConten
    Undefined offset: 0
Exception Notice     layout.test        618 LayoutInterfaceTest->testMainConten
    Undefined offset: 0
Fail      Other      layout.test        621 LayoutInterfaceTest->testMainConten
    Found the Save layout button
Fail      Other      layout.test        621 LayoutInterfaceTest->testMainConten
    Found the requested form fields at
Fail      Other      layout.test        622 LayoutInterfaceTest->testMainConten
    Layout saved successfully after removing old content block.
Fail      Other      layout.test        632 LayoutInterfaceTest->testMainConten
    Main content shown in sidebar after moving main content block.


---- Module: Module API (ModuleUnitTest) ----


Status    Group      Filename          Line Function                            
--------------------------------------------------------------------------------
Fail      Other      module.test         84 ModuleUnitTest->testModuleImplement
    The module implements cache is populated after requesting a page.

[quicksketch]

Details about the "User admin test case" fail:

---- User: User administration (UserAdminTestCase) ----
Status    Group      Filename          Line Function                            
--------------------------------------------------------------------------------
Fail      Other      user.test         1441 UserAdminTestCase->testUserAdmin() 
    User C blocked
Fail      E-mail     user.test         1450 UserAdminTestCase->testUserAdmin() 
    Activation mail sent to user C

[quicksketch]

Filed a PR at backdrop/backdrop#1200 that addresses 4 of these failures. Updated the summary.

[quicksketch]

I think I've got all these worked out. For image styles and module API, the added sleep() timeouts should help the slower servers successfully pass when dealing with time-sensitive issues. The user and path alias tests I rewrote to use more robust XPath checking of elements instead of guessing based on the User ID.

The Layout: Layout Interface Test (LayoutInterfaceTest) is the only one on which I'm unclear. If that test fail keeps coming up, hopefully we will have at least narrowed the scope of the random test failures.

All the fixes so far have gone into 1.2.x and 1.x, so I'm tagging this for 1.2.4.

[quicksketch]

Here's dump of the current fail on the Layout Interface test fail report:

---- Layout: Layout Interface Test (LayoutInterfaceTest) ----
Status    Group      Filename          Line Function                            
--------------------------------------------------------------------------------
Fail      Other      layout.test        608 LayoutInterfaceTest->testMainConten
    "The "Main page content" block may only be added once to this
    layout." found
Exception Notice     layout.test        614 LayoutInterfaceTest->testMainConten
    Undefined offset: 0
Exception Notice     layout.test        618 LayoutInterfaceTest->testMainConten
    Undefined offset: 0
Fail      Other      layout.test        621 LayoutInterfaceTest->testMainConten
    Found the Save layout button
Fail      Other      layout.test        621 LayoutInterfaceTest->testMainConten
    Found the requested form fields at
Fail      Other      layout.test        622 LayoutInterfaceTest->testMainConten
    Layout saved successfully after removing old content block.
Fail      Other      layout.test        632 LayoutInterfaceTest->testMainConten
    Main content shown in sidebar after moving main content block.

Looks like our old friend "Invalid JSON" is still happening as well:

---- User: User email administration (UserAdminSettingsEmailFormTest) ----
Status    Group      Filename          Line Function                            
--------------------------------------------------------------------------------
Exception Uncaught e config.inc         898 ConfigFileStorage->read()          
    ConfigStorageReadException: The configuration file "user.mail" is
    not properly formatted JSON. in ConfigFileStorage->read() (line 898 of
    /home/travis/build/jenlampton/backdrop/core/includes/config.inc).