[CI] UpdateResponseTests testToAndFromXContent failing

[elasticsearchmachine]

Build Scans:

• elasticsearch-intake #11980 / part1
• elasticsearch-pull-request #38584 / part-1
• elasticsearch-pull-request #38533 / part-1
• elasticsearch-pull-request #38447 / part-1

Reproduction Line:

./gradlew ":server:test" --tests "org.elasticsearch.action.update.UpdateResponseTests.testToAndFromXContent" -Dtests.seed=2D3C3ECB236FE8E8 -Dtests.locale=si -Dtests.timezone=America/Porto_Velho -Druntime.java=22

Applicable branches:
main

Reproduces locally?:
N/A

Failure History:
See dashboard

Failure Message:

java.lang.AssertionError: expected:<{
  "_index" : "SVHRjY",
  "_id" : "rEW",
  "_version" : 6625642273698185873,
  "_seq_no" : 7536933566608412484,
  "_primary_term" : 19,
  "_ignored_source" : "KmGpbXzY",
  "_field_names" : "tBMBRlp",
  "_ts_routing_hash" : "UtD",
  "found" : true,
  "_source" : {
    "FWyJwc" : 6770195550196900717,
    "XSexjsHvG" : "vGEtxpqXM",
    "ZgOcLinh" : "xRbPItEHin"
  },
  "fields" : {
    "reUEdZjk" : [
      "꓅Ÿ込\u0012뚻\uDA60\uDD79\u0006",
      "oDfc"
    ],
    "UJMbzUT" : [
      86,
      -78,
      -77
    ],
    "jNJrb" : [
      {
        "wHLpa" : "lbdDcGoIjH",
        "xlSot" : 91238994,
        "umbII" : true
      },
      {
        "mOmYB" : -1130145383,
        "hSxyu" : false,
        "vciDj" : "WqlOBYFGhm"
      },
      {
        "kWuMA" : 368930141,
        "eTwIe" : "keTJmGmKic",
        "ewKtS" : true
      }
    ],
    "JAcdo" : [
      [
        41285031,
        -544046064,
        192537760,
        61458758,
        1816143272
      ]
    ]
  }
}> but was:<{
  "_index" : "SVHRjY",
  "_id" : "rEW",
  "_version" : 6625642273698185873,
  "_seq_no" : 7536933566608412484,
  "_primary_term" : 19,
  "_ignored_source" : "KmGpbXzY",
  "_field_names" : "tBMBRlp",
  "_ts_routing_hash" : "UtD",
  "found" : true,
  "_source" : {
    "XSexjsHvG" : "vGEtxpqXM",
    "FWyJwc" : 6770195550196900717,
    "ZgOcLinh" : "xRbPItEHin"
  },
  "fields" : {
    "JAcdo" : [
      [
        41285031,
        -544046064,
        192537760,
        61458758,
        1816143272
      ]
    ],
    "reUEdZjk" : [
      "꓅Ÿ込\u0012뚻\uDA60\uDD79\u0006",
      "oDfc"
    ],
    "UJMbzUT" : [
      86,
      -78,
      -77
    ],
    "jNJrb" : [
      {
        "wHLpa" : "lbdDcGoIjH",
        "xlSot" : 91238994,
        "umbII" : true
      },
      {
        "mOmYB" : -1130145383,
        "hSxyu" : false,
        "vciDj" : "WqlOBYFGhm"
      },
      {
        "ewKtS" : true,
        "kWuMA" : 368930141,
        "eTwIe" : "keTJmGmKic"
      }
    ]
  }
}>

Issue Reasons:

• [main] 4 failures in test testToAndFromXContent (1.7% fail rate in 235 executions)
• [main] 3 failures in step part-1 (2.8% fail rate in 106 executions)
• [main] 3 failures in pipeline elasticsearch-pull-request (2.9% fail rate in 103 executions)

Note:
This issue was created using new test triage automation. Please report issues or feedback to es-delivery.

[elasticsearchmachine]

This has been muted on branch main

Mute Reasons:

• [main] 4 failures in test testToAndFromXContent (1.7% fail rate in 235 executions)
• [main] 3 failures in step part-1 (2.8% fail rate in 106 executions)
• [main] 3 failures in pipeline elasticsearch-pull-request (2.9% fail rate in 103 executions)

Build Scans:

• elasticsearch-intake #11980 / part1
• elasticsearch-pull-request #38584 / part-1
• elasticsearch-pull-request #38533 / part-1
• elasticsearch-pull-request #38447 / part-1

[elasticsearchmachine]

Pinging @elastic/es-core-infra (Team:Core/Infra)

[prdoyle]

Ran the reproduction line and got this:

> Task :server:test FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':server:test'.
> No tests found for given includes: [**/*Tests.class](include rules) [**/*$*.class, **/IndexSettingsOverrideTests.class](exclude rules) [org.elasticsearch.action.update.UpdateResponseTests.testToAndFromXContent](--tests filter)

[prdoyle]

Don't understand what's going on here. Gradle can't seem to run this test even if I annotate it with @Test.

[prdoyle]

In any case, the expected and actual JSON have the same fields, but in a different order. There are very few systems where this matters. I think the risk is low.

[rjernst]

Ran the reproduction line and got this

Possibly what's happening is the test is muted, so it doesn't get run, no tests ran, and thus the failure. Try removing the mute locally from muted-tests.yml (it would be nice if the running was intelligent about this so we could run a repro line without needing to worry about mutes, cc @elastic/es-delivery ).

Gradle can't seem to run this test even if I annotate it with @test.

We do not use @Test, we have old style junit3 tests, which finds methods beginning with test.

[ldematte]

Possibly what's happening is the test is muted, so it doesn't get run, no tests ran, and thus the failure. Try removing the mute locally from muted-tests.yml

++
We used to have -Dtests.filter=@awaitsfix to run tests even if muted, but I don't think (nor expect) this to work with muted-tests.yml