Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CI] DocsClientYamlTestSuiteIT {yaml=reference/search/async-search/line_17} failing #53891

Closed
mark-vieira opened this issue Mar 20, 2020 · 3 comments · Fixed by #53910
Closed

[CI] DocsClientYamlTestSuiteIT {yaml=reference/search/async-search/line_17} failing #53891

mark-vieira opened this issue Mar 20, 2020 · 3 comments · Fixed by #53910
Assignees
@javanna
Labels
:Search/Search Search-related issues that do not fall into other categories >test-failure Triaged test failures from CI

Comments

@mark-vieira
Copy link
Contributor

The DocsClientYamlTestSuiteIT.test {yaml=reference/search/async-search/line_17} has failed now 3 times in CI since being added in 7.x.

:docs:integTestRunner » org.elasticsearch.smoketest.DocsClientYamlTestSuiteIT » test {yaml=reference/search/async-search/line_17} (1.217s)
Failure at [reference/search/async-search:63]: stashed value not found for key [body.id]
java.lang.RuntimeException: Failure at [reference/search/async-search:63]: stashed value not found for key [body.id]Open stacktrace
Caused by: java.lang.IllegalArgumentException: stashed value not found for key [body.id]Open stacktrace
[2020-03-20T12:33:59,780][INFO ][o.e.s.DocsClientYamlTestSuiteIT] [test] [yaml=reference/search/async-search/line_17] before test
[2020-03-20T12:34:00,559][INFO ][o.e.s.DocsClientYamlTestSuiteIT] [test] Stash dump on test failure [{
  "stash" : {
    "body" : {
      "version" : 2,
      "is_partial" : false,
      "is_running" : false,
      "start_time_in_millis" : 1584722040520,
      "expiration_time_in_millis" : 1585154040520,
      "response" : {
        "took" : 28,
        "timed_out" : false,
        "_shards" : {
          "total" : 2,
          "successful" : 2,
          "skipped" : 0,
          "failed" : 0
        },
        "hits" : {
          "total" : {
            "value" : 7,
            "relation" : "eq"
          },
          "max_score" : null,
          "hits" : [ ]
        },
        "aggregations" : {
          "sale_date" : {
            "buckets" : [
              {
                "key_as_string" : "2015/01/01 00:00:00",
                "key" : 1420070400000,
                "doc_count" : 3
              },
              {
                "key_as_string" : "2015/01/02 00:00:00",
                "key" : 1420156800000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/01/03 00:00:00",
                "key" : 1420243200000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/01/04 00:00:00",
                "key" : 1420329600000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/01/05 00:00:00",
                "key" : 1420416000000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/01/06 00:00:00",
                "key" : 1420502400000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/01/07 00:00:00",
                "key" : 1420588800000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/01/08 00:00:00",
                "key" : 1420675200000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/01/09 00:00:00",
                "key" : 1420761600000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/01/10 00:00:00",
                "key" : 1420848000000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/01/11 00:00:00",
                "key" : 1420934400000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/01/12 00:00:00",
                "key" : 1421020800000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/01/13 00:00:00",
                "key" : 1421107200000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/01/14 00:00:00",
                "key" : 1421193600000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/01/15 00:00:00",
                "key" : 1421280000000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/01/16 00:00:00",
                "key" : 1421366400000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/01/17 00:00:00",
                "key" : 1421452800000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/01/18 00:00:00",
                "key" : 1421539200000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/01/19 00:00:00",
                "key" : 1421625600000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/01/20 00:00:00",
                "key" : 1421712000000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/01/21 00:00:00",
                "key" : 1421798400000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/01/22 00:00:00",
                "key" : 1421884800000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/01/23 00:00:00",
                "key" : 1421971200000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/01/24 00:00:00",
                "key" : 1422057600000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/01/25 00:00:00",
                "key" : 1422144000000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/01/26 00:00:00",
                "key" : 1422230400000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/01/27 00:00:00",
                "key" : 1422316800000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/01/28 00:00:00",
                "key" : 1422403200000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/01/29 00:00:00",
                "key" : 1422489600000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/01/30 00:00:00",
                "key" : 1422576000000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/01/31 00:00:00",
                "key" : 1422662400000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/02/01 00:00:00",
                "key" : 1422748800000,
                "doc_count" : 2
              },
              {
                "key_as_string" : "2015/02/02 00:00:00",
                "key" : 1422835200000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/02/03 00:00:00",
                "key" : 1422921600000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/02/04 00:00:00",
                "key" : 1423008000000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/02/05 00:00:00",
                "key" : 1423094400000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/02/06 00:00:00",
                "key" : 1423180800000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/02/07 00:00:00",
                "key" : 1423267200000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/02/08 00:00:00",
                "key" : 1423353600000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/02/09 00:00:00",
                "key" : 1423440000000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/02/10 00:00:00",
                "key" : 1423526400000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/02/11 00:00:00",
                "key" : 1423612800000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/02/12 00:00:00",
                "key" : 1423699200000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/02/13 00:00:00",
                "key" : 1423785600000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/02/14 00:00:00",
                "key" : 1423872000000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/02/15 00:00:00",
                "key" : 1423958400000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/02/16 00:00:00",
                "key" : 1424044800000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/02/17 00:00:00",
                "key" : 1424131200000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/02/18 00:00:00",
                "key" : 1424217600000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/02/19 00:00:00",
                "key" : 1424304000000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/02/20 00:00:00",
                "key" : 1424390400000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/02/21 00:00:00",
                "key" : 1424476800000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/02/22 00:00:00",
                "key" : 1424563200000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/02/23 00:00:00",
                "key" : 1424649600000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/02/24 00:00:00",
                "key" : 1424736000000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/02/25 00:00:00",
                "key" : 1424822400000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/02/26 00:00:00",
                "key" : 1424908800000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/02/27 00:00:00",
                "key" : 1424995200000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/02/28 00:00:00",
                "key" : 1425081600000,
                "doc_count" : 0
              },
              {
                "key_as_string" : "2015/03/01 00:00:00",
                "key" : 1425168000000,
                "doc_count" : 2
              }
            ]
          }
        }
      }
    }
  }
}]
[2020-03-20T12:34:00,688][INFO ][o.e.s.DocsClientYamlTestSuiteIT] [test] [yaml=reference/search/async-search/line_17] after test
[2020-03-20T12:34:00,689][ERROR][o.e.s.DocsClientYamlTestSuiteIT] [test] This failing test was generated by documentation starting at reference/search/async-search.asciidoc:line_17. It may include many snippets. See docs/README.asciidoc for an explanation of test generation.
REPRODUCE WITH: ./gradlew ':docs:integTestRunner' --tests "org.elasticsearch.smoketest.DocsClientYamlTestSuiteIT.test {yaml=reference/search/async-search/line_17}" -Dtests.seed=E7F668D741AE1BFE -Dtests.security.manager=true -Dtests.locale=is -Dtests.timezone=Canada/Eastern -Dcompiler.java=13

https://gradle-enterprise.elastic.co/s/q7db6wdhyqzr6/tests/jbm2x6tnwtxku-ym6b76lnbgpvq

This is only in 7.x and did not reproduce locally for me.
@mark-vieira mark-vieira added :Search/Search Search-related issues that do not fall into other categories >test-failure Triaged test failures from CI labels Mar 20, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-search (:Search/Search)

@mark-vieira
Copy link
Contributor Author

This test has a failure rate of over 20% since being introduced so I've muted in 7.x with 0cfe6d9.

@javanna javanna self-assigned this Mar 20, 2020
@javanna
Copy link
Member

javanna commented Mar 20, 2020

thanks once again this stems from the same problem as #53887 . Timing issue. I will put up a PR with a fix shortly.

@javanna javanna mentioned this issue Mar 20, 2020
Merged
javanna added a commit to javanna/elasticsearch that referenced this issue Mar 23, 2020
@javanna
[DOCS] address timing issue in async search docs tests
4d9b2ec 
The docs snippets for submit async search have proven difficult to test as it is not possible to guarantee that you get a response that is not final, even when providing `wait_for_completion=0`. In the docs we want to show though a proper long-running query, and its first response should be partial rather than final.

With this commit we adapt the docs snippets to show a partial response, and replace under the hood all that's needed to make the snippets tests succeed when we get a final response. Also, increased the timeout so we always get a final response.

Closes elastic#53887
Closes elastic#53891
javanna added a commit that referenced this issue Mar 23, 2020
@javanna
[DOCS] address timing issue in async search docs tests (#53910)
aa56f91 
The docs snippets for submit async search have proven difficult to test as it is not possible to guarantee that you get a response that is not final, even when providing `wait_for_completion=0`. In the docs we want to show though a proper long-running query, and its first response should be partial rather than final.

With this commit we adapt the docs snippets to show a partial response, and replace under the hood all that's needed to make the snippets tests succeed when we get a final response. Also, increased the timeout so we always get a final response.

Closes #53887
Closes #53891
javanna added a commit to javanna/elasticsearch that referenced this issue Mar 23, 2020
@javanna
[DOCS] address timing issue in async search docs tests (elastic#53910)
4fd4e7d 
The docs snippets for submit async search have proven difficult to test as it is not possible to guarantee that you get a response that is not final, even when providing `wait_for_completion=0`. In the docs we want to show though a proper long-running query, and its first response should be partial rather than final.

With this commit we adapt the docs snippets to show a partial response, and replace under the hood all that's needed to make the snippets tests succeed when we get a final response. Also, increased the timeout so we always get a final response.

Closes elastic#53887
Closes elastic#53891
javanna added a commit that referenced this issue Mar 23, 2020
@javanna
Backport of async search changes (#53976)
932a7e3 
* Get Async Search: omit _clusters section when empty (#53907)

The _clusters section is omitted by the search API whenever no remote clusters are searched. Async search should do the same, but Get Async Search returns a deserialized response, hence a weird `_clusters` section with all values set to `0` gets returned instead. In fact the recreated Clusters object is not the same object as the EMPTY constant, yet it has the same content.

This commit addresses this by changing the comparison in the `toXContent` method to not print out the section if the number of total clusters is `0`.

* Async search: remove version from response (#53960)

The goal of the version field was to quickly show when you can expect to find something new in the search response, compared to when nothing has changed. This can also be done by looking at the `_shards` section and `num_reduce_phases` returned with the search response. In fact when there has been one or more additional reduction of the results, you can expect new results in the search response. Otherwise, the `_shards` section could notify of additional failures of shards that have completed the query, but that is not a guarantee that their results will be exposed (only when the following partial reduction is performed their results will be available).

That said this commit clarifies this in the docs and removes the version field from the async search response

* Async Search: replicas to auto expand from 0 to 1 (#53964)

This way single node clusters that are green don't go yellow once async search is used, while
all the others still have one replica.

* [DOCS] address timing issue in async search docs tests (#53910)

The docs snippets for submit async search have proven difficult to test as it is not possible to guarantee that you get a response that is not final, even when providing `wait_for_completion=0`. In the docs we want to show though a proper long-running query, and its first response should be partial rather than final.

With this commit we adapt the docs snippets to show a partial response, and replace under the hood all that's needed to make the snippets tests succeed when we get a final response. Also, increased the timeout so we always get a final response.

Closes #53887
Closes #53891
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@javanna javanna

Labels
:Search/Search Search-related issues that do not fall into other categories >test-failure Triaged test failures from CI
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[DOCS] address timing issue in async search docs tests javanna/elasticsearch
3 participants
@javanna @mark-vieira @elasticmachine