[Reporting] Add max concurrent shards setting to schema

[rshen91]

Summary

Closes #161561
This PR exposes the max_concurrent_shards setting in the schema for customers for point in time CSV report generation.

Checklist

• Unit or functional tests were updated or added to match the most common scenarios

[elasticmachine]

Pinging @elastic/appex-sharedux (Team:SharedUX)

[sebelga]

LGTM 👍

[tsullivan]

I think the maxConcurrentShardRequests option also needs to be used somewhere else in the generate_csv file.

[tsullivan]

This also needs to be added to the transport field of the data.search() call in the doSearch() method. Doing that will throw a type compilation error, since the field isn't known in the data service. You'll have to reach out to them and make sure the field is recognized and correctly sent in the requests

I think the recommended way to go in doSearch would be to add the maxConcurrentShardRequests onto the searchSource object with:

private async doSearch(searchSource, settings, ...) {
  searchSource.setField('maxConcurrentShardRequests', settings.maxConcurrentShardRequests);
  ...
}

[rshen91]

Omg thank you good catch

[tsullivan]

I did some searching around to find how this is handled in other parts of the code, and I think max_concurrent_shard_requests must be in the body, not the options.

      const response = await this.clients.es.asCurrentUser.openPointInTime(
        {
          // Note, this throws a TypeScript error, same as "ignore_throttled"
          // Check with the ES clients team to make sure this will work
          max_concurrent_shard_requests: maxConcurrentShardRequests,
          index: indexPatternTitle,
          keep_alive: duration,
          ignore_unavailable: true,
          // @ts-expect-error ignore_throttled is not in the type definition, but it is accepted by es
          ignore_throttled: includeFrozen ? false : undefined, // "true" will cause deprecation warnings logged in ES
        },
        {
          requestTimeout: duration,
          maxRetries: 0,
        }
      );

[tsullivan]

I checked through the type definitions and found other parts of the calls where max_concurrent_shard_requests should be plugged in.

[tsullivan]

Based on other code I could find and the way the types are structured, it looks like max_concurrent_shard_requests should be deep one level further:

    const searchParams: IEsSearchRequest = {
      params: {
        body: searchBody,
        max_concurrent_shard_requests: maxConcurrentShardRequests,
      },
    };

[rshen91]

awesome! changes in bd31abe

[cauemarcondes]

LGTM

[tsullivan]

Maybe these changes made it into this PR from rebasing commits that included a merge commit. You can take this change out of your PR with:

git fetch elastic main
git checkout elastic/main x-pack/plugins/infra/

And that will reset the code under this directory back to the source in main.

[tsullivan]

LGTM

[kibana-ci]

💚 Build Succeeded

• Buildkite Build
• Commit: 81460c8

Metrics [docs]

✅ unchanged

History

• 💚 Build #181161 succeeded 2cd1f23
• 💛 Build #176687 was flaky afdca86
• 💚 Build #176664 succeeded f4c5668
• 💔 Build #176638 failed 6013b2d
• 💔 Build #175951 failed 40ca21a
• 💔 Build #175371 failed eeafa51

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @rshen91

[kibanamachine]

💔 All backports failed

Status	Branch	Result
❌	7.17	Backport failed because of merge conflicts
❌	8.11	Backport failed because of merge conflicts

Manual backport

To create the backport manually run:

node scripts/backport --pr 170344

Questions ?

Please refer to the Backport tool documentation