Cross cluster search index pattern won't go to Next Step

[LeeDr]

Kibana version: 6.2.3 BC1 (I reproduced it once on master, but now it's working. Maybe intermittent?)

Elasticsearch version: 6.2.3 BC1

Server OS version: Ubuntu

Browser version: Chrome

Browser OS version: Ubuntu

Original install method (e.g. download page, yum, from source, etc.): tar.gz from release manager build

Description of the problem including expected versus actual behavior:
Existing integration-test Cross cluster search test_create index pattern for data from both clusters passed on 6.2.3-SNAPSHOT on March 8th but failed on BC1 6.2.3-317de015 build from early this morning March 13th.
Here's the screenshot that shows that the index pattern does find the correct 2 indices but says it doesn't match any indices and won't go to the Next Step;

Steps to reproduce:

1. Create cross cluster configuration (you can fake it with a single node if you added the local cluster twice using 2 different names like this);

  curl --insecure -XPUT $ESURL/_cluster/settings  -H 'Content-Type: application/json' -d '{
    "persistent": {
      "search": {
        "remote": {
          "local": {
            "seeds": [
              "localhost:9300"
            ]
          },
          "data": {
            "seeds": [
              "localhost:9300"
            ]
          }
        }
      }
    }
  }'

2. load data into both clusters (I used makelogs). If you use the trick above you only have to load it in your one node/cluster.
3. try to create an index pattern using the wildcard for the cluster identifier like *:makelogs-*
   Expected Result: can proceed to create the index pattern and use it in Discover
   Actual Result: can't go to Next Step to create the index pattern.

I initially reproduced it on master, but now I tried again and it's working.

Errors in browser console (if relevant):

Provide logs and/or server output (if relevant):

[s1monw]

@LeeDr are we sure this has been introduced in 6.2.3 or is it a preexisting issue? if it's a preexinsting issue I don't think we should hold back a bugfix release for this ie. it's not a blocker. /cc @chrisronline
please make sure it's also not transient. I need you guys to report back on the release issue on this.

[s1monw]

@epixa your input would be appreciated on this as well.

[epixa]

I agree with @s1monw's analysis, but I've confirmed with @LeeDr that this has only been seen so far in 6.2.3+, so this could be a blocker. However, @chrisronline is having trouble reproducing the issue, so it's possible there's a different problem here. If this were, for example, a timing issue that only occurs when the user is really fast in the UI (like our tests), then it's not a release blocker. @chrisronline will update both this issue and the release issue when we know for sure.

[s1monw]

@epixa thanks for digging!

[chrisronline]

Removing the blocker tag. This is an issue but in our testing, only affects certain CCS queries. The issue is we are not properly discarding outstanding requests when searching for indices that results in indeterministic state changes within the React component. We're working on a fix, but this shouldn't block 6.2.3

[chrisronline]

The fix has been merged into 6.x and 6.2 so if we choose to re-spin 6.2.3, we'll get this fix. If not, we should document it as a known issue.