Global Search - Numerical Column filter 'Show Missing Values Only' not behaving as expected

[samuel-kerrien]

Describe the bug
Two issues related to this option, I have left them in the same ticket as they may be related.

Issue 1 - filter behaviour isn't coherent
Steps to reproduce:

• Let's assume I have 62 datasets in my current selection, some 'Neuron Density' for argument sake.
• I open the numerical column 'Neuron Density' and the 'Show Missing Values Only (12)' is unticked. I understand from this label that there are 12 datasets out of 62 that do not have a numerical value for this column.
• When clicking on the option, I end up still having 62 datasets.
• When I then untick the option, I end up with 50 datasets (those that do have a numerical value).

Expected behaviour: when selecting the option, only dataset that DO NOT have numerical value are shown. When unticked, all datasets (with and without numerical) are shown.

Issue 2 - filter selection is not maintained
Steps to reproduce:

• I open Global Search
• I select datasets of type 'Neuron Density'
• I open the 'Neuron Density' column filter and tick 'Show Missing Values Only (12)'
• I close the column filter, and open it again. The option is now unticked.

Expected behaviour: When I select the option, when closing and opening the column filter again, the state of the option should remain.

[dhaneshnm]

{ "track_total_hits": true, "size": 8, "from": 0, "query": { "bool": { "filter": { "bool": { "must": { "bool": { "should": [ { "term": { "@type.keyword": "https://neuroshapes.org/NeuronDensity" } } ] } }, "should": [ { "bool": { "must_not": [{ "exists": { "field": "neuronDensity" } }] } } ] } }, "must": { "match_all": {} } } } }

[dhaneshnm]

Hey team! Please add your planning poker estimate with ZenHub @imsdu @nicwells @samuel-kerrien

[smitfire]

showing missing values tickbox unchecks on close and re-open too

[dhaneshnm]

Please add your planning poker estimate with ZenHub @smitfire

[samuel-kerrien]

During review with the team, it was noted that when filtering for (missing) values, say with a categorical column like Brain Region. It does work. !

Screenshot 2022-03-16 at 14.29.07.png

In that case, the ES query is:

{
  "bool": {
    "filter": {
      "bool": {
        "must": [
          {
            "bool": {
              "filter": {
                "term": {
                  "@type.keyword": "https://neuroshapes.org/Trace"
                }
              }
            }
          },
          {
            "bool": {
              "should": [
                {
                  "bool": {
                    "must_not": [
                      {
                        "exists": {
                          "field": "eType.label.keyword"
                        }
                      }
                    ]
                  }
                }
              ]
            }
          }
        ]
      }
    },
    "must": {
      "match_all": {}
    }
  }
}

Compared to the previous one:

{
  "bool": {
    "filter": {
      "bool": {
        "must": {
          "bool": {
            "should": [
              {
                "term": {
                  "@type.keyword": "https://neuroshapes.org/NeuronDensity"
                }
              }
            ]
          }
        },
        "should": [
          {
            "bool": {
              "must_not": [
                {
                  "exists": {
                    "field": "neuronDensity"
                  }
                }
              ]
            }
          }
        ]
      }
    },
    "must": {
      "match_all": {}
    }
  }
}