Enabler: Support multiple sites for DOC collection

[blairlearn]

Allow searches using the DOC collection to request results from multiple sites.

As presently implemented, when the DOC collection is specified, the user must supply exactly one site to return results for.

Some of the DOCs have more than one site which should show up in the results. (e.g. Search results on a DCEG microsite should return results for dceg.cancer.gov, and www.cancer.gov/connect-prevention-study.) There should be no hard limit on the number of sites.

Existing site configurations must not require updates in order to continue working.

A preliminary list of tasks:

• Create an updated query.
• Create unit tests
• Create integration tests for the new behavior (Do not break/modify the existing tests)
  This will require coordination with product owners to determine success criteria.
• Update the code.

ESTIMATE TBD

Resources:

Prerequisites

Sub-Tasks

• Short Task Description - Issue #9999
• Short Task Description - Issue #9999

Notes

DCCPS Site to index in Search
DCCPS Search - Sites to Index.xlsx

[blairlearn]

The DOC search queries for both English and Spanish use the same sub-query to eliminate results which aren't from sites matching the site parameter.

    "bool": {
        "must": [
            { "exists": { "field": "searchtitle" } },
            { "prefix": { "searchurl.raw": { "value": "<SITE>" } } }
        ]
    }

where <SITE> is the hostname and path to match (e.g. "physics.cancer.gov" or "www.cancer.gov/rare-brain-spine-tumor/espanol").

To allow multiple sites, this query is modified to

1. Remove the prefix query from the must block.
2. Add a should block, containing a minimum of one prefix query.
3. Add "minimum_should_match": 1 to require that at least one* of the prefix queries be matched.

* By default, if a bool query contains a must, then none of the should clauses are required to match. By specifying "minimum_should_match": 1, we require at least one match. (i.e. Only the ones that actually match the site parameter.)

Match for a single site

"bool": {
    "must": [
        { "exists": { "field": "searchtitle" } }
        
    ],
    "should": [
        { "prefix": { "searchurl.raw": { "value": "dceg.cancer.gov" } } }
    ],
    "minimum_should_match": 1
}

Match for multiple sites

"bool": {
    "must": [
        { "exists": { "field": "searchtitle" } }
        
    ],
    "should": [
        { "prefix": { "searchurl.raw": { "value": "dceg.cancer.gov" } } },
        { "prefix": { "searchurl.raw": { "value": "www.cancer.gov/connect-prevention-study" } } }
    ],
    "minimum_should_match": 1
}

[blairlearn]

The URL change for this is very straightforward. The site argument is already retrieved from the query string, so passing multiple values is just a matter of additional query string parameters (e.g. site=dceg.cancer.gov&site=www.cancer.gov/connect-prevention-study)

In SearchController::Get:

• The site parameter changes from string to string[] site = null (because this is already an optional parameter, it must continue as such).
• Additional validation logic is required to handle the cases where either site is null, or any of the values passed as site are null or whitespace.

Additional logic is required in the service classes to change their siteFilter from string to string[] and incorporate the query change outlined in the previous comment.

[blairlearn]

QA Notes:

This update changes the way search results are filtered.

Previously, your options were "All the results for the search string" or "Only the results that came from a specific site" (e.g. "Only results from physics.cancer.gov" or "Only results from www.cancer.gov/nano").

This change allows the filtering to be multiple sites (e.g. "Only results that came from either dceg.cancer.gov or `www.cancer.gov/connect-prevention-study'")

Given param site = "all"
Then expect results to be from multiple sites

Given param site = "www.cancer.gov/nano"
And param collection = "doc"
Then expect results.url to only start with "www.cancer.gov/nano"

Given param site = "physics.cancer.gov"
And param collection = "doc"
Then expect results.url to only start with "physics.cancer.gov"

Given params { site: ["dceg.cancer.gov", "www.cancer.gov/connect-prevention-study"] }
And param collection = "doc"
Then expect results.url to only start with "dceg.cancer.gov" or "www.cancer.gov/connect-prevention-study"

It's a bit weedy, but for that fourth case, the query string should be site=dceg.cancer.gov&site=www.cancer.gov%2Fconnect-prevention-study