Change product finding to handle lists of products

[alexgleith]

Reason for this pull request

Searching for datasets by including a list of products is possible, this makes it accessible using the normal query method.

Proposed changes

• Use index.products.search(product=product) to find products. Tested with a string or a list of strings locally.

• Closes TypeError: unhashable type: 'list' when making fused wofs and fc cache odc-tools#366

• Tests added / passed. NOT A GOOD TEST. ANY IDEAS ON A BETTER ONE?

• Fully documented, including docs/about/whats_new.rst for all changes

[codecov]

Codecov Report

Merging #1201 (7f4ee44) into develop (d268f49) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff            @@
##           develop    #1201   +/-   ##
========================================
  Coverage    93.79%   93.79%           
========================================
  Files          102      102           
  Lines        10403    10406    +3     
========================================
+ Hits          9757     9760    +3     
  Misses         646      646           

Impacted Files	Coverage Δ
datacube/api/query.py	94.78% <100.00%> (-0.03%)	⬇️
datacube/index/_products.py	94.48% <100.00%> (+0.15%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d268f49...7f4ee44. Read the comment docs.

[jeremyh]

Tests added / passed. NOT A GOOD TEST. ANY IDEAS ON A BETTER ONE?

Yeah, I'm wary of Mock tests. It's too easy for them to miss the actual useful parts to test (and be tied to internals, so break easily).

The integration_tests folder contains fixtures that create real indexes to test/query against.

For example, something like this should reproduce it, using the fixtures inside (integration_tests/index/test_search.py):

def test_query_dataset_multi_product(index: Index,  ls5_dataset_w_children: Dataset):
    # We have one ls5 level1 and its child nbar
    dc = Datacube(index)

    # Can we query a single product name?
    datasets = dc.find_datasets(product='ls5_nbar_scene')
    assert len(datasets) == 1

    # Can we query multiple products?
    datasets = dc.find_datasets(product=['ls5_nbar_scene', 'ls5_level1_scene'])
    assert len(datasets) == 2

[SpacemanPaul]

Looks good to me.

[SpacemanPaul]

Oh, and does this need an entry in whats_new?

[alexgleith]

Wait, I meant to push another commit. Hold on.

[alexgleith]

I don't think it needs to go in what's new, @SpacemanPaul as it doesn't actually change anything... and it's an internal API fix.

[Kirill888]

I think this is good enough fix for right now, but a "proper" fix might require handling the case where product is not supplied at all, but implied via other parameters of the query. But given that almost no-one uses that mode of operation and I don't even fully understand what Query class is doing with the product (removes some search terms from the query if some products are 3d?) there is certainly no rush to implement that.

[Kirill888]

Can we also handle the case when multiple products are supplied as a tuple?

datacube-core/datacube/index/_products.py

Lines 333 to 334 in d268f49

	def _listify(v):
	return v if isinstance(v, list) else [v]

will require fixing code above

def _listify(v):
    return [v] if isinstance(v, str) else [*v]

[alexgleith]

I handled it differently. Thoughts @Kirill888 ?

[Kirill888]

@alexgleith I don't see it being handled? I'm talking about supplying dc.load(product=("a", "b")) instead of dc.load(product=["a", "b"]), code you changed will handle both cases, but tuple version will break inside code that hasn't been touched by this PR

[alexgleith]

Ok, tests passing and added the tuple handling @Kirill888.

Bump @SpacemanPaul for a re-review please?