Obtain patch level metadata (e.g. geospatial bounds and cloud cover), save and demo DEP use case (sim search)

[lillythomas]

This PR will work towards a demonstration of how to obtain patch level embeddings and write them to GeoParquet files to run similarity search with.

Main tasks that need to be done:

• Given the geospatial bounds of a 512x512 parent chip, obtain the child 32x32 geospatial bounding boxes (patches)
• Map the child bounding boxes to patch level embeddings (making sure embeddings_level = "patch")
• Create geodataframes for the patch embeddings with their mapped bbox and write to GeoParquet
• Load in a lanceDB
• Use the ground truth points from Digital Earth Pacific applications #140 Digital Earth Pacific applications #140 (reply in thread) mineral extraction training data as reference for similarity search against all patch embeddings
• compute and save patch level cloud cover percentages (+ add to db table for sim search)

Reference tickets: #168 #140

[lillythomas]

The notebook docs/tutorial_digital_earth_pacific_patch_level.ipynb walks through an example of:

• generating patch level embeddings for an area where known mining extraction events occur
• saving the patch level embeddings to independent GeoParquet files
• executing similarity search based on a ground truth point's overlapping patch

@weiji14 @yellowcap ready for when you have time to review.

[weiji14]

Thanks @lillythomas! I haven't looked too closely yet, but would it be possible to show where the similarity search results are located? Maybe something like showing the bounding boxes of all the patches on a map, and also overlay where the original quarry points are.

[lillythomas]

Thanks @lillythomas! I haven't looked too closely yet, but would it be possible to show where the similarity search results are located? Maybe something like showing the bounding boxes of all the patches on a map, and also overlay where the original quarry points are.

Yes! Great idea. Working on this tomorrow.

[yellowcap]

Thanks Lilly, this is great to see! I did not yet have time get to the sim search part, but will get back to it on Monday. Left some comments for now.

Could you add some more context on what the purpose is? Do you think we going to apply this to a region for searching certain events / features?

[yellowcap]

Use url for this, then it becomes reproducible

https://raw.githubusercontent.com/digitalearthpacific/mineral-resource-detection/d117d04703f77ff21c15c7ffc424c3c55b51c492/training_data/draft_inputs/MRD_dissagregated_25.geojson

[yellowcap]

Same here, if we use huggingface its reproducible

https://huggingface.co/made-with-clay/Clay/resolve/main/Clay_v0.1_epoch-24_val-loss-0.46.ckpt

[yellowcap]

Why making one pdf file per embedding and not one that contains all? This would make everything downstream much easier. Would it be possible to create one before the loop and add rows to it?

[yellowcap]

This did not work for me, replaced with the following to get it to work

gdf["box"] = [box(*geom.bounds) for geom in gdf.geometry]

[lillythomas]

Oh interesting. It works on my end. Do you have the traceback?

[yellowcap]

The lambda function is no longer necessary if comment above is applied

[yellowcap]

Here I got the following error.

ArrowInvalid: Could not convert <POLYGON ((177.507 -17.778, 177.507 -17.775, 177.504 -17.775, 177.504 -17.77...> with type Polygon: did not recognize Python value type when inferring an Arrow data type

[lillythomas]

This was introduced by making your suggested change in #172 (comment) but I have a way to handling it. Essentially pyarrow doesn't like a polygon object, so I can get a list equivalent for the table by adding the bounds method to "box": row["box"] e.g. "box": row["box"].bounds, which I am in favor of doing as it allows us to drop the lamba function as you pointed out.

[yellowcap]

I would drop this from this PR seems not necessary for the notebook to work.

[lillythomas]

True. Will do.