Co-registration of the images with AROSICS

[kvos]

Co-registration of Landsat and Sentinel-2 images

Objective

The idea of this functionality would be to co-register the .tif files downloaded from GEE, then read them into CoastSeg/CoastSat.
This feature would improve the relative georeferencing of the images and consequently the accuracy of products derived from the imagery (shoreline change, land cover change ...etc).

Method

One great toolbox to do this is the Arosics Python Library, which is capable of performing automatic sub-pixel co-registration.
The AROSICS toolbox has been implemented in the higher-resolution PlanetScope shoreline mapping toolbox https://github.com/ydoherty/CoastSat.PlanetScope, but should be also capable of co-registering Landsat/Sentinel-2 images as demonstrated on the tool's documentation and corresponding paper.

Challenges

One big challenge is that the water pixels need to be masked are they are not stationary and cannot be used as target points between images. The reference image also needs to be selected, if possible automatically (cloud-free and noise free). The are also some general parameters to fine-tune before applying it generally (grid size, global or local co-registration). Computational cost may also be an issue, probably proportional to the .tif file size.

[ddbuscombe]

I think this is an excellent idea and clearly within the scope of CoastSeg. Thanks for this @kvos

What would be used as the reference imagery? Does a user pick a suitable image from their downloaded collection, or would it be better to standardize the approach? A standardized approach would lookup a reference image based on location. This could be any basemap imagery. In the United States, NAIP might be a good option, for example.

I see advantages to a standardized approach:
a) because CoastSeg pre-loads a shoreline (and eventually transects) from memory, it would be consistent in design principles to also pre-load baseline imagery for the purposes of registration
b) it is one less step for the user
c) it is far more reproducible (a user may choose an arbitrary image from a collection and not document the filename or reasoning behind the choice)

A small update on CoastSeg progress:

1. The ROI/image download workflows are working in draft form
2. @2320sharon is building a companion package, Sniffer, that will be used to manually weed out bad images, and train a ML model to batch process for large sets of images. This is the current focus
3. Next we could start work on the image registration module. We could start with breaking out the implementation within CoastSat.ps into a stand-alone script to explore functionality and options before creating a new CoastSeg module.

We could use that stand-alone implementation to explore the issues identified by @kvos:

1. cloud masking (including over- and under-masking?): If we use baseline imagery, there are no issues of clouds and noise. However, it is generally higher-resolution so may have to be downsampled first? E.g. from 1-m NAIP to 15-m?
2. How does CoastSat.ps decide to use global versus local methods (I didnt look at the code too closely and havent yet tried CoastSat.ps), and what parameters can be learned from the data, versus hard-coded?
3. computational cost: is it time to move to xarray and dask?

[kvos]

sounds like it's progressing fast! keen to check out Sniffer.

1. In CoastSat.ps the user manually picks the reference image from the downloaded images, so the reference image is at the same resolution. Great idea to use basemaps as reference image, I don't know if the algorithm can handle varying resolution images (maybe?), but if it does, using a baseline image would make the workflow more automated and standardised.
2. I need to check with Yarran why he chose local coreg over global coreg.
3. Computational cost was high on Planets imagery, but hopefully it's lower for the medium resolution Landsat/Sentinel-2 imagery. Keen to learn more about xarray, not very familiar with it.

Kristen and I are looking for a student to work on implementing the co-registration as part of a Bachelor thesis., will let you know when someone will start working on this.

[ddbuscombe]

Still interested in this co-registration topic @kvos any progress with recruiting a student to work on this?

I just opened a new Discussion item about building a benchmark to test (and help improve) CoastSeg shoreline identification. It occurs to me that co-registration could be implemented in parallel

[kvos]

hey @dbuscombe-usgs , yes we currently have a student that has started to work on this recently (with the AROSICS library). I haven't seen any results yet.
On the benchmarking, I agree that's what we really need to make progress on the shoreline mapping. I have started a repo where I am compiling the different publicly available shoreline datasets and was planning to organise a kind of "challenge/workshop" where everybody can submit satellite-derived shorelines obtained with their algorithm for these sites. Would be great to have CoastSeg there too. It's work in progress but you can check what's there for now (Narrabeen and Duck): https://github.com/kvos/SDS_Challenge/blob/main/notebooks/1_preprocess_datasets.ipynb

[ddbuscombe]

Awesome! Good idea, we can certainly help with the organization if helpful, and I agree it would be good to have CoastSeg there too, once we have a fully functioning workflow!

[ddbuscombe]

Still thinking about this as a 'V2' coastseg feature. From CoastSat.PlanetScope

https://github.com/ydoherty/CoastSat.PlanetScope/blob/main/coastsat_ps/preprocess_tools.py

from arosics import COREG_LOCAL, DESHIFTER, COREG

and

https://github.com/ydoherty/CoastSat.PlanetScope/blob/main/coastsat_ps/preprocess_tools.py#L266

called by

https://github.com/ydoherty/CoastSat.PlanetScope/blob/f29501ab8e6991976dea7c19dede889c5426f35b/coastsat_ps/preprocess.py#L535

Requires a reference image im_reference. How does this image get selected? Just leaving this here for the future