Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tiler module #41

Merged
merged 15 commits into from
Nov 17, 2023
Merged

Tiler module #41

merged 15 commits into from
Nov 17, 2023

Conversation

lillythomas
Copy link
Contributor

@lillythomas lillythomas commented Nov 16, 2023

Tiling module that generates 256x256 tiled xarray datasets from the larger scene-level datacube generated in datacube.py.

To do:

  • add cloud filter
  • refine extent to tile within to valid data region
  • add main script to call the datacube module followed by the tile module
  • decide on method for storing the tiled xarray datasets
  • add some docstrings
  • plot tiles to make sure they look good

@yellowcap
Copy link
Member

FYI I am using the following snippet to plot the merged data for visual cross-checks:

def plot(merged):
    """
    Plot a tile in RGB and vh, and DEM
    """
    import numpy
    from matplotlib import pyplot as plt

    # data = merged.coarsen(x=100, y=100, boundary="trim").mean()
    data = merged

    fig, axs = plt.subplots(3)

    rgb = (
        numpy.array(
            [
                data.get("B04"),
                data.get("B03"),
                data.get("B02"),
            ]
        ).clip(0, 3000)
        / 3000
    )
    axs[0].imshow(rgb.transpose(1, 2, 0))
    axs[1].imshow(numpy.log(data.get("vh")))
    axs[2].imshow(data.get("DEM"))

    plt.show()

Copy link
Member

@yellowcap yellowcap left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Haven't had time to test this, was focused on fixing the cube. But looks good from a high level.

scripts/datacube.py Outdated Show resolved Hide resolved
scripts/tile.py Outdated Show resolved Hide resolved
scripts/tile.py Outdated Show resolved Hide resolved
scripts/tile.py Outdated Show resolved Hide resolved
@lillythomas
Copy link
Contributor Author

The tiler function seems to work nicely. Here is a plotted result [RGB, VH, DEM] in order from top to bottom.

Screen Shot 2023-11-17 at 12 30 24 PM

@lillythomas
Copy link
Contributor Author

Added a catch for sampled S2 reference dates which don't have S1 scenes within the +/- 3 day surrounding interval. This should prevent any failures due to unavailable matchup as it just triggers a new sampling for S2.

@lillythomas
Copy link
Contributor Author

Going to merge this to keep things moving. Will address the tile saving method in a follow-up PR.

@lillythomas lillythomas merged commit 6d0de06 into main Nov 17, 2023
1 of 2 checks passed
@lillythomas lillythomas deleted the tiler branch November 17, 2023 21:02
@weiji14 weiji14 added the data-pipeline Pull Requests about the data pipeline label Nov 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
data-pipeline Pull Requests about the data pipeline
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants