Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix thumbnails and outline geometry #205

Merged
merged 8 commits into from Jan 5, 2021
Merged

Conversation

banesullivan
Copy link
Contributor

@banesullivan banesullivan commented Nov 10, 2020

Resolve #176

For future reference, use the following example rasters to verify that the thumbnails are working.

  • LC08_L1TP_033032_20200422_20200508_01_T1_pixel_qa.tif
  • landcover_sample_2000.tif
  • vegdri_emodis_week34_082320.tif

Summary of change:

  • new admin actions to reprocess image entries and raster entries (calls save to trigger save event and recreate the thumbnail or spatial reference)
  • added a helper method to reproject an open rasterio raster to a given ESPG
  • During image thumbnail generation routine: reproject raster to Web Mercator before extracting if the image file has a spatial reference
  • During raster outline/valid footprint calculation:
    • first, reproject the raster to the DB's spatial reference (something was off when transforming the extracted geometry)
    • use rasterio/GDAL to extract a mask and create the GeoJSON feature (as opposed to our own convex hull approach)
  • Raster detail view page now limits the view bounds on the map
  • Update centroids in the raster tests since we are extracting the geometry properly now

@banesullivan
Copy link
Contributor Author

The thumbnail is still a bit off... I have confidence in the outline and the convex hull though:

example zoomed
Screen Shot 2020-11-09 at 9 50 17 PM Screen Shot 2020-11-09 at 9 50 26 PM

Note how the image shows the roads that should be next to the boundary, just not in the right spot. This is why I have confidence in the footprint/boundary and not the thumbnail

@banesullivan banesullivan changed the title [WIP] transform rasters on load [WIP] fix thumbnail on map Nov 18, 2020
@banesullivan banesullivan changed the title [WIP] fix thumbnail on map fix thumbnail on map Dec 30, 2020
@banesullivan
Copy link
Contributor Author

Currently struggling to improve this any further. It works really well for most of the data in #221 but still has a slight issue with the larger rasters like for Landsat:

Screen Shot 2020-12-30 at 3 57 52 PM

or this continental-scale one:

Screen Shot 2020-12-30 at 4 01 06 PM

@banesullivan
Copy link
Contributor Author

For the really big raster, our outline is the same as what is shown on EarthExplorer, but I'm not sure how to correctly display the thumbnail:

Screen Shot 2020-12-30 at 4 19 03 PM

@banesullivan banesullivan marked this pull request as ready for review December 30, 2020 21:22
@manthey
Copy link
Contributor

manthey commented Dec 30, 2020

Based on your continental image example, it looks like you aren't reprojecting the image when you make the thumbnail. I think you have to either reproject the image into web mercator (to match the map) OR draw the map in the projection of the image. For single thumbnails, changing the map projection might be easiest (e.g., https://opengeoscience.github.io/geojs/examples/reprojection/).

I'll note that the smaller the footprint, the more closely a bounding quadrangle with approximate the local reprojection of the image, which is why this looks better on small images.

@banesullivan
Copy link
Contributor Author

banesullivan commented Jan 4, 2021

you have to either reproject the image into web mercator (to match the map)

I can reproject the raster to Web Mercator when extracting the thumbnail and get something a little better, but the geometry of the quad which the image is overlain is still incorrect.

Screen Shot 2021-01-04 at 9 09 38 AM

I tried transforming the geometry to ESPG:3857, but GeoJS throws an error with that geometry. i.e.:

self.object.footprint.transform(3857, clone=True)

for _SpatialDetailView in rgd/geodata/views.py

OR draw the map in the projection of the image.

I also tried setting the projection of the GeoJS map to what we use in the database but it doesn't seem to work:

let map = geo.map({
      node: '#map',
      clampBoundsX: true
    })
basemapLayer = map.createLayer('osm', {
      source: 'osm',
      gcs: 'EPSG:4326'
    });

Screen Shot 2021-01-04 at 9 08 15 AM

@banesullivan
Copy link
Contributor Author

Aha! managed to get it right with some nuanced transforms. Something was going awry with our transform_geometry helper.

Screen Shot 2021-01-04 at 11 54 49 AM

Now I just need to confirm this is working for all our examples

@banesullivan
Copy link
Contributor Author

Local changes confirmed working with the original problematic raster:

Screen Shot 2021-01-04 at 11 57 47 AM

@banesullivan
Copy link
Contributor Author

banesullivan commented Jan 4, 2021

@manthey, do you think I should cache a few thumbnails on DKC and add a regression test to verify these stay consistent moving forward?

@banesullivan banesullivan changed the title fix thumbnail on map fix thumbnails and outline geometry Jan 4, 2021
@@ -35,6 +45,9 @@

logger = get_task_logger(__name__)

GDAL_DATA = os.path.join(os.path.dirname(rasterio.__file__), 'gdal_data')
os.environ['GDAL_DATA'] = GDAL_DATA
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might need to also set a PROJ env variable for NTIF format issue in _reproject_raster

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will look into this later.

@banesullivan banesullivan merged commit 4031f86 into master Jan 5, 2021
@banesullivan banesullivan deleted the patch/thumbnails branch January 5, 2021 20:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Raster thumbnails not aligned correctly on GeoJS map
2 participants