New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Question about CORS enabled images support in ESRI Leaflet #563

Closed
pauldzy opened this Issue Jun 12, 2015 · 22 comments

Comments

Projects
None yet
4 participants
@pauldzy

pauldzy commented Jun 12, 2015

Hello,

I was looking at how to provoke a CORS response from my ArcGIS servers (10.2.2 and 10.3.1) and noticing that for what I think is the default security setup that the only way to get CORS headers Access-Control-Allow-Origin and Access-Control-Allow-Credentials is to pass an Origin header in the request.

If I just submit a request for either json or an image via
http://requestmaker.com/
say
http://watersgeo.epa.gov/arcgis/rest/services/OWPROGRAM/SDWIS_WMERC/MapServer?f=pjson
the only way to get a CORS response is by adding an Origin header (asterisk does the job).

Now in Esri-leaflet I looked for somewhere that useCors: true triggers this but don't seem to see anything anywhere. Is this an option that should be supported in the library?

Thanks,
Paul

capture

capture2

@patrickarlt

This comment has been minimized.

Show comment
Hide comment
@patrickarlt

patrickarlt Jun 12, 2015

Member

CORS will be automatically used IF it is supported by the browser. If it is not supported by the browser (IE 9 and 10) a JSONP fallback will be used. Their is a useCors option that defaults to true IF the browser supports CORS. If your server doesn't support CORS you will need to set this to false.

To see if your browser supports CORS you can check L.esri.Support.CORS if it is true CORS should be automatic. You can test it with:

// use whatever Esri Leaflet thinks is best (based on browser support)
L.esri.get('http://watersgeo.epa.gov/arcgis/rest/services/OWPROGRAM/SDWIS_WMERC/MapServer', {}, function(error, response){console.log(error, response);});

// make a CORS request
L.esri.Request.get.CORS('http://watersgeo.epa.gov/arcgis/rest/services/OWPROGRAM/SDWIS_WMERC/MapServer', {}, function(error, response){console.log(error, response);});

// make a JSONP request
L.esri.Request.get.JSONP('http://watersgeo.epa.gov/arcgis/rest/services/OWPROGRAM/SDWIS_WMERC/MapServer', {}, function(error, response){console.log(error, response);});
Member

patrickarlt commented Jun 12, 2015

CORS will be automatically used IF it is supported by the browser. If it is not supported by the browser (IE 9 and 10) a JSONP fallback will be used. Their is a useCors option that defaults to true IF the browser supports CORS. If your server doesn't support CORS you will need to set this to false.

To see if your browser supports CORS you can check L.esri.Support.CORS if it is true CORS should be automatic. You can test it with:

// use whatever Esri Leaflet thinks is best (based on browser support)
L.esri.get('http://watersgeo.epa.gov/arcgis/rest/services/OWPROGRAM/SDWIS_WMERC/MapServer', {}, function(error, response){console.log(error, response);});

// make a CORS request
L.esri.Request.get.CORS('http://watersgeo.epa.gov/arcgis/rest/services/OWPROGRAM/SDWIS_WMERC/MapServer', {}, function(error, response){console.log(error, response);});

// make a JSONP request
L.esri.Request.get.JSONP('http://watersgeo.epa.gov/arcgis/rest/services/OWPROGRAM/SDWIS_WMERC/MapServer', {}, function(error, response){console.log(error, response);});
@patrickarlt

This comment has been minimized.

Show comment
Hide comment
@patrickarlt

patrickarlt Jun 12, 2015

Member

I was able to get a CORS response from your server with the above sample code in Chrome.

Member

patrickarlt commented Jun 12, 2015

I was able to get a CORS response from your server with the above sample code in Chrome.

@pauldzy

This comment has been minimized.

Show comment
Hide comment
@pauldzy

pauldzy Jun 12, 2015

Thanks,
That makes sense and does pull the CORS response for JSON. I was confused in that I was hoping to get a CORS response also with the images that leaflet is pulling in for a dynamic layer. I was toying with the idea of storing them in a canvas object for local processing.

http://watersgeo.epa.gov/arcgis/rest/services/OW/LEGACYWBD_WMERC/MapServer/export?dpi=96&transparent=true&format=png8&bbox=-8601852.88372093%2C4690058%2C-8549275.11627907%2C4715020&bboxSR=102100&imageSR=102100&size=1268%2C602&f=image

Thanks for the fast response,
Paul

pauldzy commented Jun 12, 2015

Thanks,
That makes sense and does pull the CORS response for JSON. I was confused in that I was hoping to get a CORS response also with the images that leaflet is pulling in for a dynamic layer. I was toying with the idea of storing them in a canvas object for local processing.

http://watersgeo.epa.gov/arcgis/rest/services/OW/LEGACYWBD_WMERC/MapServer/export?dpi=96&transparent=true&format=png8&bbox=-8601852.88372093%2C4690058%2C-8549275.11627907%2C4715020&bboxSR=102100&imageSR=102100&size=1268%2C602&f=image

Thanks for the fast response,
Paul

@jgravois

This comment has been minimized.

Show comment
Hide comment
@jgravois

jgravois Jun 12, 2015

Member

CORS restrictions don't apply to requests for images

Member

jgravois commented Jun 12, 2015

CORS restrictions don't apply to requests for images

@patrickarlt

This comment has been minimized.

Show comment
Hide comment
@patrickarlt

patrickarlt Jun 12, 2015

Member

@jgravois is right you can ALWAYS request images cross domain wether or not the browser supports CORS.

If you want to intercept the URL of the image I think you could probally listen to the load event and set the layers opacity to 0 to place the image yourself.

layer = L.imageLayer(url, {
  opacity: 0
});

layer.on('load', function(){
  // do something with layer._currentImage._url and layer._currentImage._bounds
});
Member

patrickarlt commented Jun 12, 2015

@jgravois is right you can ALWAYS request images cross domain wether or not the browser supports CORS.

If you want to intercept the URL of the image I think you could probally listen to the load event and set the layers opacity to 0 to place the image yourself.

layer = L.imageLayer(url, {
  opacity: 0
});

layer.on('load', function(){
  // do something with layer._currentImage._url and layer._currentImage._bounds
});
@pauldzy

This comment has been minimized.

Show comment
Hide comment
@pauldzy

pauldzy Jun 12, 2015

Thanks for the explanation,

This is the issue I thought I was running into when scraping off the layer._currentImage._image into a canvas
https://developer.mozilla.org/en-US/docs/Web/HTML/CORS_enabled_image

When I sought to canvas.toDataURL on the canvas in order to create a file object, I would get a cross domain error. However, at this point on a Friday I realize I need to redo my example and distill out a working example of the issue. Perhaps I missed something simple..

Thanks for your time,
Paul

pauldzy commented Jun 12, 2015

Thanks for the explanation,

This is the issue I thought I was running into when scraping off the layer._currentImage._image into a canvas
https://developer.mozilla.org/en-US/docs/Web/HTML/CORS_enabled_image

When I sought to canvas.toDataURL on the canvas in order to create a file object, I would get a cross domain error. However, at this point on a Friday I realize I need to redo my example and distill out a working example of the issue. Perhaps I missed something simple..

Thanks for your time,
Paul

@patrickarlt

This comment has been minimized.

Show comment
Hide comment
@patrickarlt

patrickarlt Jun 12, 2015

Member

Now that I see what you are doing it looks like we aren't serving images from ArcGIS Server with the proper CORS headers so you might not be able to do this. I'll need to investigate more.

Member

patrickarlt commented Jun 12, 2015

Now that I see what you are doing it looks like we aren't serving images from ArcGIS Server with the proper CORS headers so you might not be able to do this. I'll need to investigate more.

@pauldzy pauldzy changed the title from Question about CORS support in ESRI Leaflet to Question about CORS enabled images support in ESRI Leaflet Jun 14, 2015

@pauldzy

This comment has been minimized.

Show comment
Hide comment
@pauldzy

pauldzy Jun 14, 2015

Hi John and Patrick,

Came back to the issue this morning. Here is a CodePen example that more succinctly shows the problem.
http://codepen.io/pauldzy/pen/dozoXb

Now it still seems to me that all you need to do is add an Origin header with the image request and then the CORS headers return fine from AGS. But I do see that this would entail digging into Leaflet itself via ImageOverlay.

Thanks,
Paul

capture

pauldzy commented Jun 14, 2015

Hi John and Patrick,

Came back to the issue this morning. Here is a CodePen example that more succinctly shows the problem.
http://codepen.io/pauldzy/pen/dozoXb

Now it still seems to me that all you need to do is add an Origin header with the image request and then the CORS headers return fine from AGS. But I do see that this would entail digging into Leaflet itself via ImageOverlay.

Thanks,
Paul

capture

@patrickarlt

This comment has been minimized.

Show comment
Hide comment
@patrickarlt

patrickarlt Jun 15, 2015

Member

I took another look at this this morning. I managed to get the sample from MDN working with just a raw image request. Take a look at this JS Bin http://jsbin.com/yudeda/1/edit?html,js,output. I haven't read the spec but I'm willing to bet that you have to set crossOrigin on the image before you set src as in the MDN example.

So I think something like this should work:

  1. Get the image URL and bounds on every load event (as per my sample above)
  2. Create a new Image
  3. Set crossOrigin
  4. Set src to be the url of the image
  5. Now draw your new image to canvas and then save it to local storage

At this point your pretty much responsible for pulling the image out of local storage and drawing it again.

Member

patrickarlt commented Jun 15, 2015

I took another look at this this morning. I managed to get the sample from MDN working with just a raw image request. Take a look at this JS Bin http://jsbin.com/yudeda/1/edit?html,js,output. I haven't read the spec but I'm willing to bet that you have to set crossOrigin on the image before you set src as in the MDN example.

So I think something like this should work:

  1. Get the image URL and bounds on every load event (as per my sample above)
  2. Create a new Image
  3. Set crossOrigin
  4. Set src to be the url of the image
  5. Now draw your new image to canvas and then save it to local storage

At this point your pretty much responsible for pulling the image out of local storage and drawing it again.

@patrickarlt

This comment has been minimized.

Show comment
Hide comment
@patrickarlt

patrickarlt Jun 15, 2015

Member

Looks like my suspicions are correct if you set crossOrigin after src you get the security warning.

Member

patrickarlt commented Jun 15, 2015

Looks like my suspicions are correct if you set crossOrigin after src you get the security warning.

@pauldzy

This comment has been minimized.

Show comment
Hide comment
@pauldzy

pauldzy Jun 15, 2015

Thanks Patrick,

So the img.crossOrigin="anonymous"; triggers the Origin header in the request which then triggers the CORS headers. So then as my goal here is to steal the image directly from Leaflet, is it possible to set the crossOrigin attribute before the layer image is requested from ArcGIS Server?

Cheers,
Paul

pauldzy commented Jun 15, 2015

Thanks Patrick,

So the img.crossOrigin="anonymous"; triggers the Origin header in the request which then triggers the CORS headers. So then as my goal here is to steal the image directly from Leaflet, is it possible to set the crossOrigin attribute before the layer image is requested from ArcGIS Server?

Cheers,
Paul

@patrickarlt

This comment has been minimized.

Show comment
Hide comment
@patrickarlt

patrickarlt Jun 15, 2015

Member

Extending the core L.ImageOverlay class would be extremely fragile but you could do it. The image element is created here https://github.com/Leaflet/Leaflet/blob/v0.7.3/src/layer/ImageOverlay.js#L83-L102

I think this would probably do the trick:

L.ImageOverlay.include({
    _initImage: function () {
        this._image = L.DomUtil.create('img', 'leaflet-image-layer');

        if (this._map.options.zoomAnimation && L.Browser.any3d) {
            L.DomUtil.addClass(this._image, 'leaflet-zoom-animated');
        } else {
            L.DomUtil.addClass(this._image, 'leaflet-zoom-hide');
        }

        this._updateOpacity();

                // add crossOrigin attribute before source
                this._image.crossOrigin  = 'anonymous';

        //TODO createImage util method to remove duplication
        L.extend(this._image, {
            galleryimg: 'no',
            onselectstart: L.Util.falseFn,
            onmousemove: L.Util.falseFn,
            onload: L.bind(this._onImageLoad, this),
            src: this._url
        });
    },
});

However this would be INCREDIBLY DANGEROUS and probably subject to breaking at every future release of Leaflet.

I would probably recommend extending L.esri.DynamicMapLayer or its base class.L.esri.RasterLayer. My recommendation would probably be to extend L.esri.DymanicMapLayer like this.

var MyCrazyCanvasLayer = L.esri.DymanicMapLayer.extend({
  //override the _renderImage function
  _renderImage: function(url, bounds) {
    // 1. load image (new Image(), set crossOrigin, set src to url)
    // 2. load the image into your canvas, manupulate it
    // 3. export your canvas to a data uri
    // 4. call the original render image function with your data URI

    // call the original _renderImage function with your new manipulated data uri
    // this will create a new L.ImageOverlay with your data uri as the url and
    // trigger all proper behavior for showing/hiding the overlays.
    // you might run into issues with the load event not firing, but I'm not sure.
    // http://stackoverflow.com/questions/4776670/should-setting-an-image-src-to-data-url-be-available-immediately
    L.esri.DymanicMapLayer._renderImage.call(this, dataUri, bounds);
  }
});
Member

patrickarlt commented Jun 15, 2015

Extending the core L.ImageOverlay class would be extremely fragile but you could do it. The image element is created here https://github.com/Leaflet/Leaflet/blob/v0.7.3/src/layer/ImageOverlay.js#L83-L102

I think this would probably do the trick:

L.ImageOverlay.include({
    _initImage: function () {
        this._image = L.DomUtil.create('img', 'leaflet-image-layer');

        if (this._map.options.zoomAnimation && L.Browser.any3d) {
            L.DomUtil.addClass(this._image, 'leaflet-zoom-animated');
        } else {
            L.DomUtil.addClass(this._image, 'leaflet-zoom-hide');
        }

        this._updateOpacity();

                // add crossOrigin attribute before source
                this._image.crossOrigin  = 'anonymous';

        //TODO createImage util method to remove duplication
        L.extend(this._image, {
            galleryimg: 'no',
            onselectstart: L.Util.falseFn,
            onmousemove: L.Util.falseFn,
            onload: L.bind(this._onImageLoad, this),
            src: this._url
        });
    },
});

However this would be INCREDIBLY DANGEROUS and probably subject to breaking at every future release of Leaflet.

I would probably recommend extending L.esri.DynamicMapLayer or its base class.L.esri.RasterLayer. My recommendation would probably be to extend L.esri.DymanicMapLayer like this.

var MyCrazyCanvasLayer = L.esri.DymanicMapLayer.extend({
  //override the _renderImage function
  _renderImage: function(url, bounds) {
    // 1. load image (new Image(), set crossOrigin, set src to url)
    // 2. load the image into your canvas, manupulate it
    // 3. export your canvas to a data uri
    // 4. call the original render image function with your data URI

    // call the original _renderImage function with your new manipulated data uri
    // this will create a new L.ImageOverlay with your data uri as the url and
    // trigger all proper behavior for showing/hiding the overlays.
    // you might run into issues with the load event not firing, but I'm not sure.
    // http://stackoverflow.com/questions/4776670/should-setting-an-image-src-to-data-url-be-available-immediately
    L.esri.DymanicMapLayer._renderImage.call(this, dataUri, bounds);
  }
});
@patrickarlt

This comment has been minimized.

Show comment
Hide comment
@patrickarlt

patrickarlt Jun 15, 2015

Member

@pauldzy Its worth noting this is all how it would work in theory. Can I ask WHY you might want to manipulate the image data in canvas? There might be a better solution.

Member

patrickarlt commented Jun 15, 2015

@pauldzy Its worth noting this is all how it would work in theory. Can I ask WHY you might want to manipulate the image data in canvas? There might be a better solution.

@pauldzy

This comment has been minimized.

Show comment
Hide comment
@pauldzy

pauldzy Jun 15, 2015

Hi Patrick,

I appreciate your help, I was not going to bore you with such things. :)

Well, I don't have a well thought out usage case. We have several ArcGIS Servers on several platforms under several different administrators drawing from many data sources with many layers having many possible configurations. Being able to say with any kind of assurance that the services and data on server A are equal to the services and data on server B are equal to the services and data on server C in the cloud can be time consuming and often comes down to a matter of trust over any kind of verification. I was toying with an idea of a prototype tool for sampling images from AGS and saving them to local storage for comparison between servers and between updates. In this scenario I would use Leaflet and Esri-Leafet to do all the work of showing and navigating to the images I would want to stash. As you pointed out I can still do this, I just need to fetch the image myself a second time after grabbing the src. However this then tied into to some larger issue of CORS, CORS on AGS and if I should be pressing my organization to move on from JSONP. CORS on AGS is pretty much undocumented so thanks for helping me figure it out.

Cheers,
Paul

pauldzy commented Jun 15, 2015

Hi Patrick,

I appreciate your help, I was not going to bore you with such things. :)

Well, I don't have a well thought out usage case. We have several ArcGIS Servers on several platforms under several different administrators drawing from many data sources with many layers having many possible configurations. Being able to say with any kind of assurance that the services and data on server A are equal to the services and data on server B are equal to the services and data on server C in the cloud can be time consuming and often comes down to a matter of trust over any kind of verification. I was toying with an idea of a prototype tool for sampling images from AGS and saving them to local storage for comparison between servers and between updates. In this scenario I would use Leaflet and Esri-Leafet to do all the work of showing and navigating to the images I would want to stash. As you pointed out I can still do this, I just need to fetch the image myself a second time after grabbing the src. However this then tied into to some larger issue of CORS, CORS on AGS and if I should be pressing my organization to move on from JSONP. CORS on AGS is pretty much undocumented so thanks for helping me figure it out.

Cheers,
Paul

@foolmoron

This comment has been minimized.

Show comment
Hide comment
@foolmoron

foolmoron Jul 1, 2015

I'm glad someone else has been trying to do this. For my app, I need to use the canvas so that I can read the color of the map pixel currently under the mouse. I've been struggling to get past this CORS issue, even after setting up the server to have the correct headers and everything.

The problem does indeed seem to be that src is being set before crossOrigin on the img elements (I'm using ArcGIS JS but it seems like the same problem is in Leaflet). Using @patrickarlt's trick to create a new Image object and then drawing that to the canvas worked perfectly, and due to browser caching it's totally fast too (400~ nanos by my measurement).

foolmoron commented Jul 1, 2015

I'm glad someone else has been trying to do this. For my app, I need to use the canvas so that I can read the color of the map pixel currently under the mouse. I've been struggling to get past this CORS issue, even after setting up the server to have the correct headers and everything.

The problem does indeed seem to be that src is being set before crossOrigin on the img elements (I'm using ArcGIS JS but it seems like the same problem is in Leaflet). Using @patrickarlt's trick to create a new Image object and then drawing that to the canvas worked perfectly, and due to browser caching it's totally fast too (400~ nanos by my measurement).

@patrickarlt

This comment has been minimized.

Show comment
Hide comment
@patrickarlt

patrickarlt Jul 1, 2015

Member

@pauldzy

I was not going to bore you with such things.

Please do. Sometimes issues can be resolved in a totally different way. It's also nice to get a view into what people are doing with the libraries and tools I am building.


@pauldzy since both you and @foolmoron have pretty legit use cases for this I'm going to pass this up to the backend teams to see if this can be enabled by default in future releases.

Member

patrickarlt commented Jul 1, 2015

@pauldzy

I was not going to bore you with such things.

Please do. Sometimes issues can be resolved in a totally different way. It's also nice to get a view into what people are doing with the libraries and tools I am building.


@pauldzy since both you and @foolmoron have pretty legit use cases for this I'm going to pass this up to the backend teams to see if this can be enabled by default in future releases.

@patrickarlt

This comment has been minimized.

Show comment
Hide comment
@patrickarlt

patrickarlt Jul 2, 2015

Member

@foolmoron @pauldzy if it really is as fast as @foolmoron suggests I could do this automatically in L.esri.RasterLayer in a future version.

Member

patrickarlt commented Jul 2, 2015

@foolmoron @pauldzy if it really is as fast as @foolmoron suggests I could do this automatically in L.esri.RasterLayer in a future version.

@foolmoron

This comment has been minimized.

Show comment
Hide comment
@foolmoron

foolmoron Jul 2, 2015

It's fast, but it does rely on the opaque browser caching behavior that might not be consistent. It's technically a new network request too. So i think ideally you would just set the crossOrigin tag before src. But it's been been working fine for my purpose so far.

foolmoron commented Jul 2, 2015

It's fast, but it does rely on the opaque browser caching behavior that might not be consistent. It's technically a new network request too. So i think ideally you would just set the crossOrigin tag before src. But it's been been working fine for my purpose so far.

@patrickarlt

This comment has been minimized.

Show comment
Hide comment
@patrickarlt

patrickarlt Jul 2, 2015

Member

So i think ideally you would just set the crossOrigin tag before src.

I've opened a PR for this on Leaflet. Leaflet/Leaflet#3594 so once that gets merged in I'll update Esri Leaflet to pass the crossOrigin option and then this will actually work like like everyone expects.

At this point it would be REALLY trivial to just add a function to grab the current pixel from the image.

Member

patrickarlt commented Jul 2, 2015

So i think ideally you would just set the crossOrigin tag before src.

I've opened a PR for this on Leaflet. Leaflet/Leaflet#3594 so once that gets merged in I'll update Esri Leaflet to pass the crossOrigin option and then this will actually work like like everyone expects.

At this point it would be REALLY trivial to just add a function to grab the current pixel from the image.

@foolmoron

This comment has been minimized.

Show comment
Hide comment
@foolmoron

foolmoron Jul 2, 2015

👍 As a temporary thing I'd say go for it.

foolmoron commented Jul 2, 2015

👍 As a temporary thing I'd say go for it.

@pauldzy

This comment has been minimized.

Show comment
Hide comment
@pauldzy

pauldzy commented Jul 3, 2015

Thanks!

@patrickarlt patrickarlt added the 1.0 label Jul 12, 2015

@jgravois

This comment has been minimized.

Show comment
Hide comment
@jgravois

jgravois Jul 13, 2015

Member

safe to close after fix in 85732a5 is merged from leaflet-1.0 into master

Member

jgravois commented Jul 13, 2015

safe to close after fix in 85732a5 is merged from leaflet-1.0 into master

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment