Map visualizations not working anymore #7717

Closed
renevdm opened this Issue Jul 11, 2016 · 50 comments

Projects

None yet
@renevdm
renevdm commented Jul 11, 2016 edited

Edit by @epixa
We've released a fix for this issue in 4.5.3 and 4.1.10: https://www.elastic.co/blog/kibana-4-5-3-and-4-1-10. This issue will remain open until the changes get into master, though.


Looks like since today the map visualizations are not working anymore. Only error messages are displayed. This is caused by a change of rules by MapQuest.

As of July 11, 2016, direct tile access has been discontinued.

Please visit our blog post for more information:
http://goo.gl/xB0xXt

http://devblog.mapquest.com/2016/06/15/modernization-of-mapquest-results-in-changes-to-open-tile-access/

@markwalkom
Member

Thanks for reporting this!

Discuss thread here as well - https://discuss.elastic.co/t/mapquest-change-july-11-2016/55231/4

@w33ble
Member
w33ble commented Jul 11, 2016 edited

Yeah, looks like MapQuest will now require an AppKey to use it. Thankfully they do offer a free tier in their pricing plans. We need to expose this in Kibana though, or find another tilemap provider.

I haven't looked in to whether or not you can use your own key now by using the custom providers settings.

EDIT: We may also need to move to another default provider. http://mc.bbbike.org/mc/ has a good list of OSM providers and gives you an idea of what each of their tilesets looks like.

@w33ble w33ble added bug P1 labels Jul 11, 2016
@uboness
uboness commented Jul 11, 2016

UPDATE:

We're aware that this issue currently blocks the use of tile maps in all maintained Kibana versions, and we sincerely apologize for this inconvenience.

For this reason, our No. 1 goal now is to unblock this functionality. We're currently looking at all feasible alternatives such that we can release a bug/hot fix release ASAP.

Please note that this may require us to provide a temporary solution that will be replaced in the future with another more solid one. We apologise in advance for this potential "zig zag".

We'll update this issue with our findings and as we progress. So please stay tuned.... and again... apologies!

@bryangreencharge

It's pretty low that MapQuest did this. Is there any open map platform we can use instead of MapQuest ?

@tbragin
Contributor
tbragin commented Jul 11, 2016

Just a reminder that for those organizations that have access to a WMS-compliant tile map server internally, you can configure those in Tilemap Options, starting with Kibana 4.2: https://www.elastic.co/guide/en/kibana/current/tilemap.html

There are some other public WMS servers out there as well, but we have not yet had a chance validate that any of them are appropriate alternatives.

screen shot 2016-07-11 at 4 29 17 pm

@bigjocker
bigjocker commented Jul 12, 2016 edited

We had the same issue in kibana 4.1.1-linux. We solved this by modifying the src/public/index.js file. Replace line 158877:

var tileLayer = L.tileLayer('https://otile{s}-s.mqcdn.com/tiles/1.0.0/map/{z}/{x}/{y}.jpeg', {

with

var tileLayer = L.tileLayer('http://a.tile.openstreetmap.org/{z}/{x}/{y}.png', {

This uses the tiles directly from Open Street Map's tile server. The maps look a bit different, but so far it's working ok for us, at least until we find a definitive solution.

@markwalkom
Member

@bigjocker you may be breaching the Open Street Maps Terms of Service with that - http://wiki.openstreetmap.org/wiki/Tile_usage_policy.

Please be careful.

@bigjocker

Thanks for the heads up @markwalkom. We've reviewed the tile usage policy and we don't breach any of the requirements. This is an internal tool with no heavy use, however we are in the process of installing our own tile server to prevent this from happening again.

We would gladly pay for such a service by the way, but have not found any. If anybody can point us to an alternative tile server (free or paid) we would appreciate it.

@ppisljar
Member
ppisljar commented Jul 12, 2016 edited

another list of free osm providers: http://leaflet-extras.github.io/leaflet-providers/preview/index.html

however free does not imply there are no limits ...

@asatsi
asatsi commented Jul 12, 2016

This almost broke in the middle of a demo! Is it possible to integrate google maps with Kibana? Any detailed write-ups anywhere?

@Shamanoid
Shamanoid commented Jul 12, 2016 edited

This WMS seems available :
http://www2.demis.nl/wms/wms.asp?wms=WorldMap&request=getcapabilities&version=1.0.7

The provided XML does specify it has no access constraints, maybe can it be used temporarily?

The only issue (as shown below) is that it would need to use the EPSG projection 4326 rather than the 3857, both are supported by Leaflet.

Projection 3857

Workaround from Kibana master branch:

index.js --> line 137496
Just add an option crs to mapOptions like :
var mapOptions = { minZoom: 1, maxZoom: 18, layers: tileLayer, center: self._attr.mapCenter, zoom: self._attr.mapZoom, noWrap: true, maxBounds: worldBounds, scrollWheelZoom: false, fadeAnimation: false, crs:L.CRS.EPSG4326 };

Reference : http://leafletjs.com/reference.html#map-crs

@blues-man

Hello,
I could also solve it on Kibana 4.3.0 by adding OpenStreetMap as suggested above, and modifying src/ui/public/vislib/visualizations/_map.js:11 :

    var mapTiles = {
      url: 'http://a.tile.openstreetmap.org/{z}/{x}/{y}.png',
      options: {
        attribution: 'Map data &copy; <a href="http://openstreetmap.org">OpenStreetMap</a> contributors, ' +
          '<a href="http://creativecommons.org/licenses/by-sa/2.0/">CC-BY-SA</a>'
      }
    };

Take care also of any existing mapTiles into /optimize dir

@jamesspi

Hi All,

I ran into this problem this morning, but I was able to use a different WMS provider. Settings below:

WMS URL: http://ows-tile.terrestris.de/osm-basemap/service?
WMS Layers: OSM-WMS
WMS Version: 1.1.1
WMS format: image/jpeg

image

James

@t3chn0m4g3 t3chn0m4g3 referenced this issue in dtag-dev-sec/tpotce Jul 12, 2016
Closed

Honeypot Map broken since July 11 #45

@meenalluktuke

@jamesspi Works perfectly.. Thanks 👍

@mathewmeconry

@jamesspi Thanks a lot

@german23

Any good workrounds for the Kibana 3.X Branch(i hope i did not overlook one)? I am sure a huge amount of people continue still using it

@khalanthrajeev

Thank you. working fine on checking WMS server

@rostiar
rostiar commented Jul 13, 2016

+1

@remip2
Contributor
remip2 commented Jul 13, 2016

for kibana3 (I don't know if the branch is still maintained..) remip2@9623a31

@german23

@remip2 Thank you very much, it works flawlessly (Kibana3 get maintained mostly by community sadly)

@epixa
Member
epixa commented Jul 14, 2016 edited

Sorry for the radio silence in this ticket. We've been heads down in getting a fix in for this since Monday, and we bypassed our normal workflow to get the necessary changes into official 4.5.3 and 4.1.10 releases ASAP, so the changes have not made it into master yet.

We've brought up our own tile service to use as a default for Kibana going forward, and we've made the tile service configurable so people can use other solutions if they need more robust capabilities.

The PR #7724 will be updated today to reflect all of these initial changes.

@epixa
Member
epixa commented Jul 14, 2016

We've released 4.5.3 and 4.1.10 with fixes for tile maps: https://www.elastic.co/blog/kibana-4-5-3-and-4-1-10

We'll now start getting the changes into master and backported for future releases.

@asatsi
asatsi commented Jul 15, 2016

You guys are rockstars!

@mkoertgen
mkoertgen commented Jul 15, 2016 edited

Note that for working around this in Kibana 5.x you need to replace the tilemap url template - see #7717 (comment) - in optimize/kibana.bundle.js. Then restart kibana service.

@mkoertgen
mkoertgen commented Jul 15, 2016 edited

Working around with WMS blows this for me. Needing to set up a WMS server in production adds so much bloat as compared to a simple tilemap provider.

Instead i very much appreciate PR #7724 which allows configuring the tilemap url template. Will be very happy when this gets merged in.

@kchendil

Hello,

The workaround suggested by blues-man for 4.3.0 works for 4.4.2.

Thanks
Chendil

@islandValent

@blues-man Great job, perfectly solve my problem.

@edjez edjez referenced this issue in instedd/telemetry_rails Jul 20, 2016
Closed

Maps don't work, Kibana update needed #127

@epixa epixa closed this in #7724 Jul 20, 2016
@mostolog

This issue is not fixed (yet) on kibana 5.0.0-alpha4, right?

@epixa
Member
epixa commented Jul 21, 2016

That is correct as 5.0.0-alpha4 was released prior to this issue. The fix will ship in alpha5, though.

@realynot

hi

where is the files to change into kabana3 . i don't have this file... src/ui/public/vislib/visualizations/_map.js:11

thanks

@15M92
15M92 commented Jul 28, 2016

i have installed kibana 4.5 and its dependencies now I am trying to use MapQuest consumer key along with its url in kibana.yml file in tilemap-url ,but somehow it is not working . is there anything i am missing ? i am not getting why it is not showing up? any help would be greatly appreciated .

@epixa
Member
epixa commented Jul 28, 2016

@15M92 I don't think it's possible to use mapquest's new tile service in the way you're trying to use it. They provide an entire custom javascript plugin now as the means the consume the service, which is not integrated in Kibana.

There are other leaflet-compatible providers out there that you can configure in Kibana, but if it's possible to use mapquest, we haven't figured out how.

@averri
averri commented Aug 6, 2016

I have tried with Kibana 5.0.0-alpha5 with no success.

{
  "enabled": true,
  "url": "http://ows-tile.terrestris.de/osm-basemap/service?",
  "options": {
    "version": "1.1.1",
    "format": "image/jpeg",
    "attribution": "Maps provided by USGS"
  }
}

image

It still show MapQuest:

image

@epixa
Member
epixa commented Aug 6, 2016

@averri The settings you are modifying are for the existing custom WMS server option, which has been an alternative mapping option to the tile server approach since 4.2. To use that, you would need to have access to a WMS server, and you need to explicitly enable the WMS server option for each visualization under the visualization's "Options" tab. Note that WMS is a totally different technology from the tile service approach which we default to for all visualizations. If you're just looking for a drop-in replacement for mapquest, this is not what you want.

Tile service-backed maps in 5.0.0-alpha4 are broken. 5.0.0-alpha5 hasn't been released yet, but it uses the new Elastic Tile Service instead of MapQuest. It also introduces the ability to use a different tile service entirely, which you need to configure via your kibana.yml: https://www.elastic.co/guide/en/kibana/master/kibana-server-properties.html#tilemap-settings

@averri
averri commented Aug 6, 2016

@epixa, that's great. So, let's wait for alpha5. Cheers.

@dayjavid

Just checking in - I read that the Tile Maps are fixed for 4.5.3... did that make it into 4.5.4 as well?

@epixa
Member
epixa commented Aug 18, 2016

@dayjavid Yep, it is fixed for all currently stable and future releases of Kibana.

@dayjavid

Hm.. Ok.

I've got 2 issues that are supposed to be fixed by 4.5.4, but neither of them have actually been fixed by it..
I need to investigate more on my end I guess. Thanks!

@epixa
Member
epixa commented Aug 18, 2016

Which issues? It's entirely possible something has been incorrectly labeled.

@dayjavid

The tilemaps and the Chrome 52 bug #7801

I performed an upgrade by grabbing the new tar.gz file, and running my deploy process.
The way I'm doing it, I am dropping it on top of the existing 4.4.2 deployment. My expectation was that it would overwrite the necessary stuff, but then looking back on it now (I'm actually still testing this as a theory to why it didn't fix my problems), I'm thinking that if a file was removed as part of the fix, then that file would still exist in my case, since I'm just untarring the new version on top of the old version.

A coworker actually recommended I ask about this as it might be considered a bug, but I'm not sure.

@epixa
Member
epixa commented Aug 18, 2016

Existing stray files shouldn't be a problem in theory, since they'd only be bundled in the event that a legit file imported them.

Do you have any levels of caching in between? Have you restarted Kibana?

@dayjavid
dayjavid commented Aug 18, 2016 edited

So the problems I'm experiencing seem to definitely be rooted in how I upgraded Kibana.
Upon doing a complete wipe/reinstall of Kibana 4.5.4, NOW it won't even connect to Elasticsearch because of version incompatibilities.

Before, when I upgraded by just dropping the new version on top of the old version, Kibana would start up and even display 4.5.4 as the version in the Settings -> About, but didn't seem to actually include any of the features.

Edit to answer your previous question
And yes, I tried wiping the browser cache as well as using Incognito, and all of that was after a restart.

I actually have 4.5.4 dropped onto other nodes that I haven't restarted yet because I didn't want it to go into effect yet.

@epixa
Member
epixa commented Aug 18, 2016

Was Kibana running when you did the original upgrade?

@dayjavid

It was

@epixa
Member
epixa commented Aug 18, 2016

If you're still having issues after the fresh upgrade, I recommend that you seek help on our forums: https://discuss.elastic.co/c/kibana

I just realized we've been spamming 27 subscribers with each of these comments... sorry folks!

@markygab
markygab commented Sep 1, 2016

Hi, we use Kibana through the AWS Elastic search service, which is running version 4.12. What can i do?

@markwalkom
Member

Migrate to https://www.elastic.co/cloud, it definitely works there :)

On 1 Sep 2016 12:23 PM, "Mark Gabriel" notifications@github.com wrote:

Hi, we use Kibana through the AWS Elastic search service, which is running
version 4.12. What can i do?


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#7717 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADCYTrKE0hheDMwsuL9z-lpz6UC9PDWhks5qljcHgaJpZM4JJx9H
.

@epixa
Member
epixa commented Sep 1, 2016

There is no option available on that old of a version of Kibana, sorry :-/

If you prefer a hosted solution, then I echo the previous recommendation for Elastic Cloud. You can use the latest versions of Kibana on there, which include the necessary fixes for this.

-Court

On Aug 31, 2016, at 10:23 PM, Mark Gabriel notifications@github.com wrote:

Hi, we use Kibana through the AWS Elastic search service, which is running version 4.12. What can i do?


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

@vg15
vg15 commented Oct 13, 2016

I am using kibana4.6.1 and in MAP visualization I can only see dots , background map is missing . I checked my console and I can see errors in it "failed to load resources : net::errorERR_CONNECTION_TIMED_OUT https://tiles.elastic.co/v1/default......." Please suggest what could be an issue here ?

@markwalkom
Member

@vg15 please don't comment on closed issues like this. Given you have also asked on the forums, let's continue the discussion there.

@gardart gardart referenced this issue in gardart/amazon-mws-elastic Jan 9, 2017
Open

Map visualizations not working anymore #1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment