Offline Map for Unity

[dorukeker]

Hello MapBox Team,

First of thank you for the good work and thank you for the support in Unity. Much appreciated.

We are implementing a map based application and we need support for implementing additional caching.

First; below thing I read and understand in your documentation:

1. I cannot make a build with pre-cached tiles. That is against your TOC
2. As you use the app and view different tiles, they are automatically cahced (with a limit of 50MB)
3. There are clear documents explaining how this can be done for iOS SDK and Android SDK. But not for the Unity SDK.

What we want implement is a function with the parameters of the bounding coordinates, zoom level and map style. I call this function when there is internet connection. It will download and cached the given area in the given zoom level in the given style. And when the download is finished it will throw a success event. So those tiles will be available when offline.

Can you direct us to the correct place of the documentation so we can implement this functionality?

Thanks in advanced.
Cheers,
Doruk

[nilsk123]

I second this issue. Offline functionality is vital for our use case. We would like to download tiles for a given bounding box and zoom level in a splashscreen like setting.

[jordy-isaac]

The Unity SDK currently has ambient caching. We're looking into a more robust way of supporting offline maps based on the interest of such a feature. However, we don't have a timeline for it yet.

[brnkhy]

Hello @dorukeker and @nilsk123,
SDK itself doesn't have helper functions like that at the moment but I created something quick to show the idea here; https://github.com/mapbox/mapbox-unity-sdk/tree/TilePrefetching
In the TileCacher script, you can see I'm fetching tiles independent of the map/abstract map modules. All fetched tiles are cached automatically so that region will be available later from cache.
It's kinda limited at the moment but we'll improve it in the future and probably add it to sdk as a feature.

[ejgutierrez74]

I would second this, as you can define a cache size in mapbox settings, you can assign to an abstract map a offline size, so you can have from origin of the map

In #1299 and #1296, im facing problems.
Another solution would be to have a lodedmap boolean, when you can check if the game have been loaded or not.

Ive very wierd bugs related, for example the same code, in one ubuntu 18.04 works fine, but in another one it doesnt work because map is not loaded, i cant guess the reason.

@jordy-isaac id mail you this week about the bug as you told me

[AnushaFatima]

Hi,

I am also looking for a solution for offline maps access. The purpose is to build an app for UWP , hololens platform. Is this support available?

[dorukeker]

Hello All,

Following @brnkhy demo script we implemented the tile caching in our project.
We made some changes to the script file to fit to our project. Below I share the script.

Please not this is not a one-size-fits-all solution and there are parts that are tightly coupled to our project. But it would give an idea for implementation.

Another note before the code:
When you implement the script Unity will give errors in some files form the SDK. To solve this:

• Check out the branch mentioned in @brnkhy post
• Replace the files with this check out

I hope this helps someone.
Cheers,
Doruk

using System;
using System.Collections;
using System.Collections.Generic;
using Mapbox.Map;
using Mapbox.Unity.MeshGeneration.Data;
using Mapbox.Unity.Utilities;
using Mapbox.Utils;
using UnityEngine;
using UnityEngine.UI;

public class TileCacher : MonoBehaviour
{
    public static TileCacher current;

    public enum Status{
        ALL_CACHED,
        SOME_CACHED,
        ALL_FALIED,
        NOTHING_TO_CAHCE
    }
    public delegate void TileCacherEvent(Status result , int FetchedTileCount);

    [Header("Area Data")]
    public List<string> Points;
    public string ImageMapId;
    public int ZoomLevel;

    [Header("Output")]
    public float Progress;
    [TextArea(10,20)]
    public string Log;
    private ImageDataFetcher ImageFetcher;
    private int _tileCountToFetch;
    private int _failedTileCount;
    [SerializeField] private int _currentProgress;
    private Vector2 _anchor;
    [SerializeField] private Transform _canvas;
    [SerializeField] bool DoesLog = false;
    [SerializeField] bool DoesRender = false;
    [SerializeField] Image progressBarImage;
    public event TileCacherEvent OnTileCachingEnd;

    void Awake() { current = this;}
    private void Start()
    {
        // ImageFetcher = new ImageDataFetcher();
        ImageFetcher = ScriptableObject.CreateInstance<ImageDataFetcher>();
        ImageFetcher.DataRecieved += ImageDataReceived;
        ImageFetcher.FetchingError += ImageDataError;
    }

    public void CacheTiles(int _zoomLevel, string _topLeft, string _bottomRight){
        ZoomLevel = _zoomLevel;
        Points = new List<string>();
        Points.Add(_topLeft);
        Points.Add(_bottomRight);
        PullTiles();
    }


    [ContextMenu("Download Tiles")]
    public void PullTiles()
    {
	    Progress = 0;
	    _tileCountToFetch = 0;
	    _currentProgress = 0;
        _failedTileCount = 0;

        var pointMeters = new List<UnwrappedTileId>();
        foreach (var point in Points)
        {
            var pointVector = Conversions.StringToLatLon(point);
            var pointMeter = Conversions.LatitudeLongitudeToTileId(pointVector.x, pointVector.y, ZoomLevel);
            pointMeters.Add(pointMeter);
        }

        var minx = int.MaxValue;
        var maxx = int.MinValue;
        var miny = int.MaxValue;
        var maxy = int.MinValue;

        foreach (var meter in pointMeters)
        {
            if (meter.X < minx)
            {
                minx = meter.X;
            }

            if (meter.X > maxx)
            {
                maxx = meter.X;
            }

            if (meter.Y < miny)
            {
                miny = meter.Y;
            }

            if (meter.Y > maxy)
            {
                maxy = meter.Y;
            }
        }

        // If there is only one tile to fetch, this makes sure you fetch it
        if(maxx == minx){
            maxx++;
            minx--;
        }

        if(maxy == miny){
            maxy++;
            miny--;
        }

        _tileCountToFetch = (maxx - minx) * (maxy - miny);
        if(_tileCountToFetch == 0){
            OnTileCachingEnd.Invoke(Status.NOTHING_TO_CAHCE , 0);
        } else {
            _anchor = new Vector2((maxx + minx) / 2, (maxy + miny) / 2);
            PrintLog(string.Format("{0}, {1}, {2}, {3}", minx, maxx, miny, maxy));
            StartCoroutine(StartPulling(minx, maxx, miny, maxy));
        }
    }

    private IEnumerator StartPulling(int minx, int maxx, int miny, int maxy)
    {

        for (int i = minx; i < maxx; i++)
        {
            for (int j = miny; j < maxy; j++)
            {

                ImageFetcher.FetchData(new ImageDataFetcherParameters()
                {
                    canonicalTileId = new CanonicalTileId(ZoomLevel, i, j),
                    mapid = ImageMapId,
                    tile = null
                });

                yield return null;
            }
        }
    }

    #region Fetcher Events

    private void ImageDataError(UnityTile arg1, RasterTile arg2, TileErrorEventArgs arg3)
    {
        PrintLog(string.Format("Image data fetching failed for {0}\r\n",  arg2.Id));
        _failedTileCount++;
    }

    private void ImageDataReceived(UnityTile arg1, RasterTile arg2)
    {
        _currentProgress++;
	    Progress = (float)_currentProgress / _tileCountToFetch * 100;
        if(progressBarImage != null && progressBarImage.gameObject.activeInHierarchy) progressBarImage.fillAmount = Progress / 100;
        RenderImagery(arg2);
        if(Progress == 100) CheckEnd();
    }
    #endregion

    #region Utility Functions
    private void CheckEnd(){
        if(OnTileCachingEnd != null){
            if(_failedTileCount == 0){
                OnTileCachingEnd.Invoke(Status.ALL_CACHED , _tileCountToFetch);
            } else if(_failedTileCount == _tileCountToFetch){
                OnTileCachingEnd.Invoke(Status.ALL_FALIED , 0);
            } else if(_failedTileCount > 0 && _failedTileCount < _tileCountToFetch){
                OnTileCachingEnd.Invoke(Status.SOME_CACHED ,  _tileCountToFetch - _failedTileCount);
            }
        }
    }
    private void RenderImagery(RasterTile rasterTile)
    {
        if(!DoesRender || _canvas == null || !_canvas.gameObject.activeInHierarchy) return;

        GameObject targetCanvas = GameObject.Find("canvas_" + ZoomLevel);
        if(targetCanvas == null){
            targetCanvas = new GameObject("canvas_" + ZoomLevel);
            targetCanvas.transform.SetParent(_canvas);    
        }

        var go = new GameObject("image");
        go.transform.SetParent(targetCanvas.transform);
        var img = go.AddComponent<RawImage>();
        img.rectTransform.sizeDelta = new Vector2(10,10);
        var txt = new Texture2D(256,256);
        txt.LoadImage(rasterTile.Data);
        img.texture = txt;
        (go.transform as RectTransform).anchoredPosition = new Vector2((float)(rasterTile.Id.X - _anchor.x) * 10, (float)-(rasterTile.Id.Y - _anchor.y) * 10);
    }
    private void PrintLog(string message){
        if(!DoesLog) return;
        Log += message;
    }
    #endregion
}

[brnkhy]

Thanks a lot for sharing the code @dorukeker !

[StarKing777]

Hello All,

Following @brnkhy demo script we implemented the tile caching in our project.
We made some changes to the script file to fit to our project. Below I share the script.

Please not this is not a one-size-fits-all solution and there are parts that are tightly coupled to our project. But it would give an idea for implementation.

Another note before the code:
When you implement the script Unity will give errors in some files form the SDK. To solve this:

• Check out the branch mentioned in @brnkhy post
• Replace the files with this check out

I hope this helps someone.
Cheers,
Doruk

using System;
using System.Collections;
using System.Collections.Generic;
using Mapbox.Map;
using Mapbox.Unity.MeshGeneration.Data;
using Mapbox.Unity.Utilities;
using Mapbox.Utils;
using UnityEngine;
using UnityEngine.UI;

public class TileCacher : MonoBehaviour
{
    public static TileCacher current;

    public enum Status{
        ALL_CACHED,
        SOME_CACHED,
        ALL_FALIED,
        NOTHING_TO_CAHCE
    }
    public delegate void TileCacherEvent(Status result , int FetchedTileCount);

    [Header("Area Data")]
    public List<string> Points;
    public string ImageMapId;
    public int ZoomLevel;

    [Header("Output")]
    public float Progress;
    [TextArea(10,20)]
    public string Log;
    private ImageDataFetcher ImageFetcher;
    private int _tileCountToFetch;
    private int _failedTileCount;
    [SerializeField] private int _currentProgress;
    private Vector2 _anchor;
    [SerializeField] private Transform _canvas;
    [SerializeField] bool DoesLog = false;
    [SerializeField] bool DoesRender = false;
    [SerializeField] Image progressBarImage;
    public event TileCacherEvent OnTileCachingEnd;

    void Awake() { current = this;}
    private void Start()
    {
        // ImageFetcher = new ImageDataFetcher();
        ImageFetcher = ScriptableObject.CreateInstance<ImageDataFetcher>();
        ImageFetcher.DataRecieved += ImageDataReceived;
        ImageFetcher.FetchingError += ImageDataError;
    }

    public void CacheTiles(int _zoomLevel, string _topLeft, string _bottomRight){
        ZoomLevel = _zoomLevel;
        Points = new List<string>();
        Points.Add(_topLeft);
        Points.Add(_bottomRight);
        PullTiles();
    }


    [ContextMenu("Download Tiles")]
    public void PullTiles()
    {
	    Progress = 0;
	    _tileCountToFetch = 0;
	    _currentProgress = 0;
        _failedTileCount = 0;

        var pointMeters = new List<UnwrappedTileId>();
        foreach (var point in Points)
        {
            var pointVector = Conversions.StringToLatLon(point);
            var pointMeter = Conversions.LatitudeLongitudeToTileId(pointVector.x, pointVector.y, ZoomLevel);
            pointMeters.Add(pointMeter);
        }

        var minx = int.MaxValue;
        var maxx = int.MinValue;
        var miny = int.MaxValue;
        var maxy = int.MinValue;

        foreach (var meter in pointMeters)
        {
            if (meter.X < minx)
            {
                minx = meter.X;
            }

            if (meter.X > maxx)
            {
                maxx = meter.X;
            }

            if (meter.Y < miny)
            {
                miny = meter.Y;
            }

            if (meter.Y > maxy)
            {
                maxy = meter.Y;
            }
        }

        // If there is only one tile to fetch, this makes sure you fetch it
        if(maxx == minx){
            maxx++;
            minx--;
        }

        if(maxy == miny){
            maxy++;
            miny--;
        }

        _tileCountToFetch = (maxx - minx) * (maxy - miny);
        if(_tileCountToFetch == 0){
            OnTileCachingEnd.Invoke(Status.NOTHING_TO_CAHCE , 0);
        } else {
            _anchor = new Vector2((maxx + minx) / 2, (maxy + miny) / 2);
            PrintLog(string.Format("{0}, {1}, {2}, {3}", minx, maxx, miny, maxy));
            StartCoroutine(StartPulling(minx, maxx, miny, maxy));
        }
    }

    private IEnumerator StartPulling(int minx, int maxx, int miny, int maxy)
    {

        for (int i = minx; i < maxx; i++)
        {
            for (int j = miny; j < maxy; j++)
            {

                ImageFetcher.FetchData(new ImageDataFetcherParameters()
                {
                    canonicalTileId = new CanonicalTileId(ZoomLevel, i, j),
                    mapid = ImageMapId,
                    tile = null
                });

                yield return null;
            }
        }
    }

    #region Fetcher Events

    private void ImageDataError(UnityTile arg1, RasterTile arg2, TileErrorEventArgs arg3)
    {
        PrintLog(string.Format("Image data fetching failed for {0}\r\n",  arg2.Id));
        _failedTileCount++;
    }

    private void ImageDataReceived(UnityTile arg1, RasterTile arg2)
    {
        _currentProgress++;
	    Progress = (float)_currentProgress / _tileCountToFetch * 100;
        if(progressBarImage != null && progressBarImage.gameObject.activeInHierarchy) progressBarImage.fillAmount = Progress / 100;
        RenderImagery(arg2);
        if(Progress == 100) CheckEnd();
    }
    #endregion

    #region Utility Functions
    private void CheckEnd(){
        if(OnTileCachingEnd != null){
            if(_failedTileCount == 0){
                OnTileCachingEnd.Invoke(Status.ALL_CACHED , _tileCountToFetch);
            } else if(_failedTileCount == _tileCountToFetch){
                OnTileCachingEnd.Invoke(Status.ALL_FALIED , 0);
            } else if(_failedTileCount > 0 && _failedTileCount < _tileCountToFetch){
                OnTileCachingEnd.Invoke(Status.SOME_CACHED ,  _tileCountToFetch - _failedTileCount);
            }
        }
    }
    private void RenderImagery(RasterTile rasterTile)
    {
        if(!DoesRender || _canvas == null || !_canvas.gameObject.activeInHierarchy) return;

        GameObject targetCanvas = GameObject.Find("canvas_" + ZoomLevel);
        if(targetCanvas == null){
            targetCanvas = new GameObject("canvas_" + ZoomLevel);
            targetCanvas.transform.SetParent(_canvas);    
        }

        var go = new GameObject("image");
        go.transform.SetParent(targetCanvas.transform);
        var img = go.AddComponent<RawImage>();
        img.rectTransform.sizeDelta = new Vector2(10,10);
        var txt = new Texture2D(256,256);
        txt.LoadImage(rasterTile.Data);
        img.texture = txt;
        (go.transform as RectTransform).anchoredPosition = new Vector2((float)(rasterTile.Id.X - _anchor.x) * 10, (float)-(rasterTile.Id.Y - _anchor.y) * 10);
    }
    private void PrintLog(string message){
        if(!DoesLog) return;
        Log += message;
    }
    #endregion
}

Hey bud,

nice work with the script and sharing it.

I am busy studying it but I am having trouble understanding what this script is processing exactly.

What I am trying to do is get each users phone to cache one region within a selected area so that that cache consistently stays there and does not get updated via ambient caching.

I will keep studying it to try understand it better but if you could supply a pseudo code guideline I would really appreciate that.

For instance what game object should I attach this component to and how to get it to cache a radius around a selected area.

total noob question I know but thanks anyways!

Kind regards,

Jesse

[dorukeker]

Hi Jesse,
This code is made to attache to an empty game object. And the fetching function is called using the Context Menu.

In a different use case you can call that function from another script etc.

Regarding the use case: The example is actually doing what you asked. You feed the top left and bottom right corner of a region; and the desired zoom level... and it caches the tiles for that area and zoom level.

I hope this helps.
Cheers,
Doruk

[StarKing777]

Hi Jesse,
This code is made to attache to an empty game object. And the fetching function is called using the Context Menu.

In a different use case you can call that function from another script etc.

Regarding the use case: The example is actually doing what you asked. You feed the top left and bottom right corner of a region; and the desired zoom level... and it caches the tiles for that area and zoom level.

I hope this helps.
Cheers,
Doruk

Hey bud,

thanks a lot for getting back to me I really appreciate it.

One more questions what exactly is MapId referring to on line 144.

This is the error I receive as I have not understood its definition.

Assets\TileCacher.cs(144,21): error CS0117: 'ImageDataFetcherParameters' does not contain a definition for 'mapid'

thanks again.

Kind regards

Jesse

[dorukeker]

Did you check out the branch what @brnkhy mentioned in his post?
https://github.com/mapbox/mapbox-unity-sdk/tree/TilePrefetching
AFAIK ImageDataFetcherParameters does not exist in the regular build of MapBox; only in that branch.

See here from my first post:

Another note before the code:
When you implement the script Unity will give errors in some files form the SDK. To solve this:

Check out the branch mentioned in @brnkhy post
Replace the files with this check out

[StarKing777]

Did you check out the branch what @brnkhy mentioned in his post?
https://github.com/mapbox/mapbox-unity-sdk/tree/TilePrefetching
AFAIK ImageDataFetcherParameters does not exist in the regular build of MapBox; only in that branch.

See here from my first post:

Another note before the code:
When you implement the script Unity will give errors in some files form the SDK. To solve this:

Check out the branch mentioned in @brnkhy post
Replace the files with this check out

Hey man,

great thank you for the direction sorry I am somewhat new to source control and git.

thanks I will look into it.

Kind regards,

Jesse

[brnkhy]

ImageDataFetcherParameters is in the core SDK as well but still that branch should help with offline caching. I haven't tried it with latest version (of the sdk) but still, idea stands.

[StarKing777]

n (of the sdk) but s

Hey man,

thanks for the really useful advice in pointing out the core SDK script, will help a lot, much appreciated :)

I will study it and then explore the branch and study that as well.

thanks for the solution in your branch by the way.

Kind regards,

Jesse

[Naphier]

Just wondering how this is going? Is this on the roadmap yet? Our military customers are often on restricted networks and need to be able to access maps. We really need a centralized way to distribute these maps over a closed network as having all the users connect to mapbox.com to pre-fetch the map is not really feasible.
Thanks!

[abhishektrip]

@Naphier You may want to look into the Atlas offering from Mapbox for your use-case. Maps SDK for Unity is compatible with Atlas and this solution is geared towards users like yourself.
Relevant blog - https://blog.mapbox.com/ar-and-3d-in-a-secure-environment-2cf6068d6d51

[Naphier]

Thanks, indeed interesting, but seems like this would be overkill when MB could just let us switch out cache.db files as needed or something similar.

[tomh4]

Any updates to this ? We would also need this

[Markovicho]

Is the ambient cache still limited to 50mb (refer to initial posting) for Unity? And is there a way to increase the cachesize for Unity Apps? For the Android/iOS-SDK it should be possible based on this post: https://blog.mapbox.com/new-cache-management-controls-for-mapbox-sdks-2be0302f9ba

[brnkhy]

@dorukeker @Naphier @tomh4 @StarKing777
just wanted to let you know offline maps are in a test branch at the moment. you can check the pinned ticket for a lot more info 🙏

[Naphier]

@brnkhy apologies, but I don't see the pinned ticket. Can you point us to the branch so we can check it out?

[brnkhy]

@Naphier ah I never used pin thing before but I thought it was public and visible to everyone. Anyway here's the ticket; #1671
branch link is there as well

[Naphier]

Sweet, thanks!

[brnkhy]

@Naphier please let me know if you run into troubles, it's super beta so there will be bugs but I want to finish this asap so any fixes to this will be priority for me 🙏

[Naphier]

Will certainly try!

…

On Mon, Sep 21, 2020, 17:13 Baran Kahyaoğlu ***@***.***> wrote: @Naphier <https://github.com/Naphier> please let me know if you run into troubles, it's super beta so there will be bugs but I want to finish this asap so any fixes to this will be priority for me 🙏 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1312 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACNJYPBLQWHIEVCRE535F3TSG66YTANCNFSM4G6U73FA> .

[jaspervandenbarg]

I noticed the I am missing tiles on some zoom levels zo I changed the following:

private IEnumerator StartPulling(int minx, int maxx, int miny, int maxy)
    {

        for (int i = minx; i <= maxx; i++)
        {
            for (int j = miny; j <= maxy; j++)
            {

                ImageFetcher.FetchData(new ImageDataFetcherParameters()
                {
                    canonicalTileId = new CanonicalTileId(ZoomLevel, i, j),
                    mapid = ImageMapId,
                    tile = null
                });

                yield return null;
            }
        }
    }

[Markovicho]

@jaspervandenbarg based on which commit/branch as well as which file is this fix based ? Maybe this is the right issue for this case ? :-)
#1671

[jaspervandenbarg]

@jaspervandenbarg based on which commit/branch as well as which file is this fix based ? Maybe this is the right issue for this case ? :-)
#1671

It is based on https://github.com/mapbox/mapbox-unity-sdk/tree/TilePrefetching

I'll have a look into the issue you provided.