Translations

[Tekn1ck]

I have a quick question for you that perhaps you can assist with.

I am working in EPSG:27700 + ODN height and we have applied an offset of X=-535000 and y=-180000 (we have been doing this for a long time to get objects that are in Ordnance Survey space into something more friendly for 3d editing due to floating point errors - although I am open to change this concept)

I am a bit confused about how I should write this in the manifest so that the model just pops in exactly where it should do in Unity and what to write in the VTS map make local script.

I managed to get pretty close by translating the point in EPSG:27700 origin point -535000, -180000 to WGS coords like below in QGIS (units in meters)

"srs": "enu lat0=51.502820844 lon0=-0.056331546 h0=0.00",

Then I put this in the manifest

"srs": "enu lat0=51.502820844 lon0=-0.056331546 h0=0.00",

in Unity I put these same numbers into the make map local which brings it pretty close, but the height is off - and when I take a close look, if I correct the height in one location on the dataset and then check another location then the height differences vary. Perhaps more worrying than this, there also appears to be a slight horizontal shift perhaps 30 cm or so from the reference data.

So my questions are

1. • What is the correct way to do this?
2. • Is it possible to just change the srs in the tileset.conf file after rather than reprocessing the VEF to VTS again?
3. • During the VEF to VTS process are the meshes distorted to fit any ellipsoid or is it simply slicing the mesh and retexturing where required for optimised streaming?

[vaclavblazek]

1. You have to tell ve2vts the correct SRS your data are. You have your dataset in EPSG:27700 which should be equivalent to +proj=tmerc +lat_0=49 +lon_0=-2 +k=0.9996012717 +x_0=400000 +y_0=-100000 +ellps=airy +units=m +no_defs. So, it's a Transversal Mercator projection. ENU is a local cartesian system defined by a plane tangential to the ellipsoid at the given point. These systems can be pretty close but I'm not that sure. Please, use correct SRS. As you said, your data are localized around some arbitrary origin. You can use a global transformation matrix i.e. the toplevel trafo array in VEF's manifest.json. Just add your local origin to the forth column of a unit 4x4 matrix, i.e. something like "trafo": { 1, 0, 0, X, 0, 1, 0, Y, 0, 0, 1, Z, 0, 0, 0, 1 }. You need to use lastest vts-tools if you installed from a package repo (v. 1.90) for this feature.

2. A tileset uses SRS from reference frame, all data in the tileset adhere to these SRS. You cannot change it by any configuration. Data are not converted between different spatial reference systems when rendering.

3. As in 2), all data in tileset are in reference frame's SRS, meshes are in physicalSRS. In case of melown2015 reference frame, physicalSRS is WGS84 geocentric system. I.e. +proj=geocent +datum=WGS84 +units=m +no_defs. So yes, data are transformed and cut in predefined grid.

[Tekn1ck]

Hi again,

Thanks for this that makes a lot of sense, the thing that was stopping me was that I did not understand where to put the coordinates in the Trafo command. I have followed this out now and done a quick test and it calculates fine from VEF to VTS fine.

However, now when I load the data into Unity it is not really anywhere near where I would like to be unless I press the "default position" checkbox in the "vts map make local script" , but this rotates the model and shifts it too.

I have tried quite a few combinations namely

In the manifest to apply the two below offsets since I am unsure if I am moving the data or defining the center of the data in this offset.
Trafo offset of -535000,-180000
Trafo offset of 535000,180000

In Unity- "vts map make local script"
left the lat long at 0,0
tried the lat long as before the coordinate transformed from WGS84 (-0.056331546,51.502820844)
tried the coordinates from WGS84 geocentric system (3978400.988459764,-3911.449518747)

Nothing gets close - if I manually try and adjust the lat long I am getting very strange numbers to bring it close and it is not correct anyway.

I wonder also whether there is an additional fundamental problem here if the data is being reprojected into the WGS84 geocentric system since it no longer matches to all the survey data and architectural drawings/3d that we input which are always in EPSG:27700. I could be wrong, but I think we need to make a new reference frame that means we do not reproject the data away from EPSG:27700. The problem is Unity - it is a cartesian system so when we put 3d models in they are alway xyz and geographic transformations are complex and not what architects use.

What are your thoughts on how to approach this? Ideally we would just work in EPSG:27700 from source through to final rendering in Unity

[Tekn1ck]

I have done a bit more research and it seems that strange number is not so strange after all - it is pretty close to the origin of the EPSG:27700 crs

and now when I add this into the make map local it is close to my manual guess - but still not close enough.

The grey model is my reference model - the VTS model should sit precisely over the top.

I am a bit stumped now. What to do next!?

[ladislavhorky]

Hi, I may be totally out of context but lets see what we can do.

First: start from a correct VEF - no shifts or ad-hoc transformations should be then needed. In case there is something special about SRS handling in UNITY, I am cc ing @malytomas .

VEF:
a) Are your meshes in really in EPSG: 27700 (then they will contain rather big numbers) or localized? Then you would need to use the global trafo (shift) mentioned by @vaclavblazek. That may be part of your issue.., if the model is somehow localized (not reflected in VEF manifest), you are feeding vef2vts wrong data that then get transformed, possibly turning a simple shift into something non-trivial that you are then trying to fix.

b) The 27700 is above different ellipsoid than WGS84, so your proj4 string in VEF should definitely contain +towgs84 param (7-rametric transformation between ellipsoids). Looking at https://epsg.io/27700, the correct string may be (contrary to the one mentioned by @vaclavblazek):

+proj=tmerc +lat_0=49 +lon_0=-2 +k=0.9996012717 +x_0=400000 +y_0=-100000 +ellps=airy +towgs84=446.448,-125.157,542.06,0.15,0.247,0.842,-20.489 +units=m +no_defs

Good news is this might be responsible for the shifts you see, bad news it may not be the one towgs84 key you need - these are constructed so that the transformation has some reasonable error over some area. I case of whole SRS area of use, you might not get lower that few meters on average. Then you would need to use grids and possibly SW from Ordnanace Survey (see remarks on epsg.io link) or search for different towgs84 that would work better for you. We have similar troubles with Czech Krovak, but the (painstakingly found) towgs84 works well for our use-cases.

After fixing the VEF, everything else should just work.

Hope this helps (yes, SRSs are tough, especially if you work in local one)

[Tekn1ck]

Hi,

So A) They originate in EPSG:27700 so yes they have some pretty massive numbers in them, hence why we translate these to a local origin by shifting the output by -535000,-180000. I have done @vaclavblazek trafo - and that seems fine to me, I am pretty certain that I am feeding the correct trafo data in.
"trafo": [ 1, 0, 0, -535000, 0, 1, 0, -180000, 0, 0, 1, 0, 0, 0, 0, 1 ],

b) Had a go with that proj4 string and it is close, like before but now has a height difference.

This is proving quite complex. If I were to try to create a reference frame that is EPSG:27700 so I am not reprojecting - how would I do this?

[vaclavblazek]

Custom RF? Now we are talkin'.

Before we begin: If you want to have your RF in EPSG:27700 you're gonna end up with a bit of "flat Earth" which is only locally usable. I guess it's not a problem here.

NB: we use proj4 strings exclusively because frontends (especially web) lack WKT or EPSG support.

1. Spatial references, reference frames etc. are in the /opt/vts/etc/registry directory. You may either tweam files there or make a copy and instruct VTS tools to use a different registry directory via --registry cmdline option. NB: unity does not need this since mapConfig.json serves as it's own registry.

2. You may get inspired in our prehistoric ppspace reference frame. Do not use vertical adjustmend, tho.

3. Add new SRS in srs.json, based on our utm33n. Just your name it accordingly, use proper proj4 string, and do not forget to say it's a projected system.

4. In referenceframes.json, make a copy of ppspace reference frame (choose some name]. Use your SRS as model.physicalSrs and model.navigationSrs.

5. Then update ranges properly. nodes[0].extents is a 2D area of your root tile -- rather make it square, ll stands for lower-left corner, ur stands for upper-right corner.

6. Box your world in division.extents. Use the same 2D bounds as in 2c, Z axis is minimum/maximum height value your world supports.

7. You may want to restart mapproxy with your updated registry and have a look at system spheroid in your new reference frame dir. It should show a flat square.

8. Convert your dataset to your RF. Try it in a web browser first.

[malytomas]

First, let me repeat this: always test your mapconfig in webbrowser (vts-browser-js) or vts-browser-desktop before unity!

Second, RF (reference frame) in mapconfig defines 3 separate srs:

Physical srs is what is used for meshes. VTS is designed to handle large numbers in physical srs.

Another srs in RF is navigation. It is used for camera controls in general.

Unity itself does not handle large numbers well, this is what the map make local is for. It will transform the space from navigation srs into a completely new local system - the one that suits unity - it aligns up vector and north.

Note that map make local is not srs conversion. It just rotates forward towards north and up vector out of the center of earth.

The third srs is public and it is not used in unity.

[Tekn1ck]

Happy new year - will be trying a new reference frame in the coming weeks!

[Tekn1ck]

Hi all,

So I finally had a chance to try this and I think I am pretty close, but there is still something funny going on. I have managed to make it all process but it will not load in the browser window.

I reprocessed a few tiles without the trafo on since that was not working at all. So now I am starting with a load of tiles in epsg27700 base.

In the SRS file I added this.

 "epsg27700": {
    "comment": "Projected, Ordnance Survey (epsg27700)",
    "srsDef": "+proj=tmerc +lat_0=49 +lon_0=-2 +k=0.9996012717 +x_0=400000 +y_0=-100000 +ellps=airy +towgs84=446.448,-125.157,542.06,0.15,0.247,0.842,-20.489 +units=m +no_defs",
    "type": "projected"
},

In the referenceframes I added this

{
    "version" : 1,
    "id": "epsg27700",
    "description": "Ordnance Survey Planar Reference Frame",
    "body": "Earth",

    "model": {
        "physicalSrs": "epsg27700",
        "navigationSrs": "epsg27700",
        "publicSrs": "geographic-wgs84-egm96"
    },

    "division": {
        "extents" : {
            "ll": [-90619.29,10097.13, -500], "ur": [612435.55,1234954.16,7000] },
        "heightRange": [-500,7000],
        "nodes": [
            {
                "id": { "lod": 0, "position": [0,0] },
                "srs": "epsg27700",
                "externalTexture": true,
                "extents": {
                    "ll": [-90619.29,10097.13], "ur": [612435.55,1234954.16] },
                "partitioning": "bisection"
            }
        ]
    },

Then I ran the CLI and used the new epsg27700 frame and the calc completed.

vef2vts --input /project/EPSG27700/City_4_VTS --output /project/epsg27700 --tilesetId epsg27700 --referenceFrame epsg27700 --overwrite --keepTmpset

The problem is that when I browse to it, the browser will not load the map when I first click on the output folder as it usually does and if I click on the index file it just hangs.

These are some of the last lines of the console output - it looked like it went through with no errors ;)

2021-02-26 10:41:08 I3 [467(tile:16-58063-56383)]: Generated tile #2018 of 2038 ( 99.02 % done): 16-58063-56383 (epsg27700, extents: 532266.958397,181146.500400:532277.686162,181165.190235) [mesh,atlas]. {encoder.cpp:process():304}
2021-02-26 10:41:08 I3 [467(tile:16-58052-56374)]: Generated tile #2019 of 2038 ( 99.07 % done): 16-58052-56374 (epsg27700, extents: 532148.952976,181314.708916:532159.680742,181333.398751) [mesh,atlas]. {encoder.cpp:process():304}
2021-02-26 10:41:08 I3 [467(tile:16-58061-56379)]: Generated tile #2020 of 2038 ( 99.12 % done): 16-58061-56379 (epsg27700, extents: 532245.502866,181221.259741:532256.230631,181239.949576) [mesh,atlas]. {encoder.cpp:process():304}
2021-02-26 10:41:08 I3 [467(tile:16-58058-56380)]: Generated tile #2021 of 2038 ( 99.17 % done): 16-58058-56380 (epsg27700, extents: 532213.319569,181202.569906:532224.047335,181221.259741) [mesh,atlas]. {encoder.cpp:process():304}
2021-02-26 10:41:08 I3 [467(tile:16-58059-56381)]: Generated tile #2022 of 2038 ( 99.21 % done): 16-58059-56381 (epsg27700, extents: 532224.047335,181183.880070:532234.775100,181202.569906) [mesh,atlas]. {encoder.cpp:process():304}
2021-02-26 10:41:08 I3 [467(tile:16-58053-56374)]: Generated tile #2023 of 2038 ( 99.26 % done): 16-58053-56374 (epsg27700, extents: 532159.680742,181314.708916:532170.408507,181333.398751) [mesh,atlas]. {encoder.cpp:process():304}
2021-02-26 10:41:08 I3 [467(tile:16-58049-56374)]: Generated tile #2024 of 2038 ( 99.31 % done): 16-58049-56374 (epsg27700, extents: 532116.769680,181314.708916:532127.497445,181333.398751) [mesh,atlas]. {encoder.cpp:process():304}
2021-02-26 10:41:08 I3 [467(tile:16-58060-56383)]: Generated tile #2025 of 2038 ( 99.36 % done): 16-58060-56383 (epsg27700, extents: 532234.775100,181146.500400:532245.502866,181165.190235) [mesh,atlas]. {encoder.cpp:process():304}
2021-02-26 10:41:08 I3 [467(tile:16-58054-56374)]: Generated tile #2026 of 2038 ( 99.41 % done): 16-58054-56374 (epsg27700, extents: 532170.408507,181314.708916:532181.136273,181333.398751) [mesh,atlas]. {encoder.cpp:process():304}
2021-02-26 10:41:08 I3 [467(tile:16-58061-56383)]: Generated tile #2027 of 2038 ( 99.46 % done): 16-58061-56383 (epsg27700, extents: 532245.502866,181146.500400:532256.230631,181165.190235) [mesh,atlas]. {encoder.cpp:process():304}
2021-02-26 10:41:08 I3 [467(tile:16-58050-56374)]: Generated tile #2028 of 2038 ( 99.51 % done): 16-58050-56374 (epsg27700, extents: 532127.497445,181314.708916:532138.225211,181333.398751) [mesh,atlas]. {encoder.cpp:process():304}
2021-02-26 10:41:08 I3 [467(tile:16-58049-56375)]: Generated tile #2029 of 2038 ( 99.56 % done): 16-58049-56375 (epsg27700, extents: 532116.769680,181296.019081:532127.497445,181314.708916) [mesh,atlas]. {encoder.cpp:process():304}
2021-02-26 10:41:08 I3 [467(tile:16-58054-56375)]: Generated tile #2030 of 2038 ( 99.61 % done): 16-58054-56375 (epsg27700, extents: 532170.408507,181296.019081:532181.136273,181314.708916) [mesh,atlas]. {encoder.cpp:process():304}
2021-02-26 10:41:08 I3 [467(tile:16-58051-56375)]: Generated tile #2031 of 2038 ( 99.66 % done): 16-58051-56375 (epsg27700, extents: 532138.225211,181296.019081:532148.952976,181314.708916) [mesh,atlas]. {encoder.cpp:process():304}
2021-02-26 10:41:08 I3 [467(tile:16-58055-56374)]: Generated tile #2032 of 2038 ( 99.71 % done): 16-58055-56374 (epsg27700, extents: 532181.136273,181314.708916:532191.864038,181333.398751) [mesh,atlas]. {encoder.cpp:process():304}
2021-02-26 10:41:09 I3 [467(tile:16-58062-56383)]: Generated tile #2033 of 2038 ( 99.75 % done): 16-58062-56383 (epsg27700, extents: 532256.230631,181146.500400:532266.958397,181165.190235) [mesh,atlas]. {encoder.cpp:process():304}
2021-02-26 10:41:09 I3 [467(tile:16-58051-56374)]: Generated tile #2034 of 2038 ( 99.80 % done): 16-58051-56374 (epsg27700, extents: 532138.225211,181314.708916:532148.952976,181333.398751) [mesh,atlas]. {encoder.cpp:process():304}
2021-02-26 10:41:09 I3 [467(tile:16-58055-56375)]: Generated tile #2035 of 2038 ( 99.85 % done): 16-58055-56375 (epsg27700, extents: 532181.136273,181296.019081:532191.864038,181314.708916) [mesh,atlas]. {encoder.cpp:process():304}
2021-02-26 10:41:09 I3 [467(tile:16-58048-56375)]: Generated tile #2036 of 2038 ( 99.90 % done): 16-58048-56375 (epsg27700, extents: 532106.041914,181296.019081:532116.769680,181314.708916) [mesh,atlas]. {encoder.cpp:process():304}
2021-02-26 10:41:09 I3 [467(tile:16-58050-56375)]: Generated tile #2037 of 2038 ( 99.95 % done): 16-58050-56375 (epsg27700, extents: 532127.497445,181296.019081:532138.225211,181314.708916) [mesh,atlas]. {encoder.cpp:process():304}
2021-02-26 10:41:09 I3 [467(tile:16-58048-56374)]: Generated tile #2038 of 2038 (100.00 % done): 16-58048-56374 (epsg27700, extents: 532106.041914,181314.708916:532116.769680,181333.398751) [mesh,atlas]. {encoder.cpp:process():304}
2021-02-26 10:41:09 I3 [467(Tile-136-101-1-1)]: VTS Encoder: generated. Finishing. {encoder.cpp:run():134}
2021-02-26 10:41:09 I3 [467(Tile-136-101-1-1)]: Generating DTM from heightmap (1786x2551 pixels). {heightmap.cpp:dtmize():273}
2021-02-26 10:41:09 I3 [467(Tile-136-101-1-1)]: Extracting navtiles. {ntgenerator.cpp:generate():358}
2021-02-26 10:41:09 I3 [467(Tile-136-101-1-1)]: Setting navtiles at lod 11. {ntgenerator.cpp:generate():365}
2021-02-26 10:41:09 I3 [467(Tile-136-101-1-1)]: Navtiles extracted. {ntgenerator.cpp:generate():390}
2021-02-26 10:41:10 I3 [467(Tile-136-101-1-1)]: Computing surrogates at lod 16 from heightmap 223x318. {ntgenerator.cpp:fillSurrogate():263}
2021-02-26 10:41:10 I3 [467(Tile-136-101-1-1)]: Computing surrogates at lod 15 from heightmap 111x159. {ntgenerator.cpp:fillSurrogate():263}
2021-02-26 10:41:10 I3 [467(Tile-136-101-1-1)]: Computing surrogates at lod 14 from heightmap 55x79. {ntgenerator.cpp:fillSurrogate():263}
2021-02-26 10:41:10 I3 [467(Tile-136-101-1-1)]: Computing surrogates at lod 13 from heightmap 27x39. {ntgenerator.cpp:fillSurrogate():263}
2021-02-26 10:41:10 I3 [467(Tile-136-101-1-1)]: Computing surrogates at lod 12 from heightmap 13x19. {ntgenerator.cpp:fillSurrogate():263}
2021-02-26 10:41:10 I3 [467(Tile-136-101-1-1)]: Computing surrogates at lod 11 from heightmap 6x9. {ntgenerator.cpp:fillSurrogate():263}
2021-02-26 10:41:10 I3 [467(Tile-136-101-1-1)]: VTS Encoder: Flushing. {encoder.cpp:run():141}
2021-02-26 10:41:10 I3 [467(Tile-136-101-1-1)]: VTS Encoder: done. {encoder.cpp:run():145}
2021-02-26 10:41:10 I4 [467(Tile-136-101-1-1)]: All done. {vef2vts.cpp:run():1238}

Have I made any mistakes?

[Tekn1ck]

Oh to add - I was not sure what to do with the z height bounds of the world, but I figured it would not do any harm to just leave the bounds the same as the ppspace example.

[Tekn1ck]

@vaclavblazek Sorry to bother you, but would you be able to take a quick look at my posts above and suggest where I may have made a mistake?

[vaclavblazek]

All seems legit. Could you please try debugger.html instead of index.html? It should show tiles (with their masks) in 2D Leaflet browser. Green corners mark which child is valid.

[Tekn1ck]

Can you give me an example of how to do this? I am currently just loading up the output folders and clicking on the name in the browser window when I navigate to my local server http://192.158.0.10:8070/store/output/ inside these output folders there are no html files.

[vaclavblazek]

Use http://192.158.0.10:8070/store/output/debugger.html .
(FYI, .../ is the same page as .../index.html)

[Tekn1ck]

ok - so testing on one that works and the debugger shows up a nice array of squares with references in. but the epsg output juts gives me a 500 Internal Server Error