QGIS3 plugin to import survex .3d files
Current version 1.2
v1.2 - fixed CRS import methods
v1.1 - minor updates, tagged for packaging
v1.0 - migrated and updated from QGIS 2.18 plugin
For QGIS 2.18 visit https://github.com/patrickbwarren/qgis-survex-import
- no dependencies, natively reads binary (v8 format) survex .3d files;
- import stations and legs with full metadata;
- create passage walls, cross-sections, and polygons from LRUD data;
- all features have z dimensions, and (mean) elevations to assist downstream workflow;
- the co-ordinate reference system (CRS) can be imported from the .3d file (*);
- results can be saved immediately as a GeoPackage file.
(*) with the appropriate
*cs commands in the .svx source files (see below).
To install the plugin:
- clone or download this repository;
python/plugins/in the current active profile, the location of which can be found from within QGIS3 by going to 'Settings → User Profiles → Open Active Profile Folder' (*);
- enable the plugin in QGIS3 by going to 'Plugins → Manage and Install Plugins...'; make sure the box next to 'Import .3d file' is checked, in the 'Installed' tab.
(*) Alternatively if you have
pb_tool you can run
pb_tool deploy from within the
When installed, a menu item 'Import .3d file' should appear on the 'Vector' drop-down menu in the main QGIS3 window, and (if enabled) a .3d icon in a toolbar.
Debian users may be able to install from a packaged version
Selecting 'Import .3d file' (or clicking on the .3d icon) brings up a window for the user to select a .3d file with a number of options:
- Import legs, with options to include splay, duplicate, and surface legs;
- Import stations, with the option to include surface stations (*);
- Import passage data computed from LRUDs, with the option to use
clino weights (see below):
- as polygons, with an option to include mean up / down data;
- as walls;
- as cross sections;
- as traverses, showing the centrelines used for above;
- Optionally, set the co-ordinate reference system (CRS) from the .3d file or inherit from the QGIS3 project;
- Keep features from previous import(s) (optional);
- Select a GeoPackage (.gpkg) file to save results (optional).
(*) In rare cases a station may be flagged both surface and underground, in which case it is imported even if the 'surface' option is left unchecked.
On clicking OK, vector layers are created to contain the imported features as desired. Legs, walls, cross sections, and traverses are imported as line strings in separate vector layers for convenience. All created layers are saved to the GeoPackage file if requested (any existing content is overwritten).
A CRS selector dialog box will appear, if neither of the CRS selector options are checked, or if CRS from .3d file is selected but there is no CRS in the .3d file.
If 'keep features' is selected, then previously imported features are not discarded, and the newly-created layers will contain both the previously imported features plus any new features imported from the designated .3d file. This choice allows processed survey data sets to be combined from multiple sources. Note that cumulative imports do not result in features being overwritten, even if they happen to share the same name, since all features are assigned a unique ID.
The latest version (v1.2) fixes a bug that introduced a mismatch between the CRS and the proj4 string in the .3d file. Import of .3d files generated by Therion should now work.
All layers are created with an ELEVATION attribute, for convenience. For stations this is the just the z dimension. For all other features it is the mean elevation.
For station and leg layers, the following additional attribute fields that are created:
stations: NAME, and flags SURFACE, UNDERGROUND, ENTRANCE, EXPORTED, FIXED, ANON
legs: NAME, STYLE, DATE1, DATE2, NLEGS (*), LENGTH (*), ERROR (*), ERROR_HORIZ (*), ERROR_VERT (*), and flags SURFACE, DUPLICATE, SPLAY
(*) These fields correspond to the error data reported in the .3d file, which is only generated (by survex) if loop closures are present.
The flags are integer fields set to 0 or 1.
The STYLE field for legs is one of NORMAL, DIVING, CARTESIAN, CYLPOLAR, or NOSURVEY.
The DATE fields are either the same, or represent a date range, in the standard QGIS3 format YYYY-MM-DD.
If up / down data for passage polygons is requested, then the polygons have MEAN_UP and MEAN_DOWN attributes in addition to ELEVATION. These are computed from the LRUD data for the two stations at either end of the leg. They can be used in 3d work (see end).
Passage walls (as line strings), polygons, and cross sections (as
lines) are computed from the left and right measurements in the LRUD
data in the same way that the
aven viewer in survex displays passage
'tubes' (well, near enough...). The direction of travel (bearing) is
worked out, and used to compute the positions of points on the left
and right hand passage walls. These wall points are then assembled
into the desired features (walls, polygons, cross sections).
The direction of travel is inferred from the directions of the two legs on either side of the given station (with special treatment for stations at the start and end of a traverse). In averaging these, either the legs can be weighted equally (except true plumbs which break the sequence), or the option is given to weight legs by the cosine of the inclination (computed from the processed data, not the actual clino reading). The former is the default, and the latter corresponds to checking the 'use clino weights' box in the import dialog. This alternative option downplays the significance of the occasional steeply inclined leg in an otherwise horizontal passage.
One might want to do this for the following reason. In the 'good old days' steeply inclined legs were usually avoided as they are difficult to sight a compass along, and instead good practice was to keep legs mostly horizontal and add in the occasional plumbed leg when dealing with rough ground. Also pitches were nearly always plumbed. This meant that inferring passage direction as a simple average, ignoring plumbed legs, was most likely correct. For modern surveying with digital instruments, this is no longer the case: there is no loss of accuracy for steeply inclined legs, and shining a laser down a pitch at an off-vertical angle is no problem. Therefore, the 'use clino weights' option has been invented to give such steeply included legs less weight when inferring the passage direction. Note that in a steeply inclined passage, all legs are likely roughly equally inclined, and therefore roughly equally weighted, so using clino weights shouldn't affect the inferred direction of travel in that situation.
TL;DR: if in doubt try first with the 'use clino weights' option selected.
Note that passage wall data is inferred and any resemblance to reality may be pure coincidence: if in doubt, use splays!
Co-ordinate reference system (CRS)
To be integrated with other sources of geographical information such as
maps, GPS tracks, and so on, an imported survey should be georeferenced. This
means that the spatial reference system (SRS) should be specified;
in QGIS parlance this is referred to as a co-ordinate reference system (CRS).
The easiest way to do this is to use survex
commands in the .svx file to
set an output CRS in the .3d file, then select 'CRS from .3d file' in the import
*cs OSGB:SD *cs out EPSG:7405
This specifies that the entrance
*fix's are in the Ordnance Survey (OS)
100km x 100km SD grid square, and that the output should use the all-numeric
British National Grid
dump3d to inspect
DowProv.3d one finds the line
CS +init=epsg:7405 +no_defs
This is a proj4 string embedded in the .3d file. The input filter uses this to identify the CRS: if the string contains an EPSG number then that determines the CRS; otherwise a CRS is created using the proj4 string directly. If the .3d file does not contain a CS proj4 string then the input filter fall backs onto a CRS selector dialog.
In some cases
it may be helpful to create beforehand a user-defined CRS to select in the
For example, if the
*cs commands are
DowProv.svx, the resulting .3d file lacks a proj4 string and
all co-ordinates are relative to the OS SD grid square. This .3d file can
nevertheless still be imported into QGIS3 by first creating
a custom CRS for the SD grid square, then
specifying this custom CRS in the import dialog (or inheriting from the project
CRS if that is set appropriately).
For the OS SD grid square, the requisite custom CRS can be created from the following proj4 string
+proj=tmerc +lat_0=49 +lon_0=-2 +k=0.9996012717 +x_0=100000 +y_0=-500000 +ellps=airy +towgs84=375,-111,431,0,0,0,0 +units=m +vunits=m +no_defs
(all on one line). This is identical to the proj4 string for the British
except that the
have been shifted to a new false origin for the SD grid square.
Another example is the Austrian Loser plateau data that accompanies the survex distribution as sample data. Many of the cave entrances are recorded using a truncated form of the MGI / Gauss-Krüger (GK) Central Austria SRS (the non-truncated form is EPSG:31255). This truncated SRS corresponds to a proj4 string
+proj=tmerc +lat_0=0 +lon_0=13d20 +k=1 +x_0=0 +y_0=-5200000 +ellps=bessel +towgs84=577.326,90.129,463.919,5.137,1.474,5.297,2.4232 +units=m +no_defs
(again this should be all on one line).
This is derived from the proj4 string for
by changing the
For more details and examples of survex
*cs commands see
cave_surveying_and_gis.pdf in the present
In-depth explanations of co-ordinate reference systems can be found in the Ordnance Survey booklet entitled A Guide to Coordinate Systems in Great Britain which can be found on the Ordnance Survey website.
TL;DR: for optimal use, specify the CRS using
*cs out with an EPSG number.
What to do next
Once the data is in QGIS3 one can do various things with it.
For example, features (stations, legs, polygons) can be colored
by elevation to mimic the behaviour of the
aven viewer in survex
(hat tip Julian Todd for figuring some of this out). The easiest way
to do this is to use the
.qml style files provided in this
repository. For example to color legs by depth, open the properties
dialog and under the 'Style' tab, at the bottom select 'Style →
Load Style', then choose one of the
style files. This will apply a color scheme to the ELEVATION field
data with an inverted spectral color ramp. Use
lines for legs,
walls, cross sections and traverses;
points for stations; and
polygons for polygons.
Two versions of these style files are provided.
The first version uses a graduated, inverted spectral color ramp to color ranges of ELEVATION. A small limitation is that these ranges are not automatically updated to match the vertical range of the current data set, but these can be refreshed by clicking on 'Classify' (then 'Apply' to see the changes).
The second version uses a simple marker (line, or fill) with the color set by an expression that maps the ELEVATION to a spectral color ramp. There are no ranges here, but rather these styles rely on zmin and zmax variables being set (see 'Variables' tab under layer → Properties). By matching zmin and zmax between layers with these styles, one can be assured that a common coloring scheme is being applied. A handy way to choose values for zmin and zmax is to open the statistics panel (View → Panels → Statistics Panel) to check out the min and max values in the ELEVATION field.
Color legs by date is possible using an expression like
day(age("DATE1",'1970-01-01')) (which gives the number of days
between the recorded DATE1 and the given date). Color legs by error
is also possible.
Another thing one can do is enable 'map tips', for example to use the NAME field. Then, hovering the mouse near a station (or leg) will show the name as a pop-up label. For this to work:
- 'View → Map Tips' should be checked in the main menu;
- the map tip has to be set up to use the NAME field ('Properties → Display') in the relevant layer;
- the layer has to be the currently selected one, though one can set the symbology to 'No symbols' to avoid having to display the features.
With a digital elevation model (DEM raster layer) even more interesting things can be done. For example one can use raster interpolation to find the surface elevation at all the imported stations and save for example to a SURFACE_ELEV field. Then, one can use the field calculator to make a DEPTH field containing the depth below surface, as SURFACE_ELEV minus ELEVATION. Stations can be colored by this, or the information can be added to the 'map tip', etc.
Three dimensional views can be made directly in QGIS3 with 3D Map View though more conveniently with the Qgis2threejs plugin, usually in combination with a DEM. To render features in 3d use the z co-ordinate for points and lines. Passage 'tubes' like those in aven can be approximately rendered using LRUD polygons, with the base set to floor level and the extruded height set to roof level. To do this import the MEAN_UP and MEAN_DOWN fields mentioned above and use the field calculator to make two new floating point (double) fields: FLOOR equal to ELEVATION minus MEAN_DOWN, and HEIGHT equal to MEAN_DOWN plus MEAN_UP. Then render the polygons with the z co-ordinate as the absolute FLOOR, and extruded height as HEIGHT.
Note that there is currently a bug in the Qgis2threejs plugin for QGIS3 that causes a python error when features have data defined properties, such as color by elevation using zmin and zmax variables (second option above). The error looks like
AttributeError:'QgsSimpleLineSymbolLayer' object has no attribute 'dataDefinedProperty'
(The problem doesn't arise if features are colored by ranges as in the first option above.)
A workaround is as follows. First add zmin and zmax variables into the layer properties (bring up the Properties window and go to the Variables tab): use the green '+' button to add two new variables then click on Apply and OK. Choose values suited to the data set of interest (as above), for example for the DowProv case they can be set to 320 and 400 respectively (elevation in metres ODN). Second, make sure in the main QGIS map window the features in the layer of interest (eg legs) use only a simple style with a fixed colour (this is the default). Third, in the Qgis2threejs Exporter window, double click on the layer of interest (eg legs) to bring up the layer properties, and in the Style panel select Color → Expression. Paste the following into the Expression box.
If all is well the lines in the Qgis2threejs Exporter preview window should change to be colored by elevation.
Note that if you encountered the python error the plugin may not function correctly any more. It may have to reloaded (which can be done if you have installed the 'Plugin Reloader' plugin); or QGIS3 restarted.
Sample georeferenced survey data can be found in the
example directory as
The corresponding GeoPackage file is in the
example directory as
Further notes on cave surveying and GIS are in
Code in this repository is licensed under GPL v2:
This program is free software: you can redistribute it and / or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 2 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program. If not, see http://www.gnu.org/licenses/.
Modifications and extensions copyright © (2018-2020) Patrick B Warren.