Merge branch 'github-pages'

docs/index.md

@@ -6,7 +6,7 @@ sidebar: 3dfier_sidebar
 permalink: index.html
 ---
 
-3dfier tries to fill the gap for simply creating 3D models. It takes 2D GIS datasets (e.g. topographical datasets) and "3dfies" them (as in "making them three-dimensional"). The elevation is obtained from a point cloud (we support LAS/LAZ at the moment), and the semantics of every polygon is used to perform the lifting. After lifting, elevation gaps between the polygons are removed by "stitching" together the polygons based on rules so that a watertight digital surface model (DSM) fused with 3D objects is constructed. A rule based stucture is used to extrude water as horizontal polygons, create LOD1 blocks for buildings, smooth road surfaces and construct bridges (3D polygonal surface). This software is developped by the [3D Geoinformation group](https://3d.bk.tudelft.nl) at **Delft University of Technology**.
+3dfier tries to fill the gap for simply creating 3D models. It takes 2D GIS datasets (e.g. topographical datasets) and "3dfies" them (as in "making them three-dimensional"). The elevation is obtained from a point cloud (we support LAS/LAZ at the moment), and the semantics of every polygon is used to perform the lifting. After lifting, elevation gaps between the polygons are removed by "stitching" together the polygons based on rules so that a watertight digital surface model (DSM) fused with 3D objects is constructed. A rule based stucture is used to extrude water as horizontal polygons, create LOD1 blocks for buildings, smooth road surfaces and construct bridges (3D polygonal surface). This software is developed by the [3D Geoinformation group](https://3d.bk.tudelft.nl) at **Delft University of Technology**.
 
 Our aim is to obtain one model that is error-free, so no intersecting objects, no holes (the surface is watertight) and buildings are integrated in the surface.
 

docs/pages/architecture/main_flow.md

@@ -9,22 +9,62 @@ permalink: main_flow.html
 How is the architecture of 3dfier described and how would you read it.
 
 ## Data flow through the program
-Explain different steps in the flow diagram. Show what data goes were and link to following flow diagrams.
+**TODO: Explain different steps in the flow diagram. Show what data goes were and link to following flow diagrams.**
 {% include imagezoom.html file="3dfier_general_flow.png" alt="General flow diagram 3dfier" %}
 
 ### Configuration
+Reading and validation of the configuration is done using the [YAML-CPP library](https://github.com/jbeder/yaml-cpp). It parses the file into an object and allows for easy access to all values. The configuration file is read and validated first. Value types are tested and options with limited options are verified. If validation fails an error message will be printed and the program execution will stop.
 
+When validation passes, all values are used to set the corresponding parameters in the software. After this initial read the configuration file is not used anymore since all values are now stored in memory.
+
+{% include imagezoom.html file="3dfier_classes.png" alt="Diagram of 3dfier classes" %}
 
 ### Data integrity check
+For all input files configured to use a check is done if they exist and readable. The polygon files are opened using an OGR reader and closed directly after. Same is done for the LAS/LAZ files, they are opened with LASlib and directly closed. This prevents failure of the program after partly executing. Also the output filepaths are tested to be existing before execution of the algorithm.
 
+*NOTE: Output folder must exist, it's not created automatically*
 
-### Handling polygons
+### Reading polygons
+If all sanity checks are passed the program turns to reading the polygon files. It takes into account the extent setting, only polygons which bounding box intersects with the given extent will be added. 
 
+A file is opened for reading and the id and height_field attributes are loaded. If these attributes do not exist the program will stop execution with an exception. The total number of features in the layer is logged and counters are set for the amount of multipolygons. Next each feature is read, the attributes are stored in a new TopoFeature. A TopoFeature is the internal storage class used by the algorithm to store attributes, 2D and 3D geometries and functionality to lift the objects. The geometry is extracted and depending on the polygon type it is pre-processed. For a multipolygon each separate polygon is added to a new TopoFeature in which the attributes are copied. The id of the feature is altered by adding a trailing dash and counter, e.g. 'id-0, id-1'. Curvepolygons are automatically stoked into small straight line segments. 
 
-### Handling points
+*Important: When reading the geometry the boost::geometry library is used to read WKT and both functions boost::geometry::unique and boost::geometry::correct are used for removing duplicate vertices and correcting ring orientation.*
 
+During the creation of a TopoFeature and storing its geometry, all used buffers and vectors are resized to correspond to the amount of points in the polygon outer and inner rings. This preallocates most of the needed memory for storing the elevation information acquired when reading the point clouds.
 
-### Magic
+For a later stage a spatial index is created that stores the bounding box of the geometries with a reference to the TopoFeature. This speeds up filtering the TopoFeatures within range for the point to polygon distance calculations done with the points from the point cloud. The spatial index is a [Boost R-tree](https://www.boost.org/doc/libs/1_72_0/libs/geometry/doc/html/geometry/reference/spatial_indexes/boost__geometry__index__rtree.html). There are two R-trees used, one for Buildings and one for all other objects combined. The reason for this is the two vertex radius settings which can differ, [radius_vertex_elevation]({{site.baseurl}}/output_options.html#radius_vertex_elevation) and [building_radius_vertex_elevation]({{site.baseurl}}/output_options.html#building_radius_vertex_elevation).
+
+The bounding box of all polygons is calculated from the two R-trees combined plus the vertex radius. Since the R-trees contain the bounding box of the object geometries the total bounding box can be larger then the actual area.
+
+{% include imagezoom.html file="3dfier_reading_polygons.png" alt="Flow diagram polygon reading" %}
+
+### Reading points
+The elevation information is read from LAS or LAZ files. The file is opened and the header is read. The bounding box from the header is intersected with the bounding box of all polygons. When the file doesn't overlap it is skipped. When the bounds overlap the point count is logged together with thinning setting if configured. For each point read the following things are checked before going into adding a point to a TopoFeature:
+- is point to be used according to *i % thinning == 0*
+- is classification not within the list of *omit_LAS_classes*
+- is point within the bounding box of the polygons
+
+When all checks pass the points is added to the map. The next step is to query both R-trees and return the objects that intersect with this point plus vertex radius in all directions. Each returned object is iterated and the point classification is checked based on the object types. The classification allowed refers to [use_LAS_classes]({{site.baseurl}}/lifting_options.html#use_las_classes) for each object class type configured in the [Lifting options]({{site.baseurl}}/lifting_options.html#). When the point with a certain class needs to be within the polygon, as configured in [use_LAS_classes_within]({{site.baseurl}}/lifting_options.html#use_las_classes_within), an additional boolean is set.
+
+The elevation point is added to the object if its classification is configured for use. Then followed by a check if the point needs to be within the polygon, or just within range. The height is then assigned to each vertex within range of the point. The height is multiplied by 100 (for centimetre accuracy), rounded to an integer (to keep the memory footprint as low as possible) and then added to the elevation vector of the vertex.
 
+Each object class has its own additional set of rules for a point to be used.
+
+
+{% include imagezoom.html file="3dfier_reading_points.png" alt="Flow diagram point reading" %}
+
+#### Terrain and Forest (TIN)
+These classes have the [simplification settings]({{site.baseurl}}/lifting_options.html#simplification) that adds points randomly. Also it stores all elevation points that are within the polygon respecting the [innerbuffer setting]({{site.baseurl}}/lifting_options.html#innerbuffer) in a vector. This vector is used to add additional points inside the polygons for a more precise ground model. 
+
+#### Building and Water (Flat)
+For these classes there is a difference to the others. Since these features are modelled as flat surfaces the elevation vector is not maintained per vector but for the TopoFeature as a whole. This lowers the memory footprint and calculation time while maintaining the same accuracy. All points within the polygon or within range of the vertices are stored as described in [radius_vertex_elevation]({{site.baseurl}}/output_options.html#radius_vertex_elevation). For buildings there are two elevation vectors, one for the ground and one for the roof.
+
+#### Road, Separation and Bridge (Boundary3D)
+These classes do not have any specific rules for assigning elevation information.
+
+### Magic
+{% include imagezoom.html file="threedfy.png" alt="Flow diagram for threedfy" %}
 
-### Creating data
+### Writing model
+{% include imagezoom.html file="3dfier_writing_model.png" alt="Flow diagram for writing 3D models" %}

docs/pages/getting-started/file_formats.md

@@ -17,6 +17,8 @@ Multipolygons are supported. They will be split into single polygons. The unique
 
 Curvedpolygons are supported starting v1.3. When the reader comes across a curvedpolygon this is discretized using the default OGR settings. This algorithm makes sure the discretization for a curve into a linestring is identical for traversing the line from start->end as from end->start.
 
+**TODO: Write example for PostGIS reading**
+
 ### 2. Point cloud
 Point cloud reading is implemented using the [LASLib library](https://github.com/LAStools/LAStools/tree/master/LASlib) that is the OpenSource reading library of [LAStools](https://rapidlasso.com/lastools/). This library supports reading of LAS and LAZ point clouds.
 

docs/pages/getting-started/installation.md

@@ -10,7 +10,7 @@ permalink: installation.html
 
 ## Install on Windows using binaries
 Binary releases exist only for Windows users. Others will have to follow one of the other installation guides for [Linux](#ubuntu-1604) or [Docker](#docker)
-There exists a ready-to-use version of [3dfier for Windows 64-bit]({{site.github.repository_url}}/releases/latest). Download and extract the files to any given folder and follow the instructions in the [Get started guide]({{site.baseurl}}/index).
+There exists a ready-to-use version of [3dfier for Windows 64-bit](https://github.com/{{site.repository}}/releases/latest). Download and extract the files to any given folder and follow the instructions in the [Get started guide]({{site.baseurl}}/index).
 
 ### Release binaries content
 Description of files in the zip file