Skip to content
Permalink
Browse files

Add ability to read compressed Lidar LAZ files

  • Loading branch information...
gwlucastrig committed Feb 9, 2017
1 parent b8ae8d8 commit f1123797aea822b9b59e22c9325d76703dbb29fb
@@ -14,6 +14,11 @@ at http://tsusiatsoftware.net/jts/main.html and released under
the GNU Lesser General Public License available at
http://www.gnu.org/licenses/lgpl-3.0.en.html.

This product includes software developed by the laszip4j project
at https://github.com/mreutegg/laszip4j and released under
the GNU Lesser General Public License available at
http://www.gnu.org/licenses/lgpl-3.0.en.html.

This product includes icons from the FatCow web page at
http://www.fatcow.com/free-icons and distributed under the Create Commons
Attribution 3.0 License.
@@ -83,18 +83,31 @@ that I decided to include one of my own.


### Why are there external project dependencies? ###
The only external dependency in the Tinfour package is the
[Apache Commons Math Library](https://commons.apache.org/proper/commons-math/).
For your convenience, a copy of the Commons math package is included
with the Tinfour download.
The Tinfour package has two external dependencies. The first is on the
laszip4j package, which allows the test and demonstration applications
to read Lidar products in the highly specialized LAZ data compression
format. This dependency is not part of the core functions, but
is necessary for access to these data products.

The second dependency is the [Apache Commons Math Library](https://commons.apache.org/proper/commons-math/).
This dependency is required by the linear algebra and statistics functions
needed by the Geographically Weighted Regression classes. If you have
needed by the Geographically Weighted Regression classes which
are part of the core functionality. If you have
an alternate linear algebra library in your own software, it would be
possible to refactor the Tinfour code to perform the core regression
functions using the alternative. You would, however, have to remove
those functions that specifically require statistics elements
(such as the T-Distribution) or provide your own alternative

For your convenience, a copy of both the laszip4j and the Commons math
packages are included with the Tinfour download.

### What version of Java is required for Tinfour? ###
All the Tinfour code compiles under Java 7. However, the lazzip4j
libraries requires Java 8. Therefore, if you wish to use
the Tinfour Viewer or otherwise access LAZ files, you will
require a Java 8 JVM.

### Configuring Tinfour in an IDE ###
Configuring Tinfour in an IDE is pretty simple:
* Create a Java project
@@ -104,53 +117,31 @@ Configuring Tinfour in an IDE is pretty simple:
set up a source reference to (installed path)/Tinfour/src/test/java
so your IDE picks up the packages tinfour.test.*
* Set up a jar reference to (installed path)/Tinfour/lib/commons-math-3.3.6.1.jar
* Set up a jar reference to (installed path)/Tinfour/lib/laszip4j.jar
* Configure the IDE to run TinfourViewerMain. If you are working with very
large datasets, you may include the Java runtime option -Xmx1500m or larger
to increase the heap size.
to increase the heap size. However, in recent versions of Java the specification
of this option is not as critical as it once was.

### Current Work ###
The current focus of Tinfour development is the introduction of the
Constrained Delaunay Triangulation (CDT) to the software. The CDT
is a technique for representing
discontinuities in a Triangulated Irregular Network. For example, geographic
applications often need a way to represent "breaklines" -- features including
The current focus of Tinfour development is polishing aspects
of the Constrained Delaunay Triangulation (CDT) implementation. The CDT
is a technique for representing discontinuities in a Triangulated Irregular Network.
For example, geographic applications often need a way to represent "breaklines" -- features including
rivers, roads, coastlines and escarpments -- which mark a sudden change in
the local slope or terrain. Conventional Delaunay Triangulations
have a limited ability to treat boundaries where the surface undergoes a
nearly instantaneous change. By introducing linear and polygon features to
the construction of a TIN, the Constrained Delaunay Triangulation provides
an effective way of representing such features.

As of 17 December 2016, I have completed the preliminary
implementation of this feature and have posted the code to github.
I am currently implementing Rognant's algorithm for restoring
Delaunay conformity after the constraints are added.
Beyond that, my plan is to integrate CDT's into the Tinfour Viewer.
I expect to be complete with all work by the end of January 2017.
For an illustrated explanation of why CDT's are important,
see the Tinfour wiki page titled
an effective way of representing such features. For an illustrated discussion
of why CDT's are important, see the Tinfour wiki page titled
[About the Constrained Delaunay Triangulation](https://github.com/gwlucastrig/Tinfour/wiki/About-the-Constrained-Delaunay-Triangulation "About the Constrained Delaunay Triangulation")

For more detail about the Tinfour project development plans, see the
[Tinfour Project Status and Roadmap](https://github.com/gwlucastrig/Tinfour/wiki/Tinfour-Project-Roadmap) page.


### The Wish List ###
If you are interested in seeing new capabilities added to Tinfour,
I have a couple of ideas and would like to hear about yours.

I recently discovered a github project that is developing a
Java API for reading Lidar files written in the compressed LAZ format
(see [LAS Zip for Java](https://github.com/mreutegg/laszip4j) ).
I would very much like to extend the Lidar file reader to be able
to process LAZfiles since that form is used by the majority
of the websites distributing Lidar data today. Doing so would
make file access far more convenient. I'd also like to have the
extend the support for metadata obtained from LAS files, particularly
those elements using Well-Known Text (WKT) format and GeoTIFF tags.

I'd like to see an extension of Tinfour to build Voronoi Diagrams
and perhaps conduct rendering and analysis using that graphical structure
which is closely related to the Delaunay Triangulation.

### Conclusion ###
Finally, the whole point of working on a project like Tinfour is to see
it used to do something useful. To that end, I welcome ideas, requests, and
recommendations for analysis tools and applications that would
@@ -55,7 +55,7 @@
<attribute name="Specification-Title" value="Tinfour Bundled with Tests and Viewer Demo" />
<attribute name="Specification-Version" value="1.0" />
<attribute name="Main-Class" value="tinfour.test.viewer.TinfourViewerMain" />
<attribute name="Class-Path" value="Tinfour-1.0.jar ../lib/commons-math3-3.6.1.jar" />
<attribute name="Class-Path" value="Tinfour-1.0.jar ../lib/commons-math3-3.6.1.jar ../lib/laszip4j.jar" />
</manifest>
</jar>
</target>
BIN -398 Bytes (100%) dist/Tinfour-1.0.jar
Binary file not shown.
BIN +2.25 KB (100%) dist/TinfourViewer-1.0.jar
Binary file not shown.
Binary file not shown.
BIN +516 KB lib/laszip4j.jar
Binary file not shown.
@@ -120,13 +120,25 @@
private boolean isClosed;

private final List<LasVariableLengthRecord>vlrList;
private final File path;

public LasFileReader(File path) throws IOException {
this.path = path;
braf = new BufferedRandomAccessForLidar(path);
vlrList = new ArrayList<>();
readHeader(); //NOPMD
}


/**
* Get the source file for the reader.
* @return a valid file instance.
*/
public File getFile(){
return this.path;
}


/**
* Reads the header information from the beginning of a
* LAS file
@@ -77,6 +77,13 @@ public void setMaximumNumberOfVertices(long maxN) {
maximumNumberOfVertices = maxN;
}

private boolean isLazFile(File file){
String name = file.getName();
int n = name.length();
return n>4
&& ".LAZ".equalsIgnoreCase(name.substring(n-4, n));
}

/**
* Set the loader to pre-sort the vertices to improve their spatial locality
* before processing. Default is to not sort vertices.
@@ -330,7 +337,7 @@ public boolean accept(LasPoint record) {
};
}

ArrayList<Vertex> list = new ArrayList<>();
List<Vertex> list = new ArrayList<>();
long time0 = System.nanoTime();
long nVertices = reader.getNumberOfPointRecords();
this.numberOfVerticesInSource = nVertices;
@@ -345,28 +352,43 @@ public boolean accept(LasPoint record) {
}
}

LasPoint p = new LasPoint();
for (long iRecord = 0; iRecord < nVertices; iRecord++) {
if (pProgressThreshold == iProgressThreshold) {
pProgressThreshold = 0;
progressMonitor.reportProgress((int) (0.1 + (100.0 * (iRecord + 1)) / nVertices));
}
pProgressThreshold++;
reader.readRecord(iRecord, p);
if (filter.accept(p)) {
double x = (p.x - geoOffsetX) * geoScaleX;
double y = (p.y - geoOffsetY) * geoScaleY;
double z = p.z;
Vertex v = new VertexWithClassification( // NOPMD
x, y, z, (int) iRecord, p.classification);
//Vertex v = new Vertex(x, y, z, (int) iRecord);
list.add(v);
if (list.size() >= this.maximumNumberOfVertices) {
break;
File file = reader.getFile();
if (isLazFile(file)) {
VertexLoaderLaz loadLaz = new VertexLoaderLaz(
geoOffsetX,
geoScaleX,
geoOffsetY,
geoScaleY,
maximumNumberOfVertices);
list = loadLaz.loadVertices(
file,
nVertices ,
filter,
iProgressThreshold,
progressMonitor);
} else {
LasPoint p = new LasPoint();
for (long iRecord = 0; iRecord < nVertices; iRecord++) {
if (pProgressThreshold == iProgressThreshold) {
pProgressThreshold = 0;
progressMonitor.reportProgress((int) (0.1 + (100.0 * (iRecord + 1)) / nVertices));
}
pProgressThreshold++;
reader.readRecord(iRecord, p);
if (filter.accept(p)) {
double x = (p.x - geoOffsetX) * geoScaleX;
double y = (p.y - geoOffsetY) * geoScaleY;
double z = p.z;
Vertex v = new VertexWithClassification( // NOPMD
x, y, z, (int) iRecord, p.classification);
//Vertex v = new Vertex(x, y, z, (int) iRecord);
list.add(v);
if (list.size() >= this.maximumNumberOfVertices) {
break;
}
}
}
}

long time1 = System.nanoTime();
timeForLoad = (time1 - time0) / 1000000.0;
postProcessList(list);
@@ -0,0 +1,112 @@
/* --------------------------------------------------------------------
* Copyright 2016 Gary W. Lucas.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
* ---------------------------------------------------------------------
*/

/*
* -----------------------------------------------------------------------
*
* Revision History:
* Date Name Description
* ------ --------- -------------------------------------------------
* 02/2017 G. Lucas Created
*
* Notes:
*
* -----------------------------------------------------------------------
*/
package tinfour.test.utils;

import com.github.mreutegg.laszip4j.LASPoint;
import com.github.mreutegg.laszip4j.LASReader;
import java.io.File;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import tinfour.common.IMonitorWithCancellation;
import tinfour.common.Vertex;
import tinfour.las.ILasRecordFilter;
import tinfour.las.LasPoint;

public class VertexLoaderLaz {

double geoOffsetX;
double geoScaleX;
double geoOffsetY;
double geoScaleY;
long maximumNumberOfVertices;

VertexLoaderLaz(
double geoOffsetX,
double geoScaleX,
double geoOffsetY,
double geoScaleY,
long maximumNumberOfVertices) {
this.geoOffsetX = geoOffsetX;
this.geoScaleX = geoScaleX;
this.geoOffsetY = geoOffsetY;
this.geoScaleY = geoScaleY;
this.maximumNumberOfVertices = maximumNumberOfVertices;
}

public List<Vertex> loadVertices(
File file,
long nVertices,
ILasRecordFilter filter,
int iProgressThreshold,
IMonitorWithCancellation progressMonitor) throws IOException {
List<Vertex> list = new ArrayList<>();

int pProgressThreshold = 0;
LASReader reader = new LASReader(file);

LasPoint t4Point = new LasPoint();
int iRecord = 0;
for (LASPoint p : reader.getPoints()) {
if (pProgressThreshold == iProgressThreshold) {
pProgressThreshold = 0;
progressMonitor.reportProgress((int) (0.1 + (100.0 * (iRecord + 1)) / nVertices));
}
iRecord++;
pProgressThreshold++;

// TO DO: LASPoint does not yet have an accessor for "withheld" status.

// to support the use of a Tinfour filter, the LASPoint is
// transcribed to a Tinfour LasPoint. This is confusing and non-optimal
// but will have to do for now.
t4Point.x = p.getX();
t4Point.y = p.getY();
t4Point.z = p.getZ();
t4Point.classification = p.getClassification();
t4Point.returnNumber = p.getReturnNumber();
t4Point.numberOfReturns = p.getNumberOfReturns();

if (filter.accept(t4Point)) {
double x = (t4Point.x - geoOffsetX) * geoScaleX;
double y = (t4Point.y - geoOffsetY) * geoScaleY;
double z = t4Point.z;
Vertex v = new VertexWithClassification( // NOPMD
x, y, z, iRecord, t4Point.classification);
list.add(v);
if (list.size() >= this.maximumNumberOfVertices) {
break;
}
}
}
return list;
}

}
@@ -132,6 +132,8 @@ private boolean isDragAcceptable(DropTargetDragEvent dtde) {
String ext = name.substring(i, name.length());
if (".LAS".equalsIgnoreCase(ext)) {
return true;
} else if (".LAZ".equalsIgnoreCase(ext)) {
return true;
} else if (".TXT".equalsIgnoreCase(ext)) {
return true;
} else if (".CSV".equalsIgnoreCase(ext)) {
@@ -587,7 +587,7 @@ public void actionPerformed(ActionEvent e) {

@Override
public String getDescription() {
return "Lidar Files (LAS format only)";
return "Lidar Files (LAS and LAZ format)";
}

@Override
@@ -596,7 +596,7 @@ public boolean accept(File f) {
return true;
}
String ext = getFileExtension(f);
return ("LAS".equalsIgnoreCase(ext));
return "LAS".equalsIgnoreCase(ext) || "LAZ".equalsIgnoreCase(ext);
}

}
@@ -76,10 +76,7 @@ public IModel constructModel(File file) {

String ext = extractFileExtension(file);
ViewOptions view = getViewOptions();
if ("LAZ".equalsIgnoreCase(ext)) {
postStatusMessage(taskIndex, "Tinfour does not yet support LAZ files");
return null;
} else if ("LAS".equalsIgnoreCase(ext)) {
if ("LAS".equalsIgnoreCase(ext) || "LAZ".equalsIgnoreCase(ext)) {
LidarPointSelection selections = view.getLidarPointSelection();
return new ModelFromLas(file, selections);
} else if ("TXT".equalsIgnoreCase(ext)) {
@@ -188,10 +185,7 @@ public IModel loadModel(File file) {
IModel model;
ViewOptions view = getViewOptions();

if ("LAZ".equalsIgnoreCase(ext)) {
postStatusMessage(taskIndex, "Tinfour does not yet support LAZ files");
return null;
} else if ("LAS".equalsIgnoreCase(ext)) {
if ("LAZ".equalsIgnoreCase(ext) || "LAS".equalsIgnoreCase(ext)) {
LidarPointSelection selections = view.getLidarPointSelection();
model = new ModelFromLas(file, selections);
} else if ("TXT".equalsIgnoreCase(ext)) {

0 comments on commit f112379

Please sign in to comment.
You can’t perform that action at this time.