Spark SQL IQmulus Library
A library for reading and writing Lidar point cloud collections in PLY, LAS and XYZ formats from Spark SQL.
This Spark package is for Spark 1.6.2
spark-iqmulus library is published as a Spark package, which can be added to Spark jobs launched through
spark-submit by using the
--packages command line option. For example, to include it when starting the spark shell:
$ bin/spark-shell --packages IGNF:spark-iqmulus:0.1.0-s_2.10
--packages ensures that this library and its dependencies will be added to the classpath. The
--packages argument can also be used with
bin/spark-submit. For now this library is only published for Scala 2.10.
Examples below require the creation of an SQLContext and the following imports, depending on which aspect of the library is being used:
val sqlContext = new SQLContext(sc) import fr.ign.spark.iqmulus.ply._ import fr.ign.spark.iqmulus.las._ import fr.ign.spark.iqmulus.xyz._
Reading files as Spark dataframes:
val df1 = sqlContext.read.ply("myfile.ply") val df2 = sqlContext.read.las("myfile.las") val df3 = sqlContext.read.xyz("myfile.xyz")
Writing back Spark dataframes as files:
similarly, for XYZ and LAS :
val df = ... df.write.ply("/tmp/ply") df.write.las("/tmp/las") df.write.xyz("/tmp/xyz")
- the EU FP7 Project IQmulus, which leverages the information hidden in large heterogeneous geospatial data sets and make them a practical choice to support reliable decision making (N. ICT- 2011-318787).
- IGN, the French National Mapping Agency and its MATIS research lab.
To our knowledge this library has been used in the following research:
- M. Brédif, B. Vallet, B. Ferrand. Distributed dimensionality-based rendering of lidar point clouds. Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XL-3/W3, 559-564, doi:10.5194/isprsarchives-XL-3-W3-559-2015, 2015. Paper Presentation Poster