Merge branch 'master' of https://github.com/PDAL/PDAL

.travis.yml

@@ -13,7 +13,7 @@ script:
 
 after_success:
   - echo "secure travis:" "$TRAVIS_SECURE_ENV_VARS"
-  - sh -c 'if test "$TRAVIS_SECURE_ENV_VARS" = "true" -a "$TRAVIS_BRANCH" = "1.6-maintenance"; then echo "publish website"; ./scripts/ci/build_docs.sh; ./scripts/ci/add_deploy_key.sh; ./scripts/ci/deploy_website.sh $TRAVIS_BUILD_DIR/doc/build /tmp; fi'
+  - sh -c 'if test "$TRAVIS_SECURE_ENV_VARS" = "true" -a "$TRAVIS_BRANCH" = "1.7-maintenance"; then echo "publish website"; ./scripts/ci/build_docs.sh; ./scripts/ci/add_deploy_key.sh; ./scripts/ci/deploy_website.sh $TRAVIS_BUILD_DIR/doc/build /tmp; fi'
 
 notifications:
   on_success: always

CMakeLists.txt

@@ -28,14 +28,14 @@ mark_as_advanced(CMAKE_VERBOSE_MAKEFILE)
 
 # the next line is the ONLY place in the entire pdal system where
 # the version info is hard-coded
-set(PDAL_VERSION_STRING "1.6.0" CACHE STRING "PDAL version" FORCE)
+set(PDAL_VERSION_STRING "1.7.0" CACHE STRING "PDAL version" FORCE)
 
 DISSECT_VERSION()
 GET_OS_INFO()
 SET_INSTALL_DIRS()
 
-set(PDAL_API_VERSION "5")
-set(PDAL_BUILD_VERSION "6.1.0")
+set(PDAL_API_VERSION "6")
+set(PDAL_BUILD_VERSION "7.0.0")
 
 # Name of C++ library
 

HOWTORELEASE.txt

@@ -4,7 +4,7 @@ Steps for Making a PDAL Release
 
 :Author: Howard Butler
 :Contact: howard@hobu.co
-:Date: 05/31/2017
+:Date: 04/04/2018
 
 This document describes the process for releasing a new version of PDAL.
 
@@ -21,27 +21,26 @@ Release Process
       PDAL_VERSION_MAJOR, PDAL_VERSION_MINOR, PDAL_VERSION_PATCH,
       and PDAL_CANDIDATE_VERSION strings.
 
-  - python/VERSION.txt
-    python/pdal/__init.py
-
   - Update SO versioning
     set(PDAL_API_VERSION "1")
     set(PDAL_BUILD_VERSION "1.0.0")
     * https://github.com/libspatialindex/libspatialindex/pull/44#issuecomment-57088783
 
   - doc/quickstart.rst has a number of current-release references
 
+  - doc/download.rst point to new release
+
   - Make and push new release branch
 
     ::
 
-        git branch 1.6-maintenance
-        git push origin 1.6-maintenance
+        git branch 1.7-maintenance
+        git push origin 1.7-maintenance
 
 
   - Increment the doc build branch of .travis.yml:
 
-    "$TRAVIS_BRANCH" = "1.6-maintenance"
+    "$TRAVIS_BRANCH" = "1.7-maintenance"
 
   - Make DockerHub build entry for new release branch.
 
@@ -101,30 +100,25 @@ Release Process
 
 10) Write the release notes. Email PDAL mailing list with notice about release
 
-11) Upload Python extension to PyPI
-
-  ::
-
-      cd python; python setup.py sdist upload
 
-12) Upload new OSGeo4W package to download.osgeo.org:/osgeo/download/osgeo4w/x86_64/release/pdal
+11) Upload new OSGeo4W package to download.osgeo.org:/osgeo/download/osgeo4w/x86_64/release/pdal
 
   - Go to https://ci.appveyor.com/project/hobu/pdal
   - Choose ``OSGEO4W_BUILD=ON`` build
   - Scroll to very bottom
-  - Fetch tarball "OSGeo4W64 build will be uploaded to https://s3.amazonaws.com/pdal/osgeo4w/pdal-a4af2420b09725a4a0fff1ef277b1e6370c497d2.tar.bz2"
+  - Fetch tarball "OSGeo4W64 build will be uploaded to https://s3.amazonaws.com/pdal/osgeo4w/pdal-a4af2420b09725a4a0fff1ef277b1.7370c497d2.tar.bz2"
 
   - rename to match current release and set OSGeo4W build number to 1
 
     ::
 
-        mv pdal-a4af2420b09725a4a0fff1ef277b1e6370c497d2.tar.bz2 pdal-1.6.0-1.tar.bz2
+        mv pdal-a4af2420b09725a4a0fff1ef277b1.7370c497d2.tar.bz2 pdal-1.7.0-1.tar.bz2
 
   - copy to OSGeo4W server
 
     ::
 
-        scp pdal-1.6.0-1.tar.bz2 hobu@download.osgeo.org:/osgeo/download/osgeo4w/x86_64/release/pdal
+        scp pdal-1.7.0-1.tar.bz2 hobu@download.osgeo.org:/osgeo/download/osgeo4w/x86_64/release/pdal
 
   - refresh OSGeo4W
 

RELEASENOTES.txt

@@ -1,3 +1,136 @@
+================================================================================
+1.7.0
+================================================================================
+
+Changes of Note
+===============
+
+-  ``filter.ferry`` now creates output dimensions with the same type as
+   the input dimension. It also takes an arrow ‘=>’ in addition to ‘=’
+   in the ``--dimension`` specification.
+-  ``filters.hexbin`` now falls back to slow boundary creation if no
+   bounds information exists to do fast boundary creation.
+-  Dimension names can now contain the forward slash (‘/’) character.
+-  ``readers.gdal`` and ``filters.colorization`` now attempt to create
+   dimensions with the type of the associated raster.
+-  The Python PDAL extension code was removed from the PDAL source tree
+   to its own `repository <https://github.com/PDAL/python>`__.
+-  The JAVA PDAL interface was remove from the PDAL source tree to its
+   own `repository <https://github.com/PDAL/java>`__.
+-  ``pdal pipeline``\ and ``pdal translate`` now use stream mode if the
+   operation being performed is streamable. An option ``--nostream`` has
+   been added to both commands to prevent the use of stream mode. The
+   ``--stream`` option of ``pdal pipeline`` is now obsolete and ignored.
+-  A new interface has been provided for the creation of plugins
+   requiring less boilerplate code. There has been no API change.
+-  Stages and pipelines can now be tested to check whether they are
+   streamable.
+
+Enhancements
+============
+
+-  Added options ``--timegap`` and ``--speedmin`` to ``readers.mbio`` to
+   allow configuration of which points should be read.
+-  Added support for compression schemes (xz, lzma, zstd) and created a
+   standardized interface (#1722).
+-  ``writers.bpf`` now supports the option ``auto`` for the ``coord_id``
+   option to allow the UTM zone to be set from the spatial reference if
+   possible (#1723).
+-  Added the ability read stage-specific options from a file with the
+   ``--option_file`` option (#1641).
+-  Replace the GDAL point-in-polygon with a much faster implementation
+   in ``filters.crop``.
+-  Add a ``--reverse`` option to ``filters.mortonorder`` to provide a
+   point ordering for good dispersal.
+-  ``readers.bpf`` now supports the TCR (ECEF - earth centered, earth
+   fixed) coordinate system.
+-  Added option ``--use_eb_vlr`` to allow ``readers.las`` to interpret
+   an extra bytes VLR as if the file were version 1.4 even if it’s using
+   an earlier LAS version.
+-  ``readers.text`` added options ``--header`` and ``--skip`` to provide
+   an alternative header line and to allow skipping lines before reading
+   the header, respectively.
+-  ``writers.text`` now supports the ability to specify individual
+   dimension precision using a colon (‘:’) and integer following the
+   dimension name in the ``--order`` option.
+-  ``readers.numpy`` adds support for reading from Numpy (.npy) save
+   files.
+-  ``pdal info`` now provides the ``--enumerate`` option. See the
+   documentation for
+   `filters.stats <https://www.pdal.io/stages/filters.stats.html>`__ for
+   details.
+-  Added a general option ``--logtiming`` to cause log output to contain
+   the elapsed time from the start of the program. (#1882)
+
+Documentation
+=============
+
+-  Added a description of the Alpine Linux environment used for Travis
+   and Docker.
+-  Updated the documentation for building PDAL on Windows.
+-  Added an example of how to loop files with PowerShell.
+-  Corrected output shown in the documentation for ``filters.hexbin``.
+-  Reorganized the stages landing page to make it easier to find.
+   (#1880)
+
+New stages
+==========
+
+-  ``readers.greyhound`` - Allows reading points from a source using the
+   `greyhound <https://github.com/hobu/greyhound>`__ protocol.
+-  ``filters.neighborclassifier`` - Re-classifies points based on the
+   classification of neighboring points.
+
+Removed stages
+==============
+
+-  ``filters.computerange`` - Use ``filters.python`` to simulate the
+   functionality.
+
+Bug fixes
+=========
+
+-  ``filters.range`` now always rejects NaN values as out of range.
+-  Changed the default ``--timegap`` value in ``readers.mbio`` from 0 to
+   1 second.
+-  Fixed a bug when reading pointers from metadata on some OSX systems.
+-  Fixed a problem in ``pdal translate`` where overriding a reader or
+   writer from a pipeline wouldn’t create the proper pipeline.
+-  Fixed a problem where multiple LASzip VLRs would get written to LAS
+   files (#1726).
+-  Fixed an installation problem with the Python extension.
+-  Fixed a bug in ``writers.tindex`` that could cause a crash if the
+   output file wasn’t LAS.
+-  Fixed JSON output from ``filters.hexbin`` when density/area can’t be
+   calculated.
+-  Fixed a problem where output might be tagged with the wrong SRS when
+   using ``filters.reprojection`` in stream mode. (#1877)
+-  PDAL_DRIVER_PATH would be improperly parsed on Windows systems when
+   the path contained colons. Windows builds now use the semicolon as
+   the path separator. (#1889)
+-  Convert NaN and infinite double values to proper strings for output
+   as JSON.
+-  Synthetic, keypoint and withheld flags are now added to the
+   classification dimension for version 1.0 - 1.3 files in
+   ``readers.las``.
+-  Support missed case supporting points with color but no density in
+   ``filters.poisson``.
+-  Throw an error if the input file can’t be opened in ``readers.ply``.
+-  The ``--stdin`` option for ``kernels.tindex`` didn’t work. Now it
+   does. Various other fixes were made.
+-  ``writers.gdal`` now throws an error if an attempt is made to write
+   an output file with no points available.
+-  A build error that would be generated if lazsip was not found, even
+   if it was not requested, has been resolved.
+-  A build error that would be generated if Python was found but not
+   requested has been resolved.
+-  PDAL defaults to using `normal CMake
+   interface <https://cmake.org/cmake/help/v3.11/policy/CMP0022.html>`__
+   linking (#1890)
+-  Fixed an issue where dimensions from ``readers.pcd`` and
+   ``writers.pcd`` could get confused with dimensions from
+   ``readers.sbet`` and ``writers.sbet``.
+
 ================================================================================
 1.6.0
 ================================================================================

doc/apps/info.rst

@@ -13,8 +13,6 @@ Displays information about a point cloud file, such as:
 * the plain text format should be reStructured text if possible to allow a user
   to retransform the output into whatever they want with ease
 
-Processing is performed with stream mode if possible.
-
 ::
 
     $ pdal info <input>

doc/apps/pipeline.rst

@@ -23,7 +23,8 @@ more information.
       progress file.
   --stdin, -s               Read pipeline from standard input
   --metadata                Metadata filename
-
+  --nostream                Don't run in stream mode, even if technically
+      possible.
 
 Substitutions
 ................................................................................

doc/apps/translate.rst

@@ -22,6 +22,7 @@ from the command-line.  Processing is done with stream mode if possible.
     --metadata, -m     Dump metadata output to the specified file
     --reader, -r       Reader type
     --writer, -w       Writer type
+    --nostream         Don't run in stream mode, even if technically possible.
 
 The ``--input`` and ``--output`` file names are required options.
 

doc/development/compilation/windows.rst

@@ -34,6 +34,13 @@ are also available via that packaging system. This document assumes you
 will be using OSGeo4W as your base, and anything more advanced is beyond
 the scope of the document.
 
+.. note::
+
+    The AppVeyor build system uses the PDAL project's configuration on the OSGeo4W
+    system. It contains a rich resource of known working examples. See
+    https://github.com/PDAL/PDAL/blob/master/appveyor.yml and
+    https://github.com/PDAL/PDAL/tree/master/scripts/appveyor for inspiration.
+
 Required Compiler
 ------------------------------------------------------------------------------
 
@@ -113,7 +120,7 @@ Switch to the ``-maintenance`` branch.
 
    ::
 
-      c:\dev> git checkout 1.6-maintenance
+      c:\dev> git checkout 1.7-maintenance
 
 
    .. note::

doc/development/release-notes/1.7.0.md

@@ -0,0 +1,68 @@
+# Changes of Note
+
+- `filter.ferry` now creates output dimensions with the same type as the input dimension. It also takes an arrow '=>' in addition to '=' in the `--dimension` specification.
+- `filters.hexbin` now falls back to slow boundary creation if no bounds information exists to do fast boundary creation.
+- Dimension names can now contain the forward slash ('/') character.
+- `readers.gdal` and `filters.colorization` now attempt to create dimensions with the type of the associated raster.
+- The Python PDAL extension code was removed from the PDAL source tree to its own [repository](https://github.com/PDAL/python).
+- The JAVA PDAL interface was remove from the PDAL source tree to its own [repository](https://github.com/PDAL/java).
+- `pdal pipeline`and `pdal translate` now use stream mode if the operation being performed is streamable.  An option `--nostream` has been added to both commands to prevent the use of stream mode.  The `--stream` option of `pdal pipeline` is now obsolete and ignored.
+- A new interface has been provided for the creation of plugins requiring less boilerplate code. There has been no API change.
+- Stages and pipelines can now be tested to check whether they are streamable.
+
+# Enhancements
+
+- Added options `--timegap` and `--speedmin` to `readers.mbio` to allow configuration of which points should be read.
+- Added support for compression schemes (xz, lzma, zstd) and created a standardized interface (#1722).
+- `writers.bpf` now supports the option `auto` for the `coord_id` option to allow the UTM zone to be set from the spatial reference if possible (#1723).
+- Added the ability read stage-specific options from a file with the `--option_file` option (#1641).
+- Replace the GDAL point-in-polygon with a much faster implementation in `filters.crop`.
+- Add a `--reverse` option to `filters.mortonorder` to provide a point ordering for good dispersal.
+- `readers.bpf` now supports the TCR (ECEF - earth centered, earth fixed) coordinate system.
+- Added option `--use_eb_vlr` to allow `readers.las` to interpret an extra bytes VLR as if the file were version 1.4 even if it's using an earlier LAS version.
+- `readers.text` added options `--header` and `--skip` to provide an alternative header line and to allow skipping lines before reading the header, respectively.
+- `writers.text` now supports the ability to specify individual dimension precision using a colon (':') and integer following the dimension name in the `--order` option.
+- `readers.numpy` adds support for reading from Numpy (.npy) save files.
+- `pdal info` now provides the `--enumerate` option.  See the documentation for [filters.stats](https://www.pdal.io/stages/filters.stats.html) for details.
+- Added a general option `--logtiming` to cause log output to contain the elapsed time from the start of the program. (#1882)
+
+# Documentation
+
+- Added a description of the Alpine Linux environment used for Travis and Docker.
+- Updated the documentation for building PDAL on Windows.
+- Added an example of how to loop files with PowerShell.
+- Corrected output shown in the documentation for `filters.hexbin`.
+- Reorganized the stages landing page to make it easier to find. (#1880)
+
+# New stages
+
+- `readers.greyhound` - Allows reading points from a source using the [greyhound](https://github.com/hobu/greyhound) protocol.
+- `filters.neighborclassifier` - Re-classifies points based on the classification of neighboring points.
+
+# Removed stages
+
+- `filters.computerange` - Use `filters.python` to simulate the functionality.
+
+# Bug fixes
+
+- `filters.range` now always rejects NaN values as out of range.
+- Changed the default `--timegap` value in `readers.mbio` from 0 to 1 second.
+- Fixed a bug when reading pointers from metadata on some OSX systems.
+- Fixed a problem in `pdal translate` where overriding a reader or writer from a pipeline wouldn't create the proper pipeline.
+- Fixed a problem where multiple LASzip VLRs would get written to LAS files (#1726).
+- Fixed an installation problem with the Python extension.
+- Fixed a bug in `writers.tindex` that could cause a crash if the output file wasn't LAS.
+- Fixed JSON output from `filters.hexbin` when density/area can't be calculated.
+- Fixed a problem where output might be tagged with the wrong SRS when using `filters.reprojection` in stream mode. (#1877)
+- PDAL_DRIVER_PATH would be improperly parsed on Windows systems when the path contained colons. Windows builds now use the semicolon as the path separator. (#1889)
+- Convert NaN and infinite double values to proper strings for output as JSON.
+- Synthetic, keypoint and withheld flags are now added to the classification dimension for version 1.0 - 1.3 files in `readers.las`.
+- Support missed case supporting points with color but no density in `filters.poisson`.
+- Throw an error if the input file can't be opened in `readers.ply`.
+- The `--stdin` option for `kernels.tindex` didn't work. Now it does. Various other fixes were made.
+- `writers.gdal` now throws an error if an attempt is made to write an output file with no points available.
+- A build error that would be generated if lazsip was not found, even if it was not requested, has been resolved.
+- A build error that would be generated if Python was found but not requested has been resolved.
+- PDAL defaults to using [normal CMake interface](https://cmake.org/cmake/help/v3.11/policy/CMP0022.html) linking (#1890)
+- Fixed an issue where dimensions from `readers.pcd` and `writers.pcd` could get confused with dimensions from `readers.sbet` and `writers.sbet`.
+

doc/download.rst

@@ -13,22 +13,24 @@ Download
 Current Release(s)
 ------------------------------------------------------------------------------
 
-* **2017-10-12** `PDAL-1.6.0-src.tar.gz`_ `Release Notes`_ (`md5`_)
+* **2018-04-05** `PDAL-1.7.0-src.tar.gz`_ `Release Notes`_ (`md5`_)
 
-.. _`Release Notes`: https://github.com/PDAL/PDAL/releases/tag/1.6
+.. _`Release Notes`: https://github.com/PDAL/PDAL/releases/tag/1.7
 
-.. _`PDAL-1.6.0-src.tar.gz`: http://download.osgeo.org/pdal/PDAL-1.6.0-src.tar.gz
-.. _`md5`: http://download.osgeo.org/pdal/PDAL-1.6.0-src.tar.gz.md5
+.. _`PDAL-1.7.0-src.tar.gz`: http://download.osgeo.org/pdal/PDAL-1.7.0-src.tar.gz
+.. _`md5`: http://download.osgeo.org/pdal/PDAL-1.7.0-src.tar.gz.md5
 .. _`DebianGIS`: http://wiki.debian.org/DebianGis
 
 
 Past Releases
 ------------------------------------------------------------------------------
 
+* **2017-10-12** `PDAL-1.6.0-src.tar.gz`_
 * **2017-04-06** `PDAL-1.5.0-src.tar.gz`_
 * **2016-12-15** `PDAL-1.4.0-src.tar.gz`_
 
 
+.. _`PDAL-1.6.0-src.tar.gz`: http://download.osgeo.org/pdal/PDAL-1.6.0-src.tar.gz
 .. _`PDAL-1.5.0-src.tar.gz`: http://download.osgeo.org/pdal/PDAL-1.5.0-src.tar.gz
 .. _`PDAL-1.4.0-src.tar.gz`: http://download.osgeo.org/pdal/PDAL-1.4.0-src.tar.gz