diff --git a/doc/development/notes/apps.txt b/doc/development/notes/apps.txt index 6c34bc7712..d8b339a62d 100644 --- a/doc/development/notes/apps.txt +++ b/doc/development/notes/apps.txt @@ -16,13 +16,14 @@ App Commonality The following notes apply to all apps: * The apps should have a common support/utilities library for processing - command line arguments, showing version information, etc. + command line arguments, showing version information, etc. -- describe + boost program_options, and Application/AppSupport classes * Input format detection should reside in the app, not in the library. There will be no equivalent meta GDALOpen-like function to do this work for you, as it is too expensive and too easy to screw up. - +* describe common switches: --version, --help, --verbose pc2pc ===== @@ -54,19 +55,27 @@ The dump should go the stdout, but options should be provided to dump to XML or similar, where it makes sense to do so. +pcpipeline +========== + +[tbd] + pcview ====== -A simple 3D viewer for point clouds. Strictly speaking, not a command-line -app. Will be a Windows-only thing (WPF + DirectX), unless someone else -wants to do something too. +A simple 3D viewer for point clouds, largely for development and testing +as opposed to something for "rea" use. Uses OpenGL; works under Windows, +should work under Unix too. + pcindex ======= +(future) + Creates a serialized spatial index for a point cloud. First pass at this is to repurpose liblas::Index which supports serialization. @@ -75,6 +84,8 @@ is to repurpose liblas::Index which supports serialization. pcmerge ======= +(future -- likely to be part of pc2pc) + Merges multiple files together. (This is also known as mosacking.) @@ -82,5 +93,7 @@ Merges multiple files together. (This is also known as mosacking.) pctile ====== +(future -- likely to be part of pc2pc) + Chops one file up into multiple files. This could be the liblas::Chipper, leaf nodes of an index like an R-tree, or a simple quadtree build. diff --git a/doc/development/notes/bindings.txt b/doc/development/notes/bindings.txt index a58f80101b..bd6979fe14 100644 --- a/doc/development/notes/bindings.txt +++ b/doc/development/notes/bindings.txt @@ -9,6 +9,12 @@ Bindings We will support SWIG bindings for languages like Python and C#. We will avoid things that will make SWIG's life difficult, such as multiple -inheritance. +inheritance and templates. +SWIG doesn't support operator overloading, so provide alternative, "regular" +functions that do the same thing, e.g. equals() for oper== and oper!=. +In the bindings, we will not attempt to provide every member function for +the classes, nor indeed will we necessarily provide bindings for all the +classes. If you really need the "full" functionality of PDAL, you should +probably be using the native C++ interfaces. diff --git a/doc/development/notes/boost.txt b/doc/development/notes/boost.txt new file mode 100644 index 0000000000..cf28e714cb --- /dev/null +++ b/doc/development/notes/boost.txt @@ -0,0 +1,18 @@ +.. _boost: + +=================== +PDAL's Use of Boost +=================== + + +* we use boost + +* don't reinvent the wheel, use a boost package + +* ptrees + +* program_options + +* unit test + +* other packages we use diff --git a/doc/development/notes/building.txt b/doc/development/notes/building.txt new file mode 100644 index 0000000000..2f29a89754 --- /dev/null +++ b/doc/development/notes/building.txt @@ -0,0 +1,84 @@ +.. _building: + +======== +Building +======== + +This Note contains some notes on project dependencies, configurations, +building, etc. + + +Endianness +========== + +PDAL will not be supporting big-endian (BE) platforms at this time. We +recognize it would be good to do so, since endianness ports always seem to +shake out twiddly little bugs, but we don't have ready access a BE +development box. + +Where you are writing endian-sensitive code, you should comment it loudly +so future maintainers will know where to go to add BE support. + + +Project Dependencies - 3rd Party Libs +===================================== + +PDAL will use these libraries: + + +* Boost - for C++ library needs, such as streams or parsers (required) + +* libxml2 - Schema description, validation, and serialization (required) + +* GDAL - for spatial reference support (and rasterization) (optional) + +* libLAS - for LAS (compressed if available to libLAS) support. (optional) + +* MrSID - for LizardTech's MG4 lidar format (optional) + +* Oracle - OCI (optional) + +* libspatialindex - Rtree support, maybe quadtree if someone implements it for libspatialindex (optional) + +* *others?* + +Some of these libraries may be required (such as Boost). + + +Project Dependencies - Builds/Infrastructure +============================================ + +To develop and build PDAL, we'll be using these tools: + +* Sphinx and rst2pdf - for docs + +* CMake - for generating makefiles + +* Python - for miscellaneous scripting needs + +* *others?* + + +Buildbot +======== + +A buildbot will eventually be set up to prevent builds from breaking and to +keep the unit tests passing. + + +Platforms +========= + +We will support these platforms initially, corresponding to what the +developers are using regularly: + +* Windows - VS2010 (32- and 64-bit) - 2008 32-bit + +* Linux - gcc 4.x+ + +* Mac - gcc 4.0.1+ + +By virtue of CMake, other platforms may be supported such as VS 2008 or +Solaris, but we aren't targeting those specifically. + + diff --git a/doc/development/notes/coding-conventions.txt b/doc/development/notes/coding-conventions.txt index ec49e14bf3..3ac76d65db 100644 --- a/doc/development/notes/coding-conventions.txt +++ b/doc/development/notes/coding-conventions.txt @@ -16,11 +16,11 @@ Source Formatting We use astyle (http://astyle.sourceforge.net) as a tool to reformat C++ source code files in a consistent fashion. The file astylerc, at the top -of the hg repo, contains the default settings we use. +of the github repo, contains the default settings we use. Our conventions are: -* LF endings, not CRLF +* LF endings (unix style), not CRLF (windows style) * spaces, not tabs @@ -35,6 +35,10 @@ Our conventions are: foo(); } +* copyright header on every file + +* two spaces between major units, e.g. function bodies + Naming Conventions ================== @@ -58,14 +62,25 @@ Other Conventions variables * Surround all code with "namespace pdal {...}"; where justifiable, you - may introduce a nested namespace. Keep non-public names in the - pdal::detail\:: space. + may introduce a nested namespace. * Use exceptions for exceptional events that are not going to be handled directly within the context of where the event occurs. Avoid status codes. See exceptions.hpp for a set of pdal-specific exception types - you may throw. + you may throw. (See Note on error handling) + +* Describe use of "debug" and "verbose" settings. + +* Don't put member function bodies in the class declaration in the + header file, unless clearly justified for performance reasons. + Use the "inline" keyword in these cases(?). + +* Use const. + +* Don't put "using" declarations in headers. +* Document all public (and protected) member functions using + doxygen markup. Layout/Organization of Source Tree ================================== diff --git a/doc/development/notes/core-classes.txt b/doc/development/notes/core-classes.txt index 7314ecf41a..bdb26598bf 100644 --- a/doc/development/notes/core-classes.txt +++ b/doc/development/notes/core-classes.txt @@ -4,14 +4,52 @@ Core Classes ============ -This Notes describes the core classes used by PDAL. +This Notes describes the core classes used by PDAL, such as Range and +Dimension, that are not described in Notes of their own. + +The classes describing the Pipeline/Stage system, howver, are +described in a different Note. + +* Bounds + +* Color + +* Dimension, DimensionLayout + +* pdal_error + +* FileUtils + +* Filter, FilterIterator + +* MetadataRecord + +* MultiFilter, MultiFilterIterator + +* Options + +* PipelineManager, PipelineReader, PipelineWriter + +* PointBuffer * Range -* Dimension +* Reader, ReaderIterator + +* Schema, SchemaLayout + +* SpatialReference + +* Stage, StageBase, StageFactory, StageIterator + +* StreamManager + +* UserCallback -* ... +* Utils -[tbd] +* Vector +* Writer +* XMLSchema diff --git a/doc/development/notes/docs.txt b/doc/development/notes/docs.txt index 6ebfec52a9..8d5d9c7c9b 100644 --- a/doc/development/notes/docs.txt +++ b/doc/development/notes/docs.txt @@ -4,32 +4,47 @@ Docs ==== -There will be two kinds of documentation: API docs, and Everything Else. +PDAL comes with a few different types of documentation: + +* the API reference documentation + +* the command line applications user manual + +* Tutorials and Examples + +* the Developer Notes + +This documentation is all provided on the website, via Sphinx. API Documentation ================= -The API docs will be autogenerated by **[doxygen or something]** from the +The API docs will be autogenerated by doxygen from the header files. -**[need to provide info on what syntax to use]** +See the Note on Coding Conventions for information on what syntax to use +for this. + -These will be available on the website. +Command Line Apps +================= +The command line apps are documented via Sphinx using RST markup. -Everything Else + +Tutorials and Examples +====================== + +Similar to how libLAS was done, the PDAL website will contain a set of simple +code fragments that show how to do basic operations using PDAL. + +This documentation will also include sample Python/C# code. + + +Developer Notes =============== - -All the other documentation will be in the form of RST files, e.g. these -Notes, which will be autogenerated into web page content. Documentation -content will include: - -* usage guide for the command-line apps - -* developer-oriented information like PDAL architecture and core classes - -* information about specific file format readers and writers, e.g. details - on what versions of the LAS spec is supported - -* **[...?]** + +All development-related documentation, such as design notes, coding +conventions, and so on, will be documented in a Development Notes +section of the PDAL website, via Sphinx using RST markup. diff --git a/doc/development/notes/error_handling.txt b/doc/development/notes/error_handling.txt new file mode 100644 index 0000000000..f6a15600a5 --- /dev/null +++ b/doc/development/notes/error_handling.txt @@ -0,0 +1,41 @@ +.. _error_handling: + +========================= +Errors and Error Handling +========================= + +Exceptions +========== + +* throw exceptions, don't use status codes + +* use the pdal_error hiearchy (list) + + +PDAL Position on (Non)conformance +================================== + +PDAL proudly and unabashedly supports formal standards/specifications for +file formats. We recognize, however, that in some cases files will not +follow a given standard precisely, due to an unclear spec or simply out of +carelessness. + +When reading files that are not formatted correctly: + +* PDAL may try to compensate for the error. This is typically done when as + a practical matter the market needs support for well-known or pervasive, + but nonetheless "broken", upstream implementations. + +* PDAL may explicitly reject such files. This is typically done where we do + not wish to continue to promote or support mistakes that should be fixed + upstream. + +PDAL will always write correctly formatted files. In some cases, however, +PDAL may choose to offer as an option the ability to break the standard if, +as a practical matter, doing so would significantly aid the market. Such an +option would never be the default behavior, however. + +For files that are conformant but which lie, such as the extents in the +header being wrong, we will generally offer both the ability to propagate +the "wrong" information and the ability to helpfully correct it on the fly; +the latter is generally our default position. diff --git a/doc/development/notes/future.txt b/doc/development/notes/future.txt index 8b748b0ec2..c179de8f7a 100644 --- a/doc/development/notes/future.txt +++ b/doc/development/notes/future.txt @@ -7,3 +7,6 @@ PDAL Project Notes: Future This Note is just a dumping ground for things that have yet to be discussed. * SSE? + +* Use case (9b): same as (9a), but as an in-place operation. This is a +tricky one, if we want to do it, because it doesn't fit the pipeline model. diff --git a/doc/development/notes/index.txt b/doc/development/notes/index.txt index 6a26fb833a..bf737401fd 100644 --- a/doc/development/notes/index.txt +++ b/doc/development/notes/index.txt @@ -4,22 +4,37 @@ Development Notes ****************************************************************************** +The design, architecture, and implementation of PDAL is captured in this +set of Notes. Each Note focues on a single major topic, such +as Use Cases, Pipeline Design, or Testing. These Notes are intended to +capture design decisions, rationales, and points still to be decided. + +Editorial comments, and points yet to be resolved, are noted in the text +**[like this]**. + +The proper spelling of the project name is PDAL, in uppercase. It is +pronounced to rhyme with "GDAL". + + .. toctree:: :maxdepth: 0 apps bindings + boost + building c-api coding-conventions core-classes docs + error_handling goals future - intro io - misc + options pipeline pipeline_xml + project_management stages testing use-cases \ No newline at end of file diff --git a/doc/development/notes/intro.txt b/doc/development/notes/intro.txt deleted file mode 100644 index ed664a64bf..0000000000 --- a/doc/development/notes/intro.txt +++ /dev/null @@ -1,21 +0,0 @@ -.. _intro: - -========================== -PDAL Project Notes: Intro -========================== - -The design, architecture, and implementation of PDAL will be captured in a -set of Notes. Each Note file will, in general, discuss a single topic, such -as Use Cases, Pipeline Design, or Testing. Notes files are intended to -capture all the decisions made on that topic, as well as keep track of -points still to be decided. - -Better yet, these notes can serve as fodder for eventual public -documentation for PDAL developers and users. - -Notes files will be RST text files and will live in the hg repo under -doc/notes. - -Editorial comments, and points yet to be resolved, are noted in the text -**[like this]**. - diff --git a/doc/development/notes/io.txt b/doc/development/notes/io.txt index d2d7cfeeea..9726efa85b 100644 --- a/doc/development/notes/io.txt +++ b/doc/development/notes/io.txt @@ -1,8 +1,8 @@ .. _io: -=== -I/O -=== +===================== +I/O Streams and Files +===================== Given the sheer volume of data being processed by PDAL apps, it is very important we pay attention to I/O speed. @@ -24,14 +24,9 @@ specific workflow, PDAL will follow these guidelines. * Streams should be instantiated using Boost's I/O library, not the native std\:: library. -* Implementations should accept as inputs std\:: base classes, and use - std\:: library calls. That is, the ctor for a class like LasReader should - accept a std\::istream& stream instead of a std\::string file name or a - FILE* handle. **[not sure why I feel this way - thoughts?]** - -* Implementations should use std\:: library calls as opposed to methods - which require Boost-specific stream classes. (Given the previous - guideline, this guideline would seem to have to be followed anyway.) +* Where appropriate, public interfaces should accept as inputs std\:: base classes + (as opposed to Boost-based classes). Implementations should use std\:: library + calls as opposed to methods which require Boost-specific stream classes. * The PDAL API provides helper functions for opening and closing file streams. Applications should prefer to use those over writing their own. @@ -41,23 +36,3 @@ specific workflow, PDAL will follow these guidelines. * Formatted (text, non-binary) I/O is (typically) not performance sensitive. For consistency, however, C++ stream I/O should be used here too; do not use printf/scanf. - - -Endianness -========== - -PDAL will not be supporting big-endian (BE) platforms at this time. We -recognize it would be good to do so, since endianness ports always seem to -shake out twiddly little bugs, but we don't have ready access a BE -development box. - -Where you are writing endian-sensitive code, you should comment it loudly -so future maintainers will know where to go to add BE support. - - -Best Practices -============== - -**[we should provide some sample code showing examples of good I/O styles, -e.g. which APIs to use. mpg even thinks this might turn into a facade -class which exposes the basics people should use and nothing more]** diff --git a/doc/development/notes/misc.txt b/doc/development/notes/misc.txt deleted file mode 100644 index 7e5e73f581..0000000000 --- a/doc/development/notes/misc.txt +++ /dev/null @@ -1,129 +0,0 @@ -.. _misc: - -==== -Misc -==== - -This document contains several issues worth documenting but are not, or are -not yet, substantive enough to merit their own separate files. - - -Spelling -======== - -The proper spelling of the project name is PDAL (uppercased P and C, -like libLAS is spelled). - - -Management -========== - -We strive for consensus, but at the end of the day Howard is the benevolent -dictator. - - -PDAL Package Contents -====================== - -PDAL will, at some level, consist of the following parts: - -* a set of C++ libraries - -* a set of command line apps - -* a set of tests - -* a set of SWIG bindings - -* documentation - - -Project Dependencies - 3rd Party Libs -===================================== - -PDAL will use these libraries: - - -* Boost - for C++ library needs, such as streams or parsers (required) - -* libxml2 - Schema description, validation, and serialization (required) - -* GDAL - for spatial reference support (and rasterization) (optional) - -* libLAS - for LAS (compressed if available to libLAS) support. (optional) - -* MrSID - for LizardTech's MG4 lidar format (optional) - -* Oracle - OCI (optional) - -* libspatialindex - Rtree support, maybe quadtree if someone implements it for libspatialindex (optional) - -* *others?* - -Some of these libraries may be required (such as Boost). - - -Project Dependencies - Builds/Infrastructure -============================================ - -To develop and build PDAL, we'll be using these tools: - -* Sphinx and rst2pdf - for docs - -* CMake - for generating makefiles - -* Python - for miscellaneous scripting needs - -* *others?* - - -Buildbot -======== - -A buildbot will eventually be set up to prevent builds from breaking and to -keep the unit tests passing. - - -Platforms -========= - -We will support these platforms initially, corresponding to what the -developers are using regularly: - -* Windows - VS2010 (32- and 64-bit) - 2008 32-bit - -* Linux - gcc 4.x+ - -* Mac - gcc 4.0.1+ - -By virtue of CMake, other platforms may be supported such as VS 2008 or -Solaris, but we aren't targeting those specifically. - - -PDAL Position on (Non)conformance -================================== - -PDAL proudly and unabashedly supports formal standards/specifications for -file formats. We recognize, however, that in some cases files will not -follow a given standard precisely, due to an unclear spec or simply out of -carelessness. - -When reading files that are not formatted correctly: - -* PDAL may try to compensate for the error. This is typically done when as - a practical matter the market needs support for well-known or pervasive, - but nonetheless "broken", upstream implementations. - -* PDAL may explicitly reject such files. This is typically done where we do - not wish to continue to promote or support mistakes that should be fixed - upstream. - -PDAL will always write correctly formatted files. In some cases, however, -PDAL may choose to offer as an option the ability to break the standard if, -as a practical matter, doing so would significantly aid the market. Such an -option would never be the default behavior, however. - -For files that are conformant but which lie, such as the extents in the -header being wrong, we will generally offer both the ability to propagate -the "wrong" information and the ability to helpfully correct it on the fly; -the latter is generally our default position. diff --git a/doc/development/notes/options.txt b/doc/development/notes/options.txt new file mode 100644 index 0000000000..bbd33a52b5 --- /dev/null +++ b/doc/development/notes/options.txt @@ -0,0 +1,8 @@ +.. _options: + +================== +The Options System +================== + + +[tbd] diff --git a/doc/development/notes/pipeline_xml.txt b/doc/development/notes/pipeline_xml.txt index a37f9f5092..efb91aa90b 100644 --- a/doc/development/notes/pipeline_xml.txt +++ b/doc/development/notes/pipeline_xml.txt @@ -9,73 +9,47 @@ This note describes the XML structure used for describing pipelines. The pipeline XML syntax mirrors very closely the Stage class hierarchy and construction format used by the C++ API. -There are two kinds of pipelines that can be expressed: "writer pipelines" -have Writer stages as their endpoint, and "reader pipelines" have general -Stage stages (Reader/Filter/Multifilter) as their endpoint. - We have two use cases specifically in mind: * a command-line app that reads an XML file to allow a user to easily construct arbitrary writer pipelines, as opposed to having to build apps custom to individual needs with arbitary options, filters, etc - * within the Python environment, the user can provide XML for a reader + * a user can provide XML for a reader pipeline, construct it via a simple call to the PipelineManager API, - and then use native Python to perform the read and do any processing - of the emitted points + and then use the Stage read() function to perform the read and and + then do any processing of the points. This style of operation + is very appropriate for using PDAL from within environments + like Python where the focus is on just getting the points, + as opposed to complex pipeline construction. Examples ======== -A writer pipeline:: +A pipeline which reads two files, input1.las and input2.las, crops the points +to a certain bounding box, and then writes the result as a compressed LAS file, +output.laz:: - - - drivers.las.writer - - - filters.crop - - - drivers.las.reader - - + + + + + + + + + + + + + + -A reader pipeline:: - - - - - filters.crop - - - drivers.las.reader - - - - - Syntax Specification ==================== @@ -93,6 +67,11 @@ Syntax Specification * element, for writer pipelines * or or element, for reader pipelines + + * attributes: + + * the "version" attribute must appear exactly once; the value of this + attribute shall be the string "1.0" * @@ -100,11 +79,15 @@ Syntax Specification * child elements: - * exactly one element - * zero or more