Skip to content

Commit

Permalink
Merge pull request #529 from Kitware/release
Browse files Browse the repository at this point in the history
Release
  • Loading branch information
mleotta committed Sep 11, 2021
2 parents fc54c70 + 34c443d commit 9e7d98b
Show file tree
Hide file tree
Showing 9 changed files with 86 additions and 112 deletions.
41 changes: 29 additions & 12 deletions doc/computeoptions.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,30 +8,47 @@ Compute Options
:align: right
:scale: 50 %

A few basic switches are now available under the Compute->Options menu to control the behavior of some of the algorithms. Each of these options corresponds to a boolean
configuration value in the project file. Checking these options will save that option in the project file. In the future, we will provide a visual interface to configure many more
of the TeleSculptor algorithm options. For now these basic options are presented in the menu:
A few basic switches are available under the Compute->Options menu
to control the behavior of some of the algorithms.
Each of these options corresponds to a boolean
configuration value in the project file.
Checking these options will save that option in the project file.
In the future, we will provide a visual interface to configure many more
of the TeleSculptor algorithm options.
For now these basic options are presented in the menu:

Ignore Metadata
=================

When checked, this option causes the Initialize Cameras/Landmarks algorithm to ignore any metadata that was loaded with the video. This option is useful because it is often the case
that the metadata is incorrect and negatively impacts the algorithm rather than helping it.
When checked, this option causes the Initialize Cameras/Landmarks algorithm
to ignore any metadata that was loaded with the video.
This option is useful because it is often the case that the metadata is
incorrect and negatively impacts the algorithm rather than helping it.

Variable Lens
===============

By default, TeleSculptor assumes that all frames in a video are collected with the same lens and zoom setting. The intrinsic camera parameters are shared across the entire
sequence. This assumption gives the best results as long as the assumption holds true. When the assumption does not hold, and the lens zooms or is changed in the middle of the
sequence, checking “Variable Lens” will instruct TeleSculptor to estimate unique camera intrinsic parameters for each video frame.
By default, TeleSculptor assumes that all frames in a video are collected
with the same lens and zoom setting.
The intrinsic camera parameters are shared across the entire sequence.
This assumption gives the best results as long as the assumption holds true.
When the assumption does not hold, and the lens zooms or is changed
in the middle of the sequence, checking “Variable Lens” will instruct
TeleSculptor to estimate unique camera intrinsic parameters for each
video frame.

Fix Geo-Origin
================

TeleSculptor automatically selects a local origin near the centroid of the data and records the geographic location of this origin point. When the data is updated by running
algorithms that origin point is recalculated and may change. In some cases, there are benefits to specifying the geographic origin to use and keeping it fixed, for example, forcing
two data sets to share a common local origin for easier comparison. Checking “Fix Geo-Origin” instructs TeleSculptor to keep the current geographic origin and not recalculate a new
one.
TeleSculptor automatically selects a local origin near the centroid
of the data and records the geographic location of this origin point.
When the data is updated by running algorithms that origin point is
recalculated and may change.
In some cases, there are benefits to specifying the geographic origin to use
and keeping it fixed, for example, forcing two data sets to share
a common local origin for easier comparison.
Checking “Fix Geo-Origin” instructs TeleSculptor to keep the current
geographic origin and not recalculate a new one.

Use GPU
=========
Expand Down
Binary file removed doc/images/colored_fused_mesh.png
Binary file not shown.
Binary file added doc/images/mesh_colored_by_mean.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added doc/images/mesh_colored_by_single_frame.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified doc/images/mesh_colored_by_views.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified doc/images/mesh_colorization_menu.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
100 changes: 8 additions & 92 deletions doc/interface.rst
Original file line number Diff line number Diff line change
Expand Up @@ -830,102 +830,18 @@ Compute Menu
and may not be able to run
if the ROI is too large for the GPU memory.

:icon:`cancel` Cancel
Cancel the current running tool.
Most, but not, all tools support this cancel action,
which causes them to exit early and return a partial result.

:icon:`blank` Advanced
Provides access to additional, lower level tools.
See :doc:`advancedtools` for details.

.. TODO consolidate following with advancedtools.rst
Compute Menu |rarrow| Advanced
------------------------------

:icon:`blank` Filter Tracks
Filter the tracks to retain a smaller subset of tracks
that is still representative of the original set.
The intent is to make bundle adjustment (:action:`- Refine Solution`)
faster without loosing critical constraints.
The filter attempts to remove the shortest tracks
that span the same frames already covered by longer tracks.

:icon:`blank` Triangulate Landmarks
For each available feature track,
back project rays from the cameras that contain each track state
and intersect those rays in 3D to estimate the location of a 3D landmark.
This requires both feature tracks and a reasonably accurate set of cameras.

:icon:`blank` Refine Solution
Applies bundle adjustment to the cameras and landmarks
in order to refine the quality of the 3D reconstruction.
It aims to minimize this distance
between the landmarks projected into each image by the cameras
and the observed location of the corresponding feature tracks.

:icon:`blank` Reverse (Necker)
Transforms the cameras and landmarks
in a manner intended to break the refinement process
out of a degenerate optimization
(which can occur due to the Necker cube phenomena\ [#nc]_),
by computing a best fit plane to the landmarks,
mirroring the landmarks about said plane,
and rotating the cameras 180 |deg|
about their respective optical axes
and 180 |deg| about the best fit plane normal
where each camera's optical axis intersects said plane.

:icon:`blank` Align
Applies a similarity transformation to the camera and landmark data
so that the data has a standard ("canonical") alignment.
Particularly, this attempts to orient the data
so that the ground plane is parallel with the :f:`z = 0` plane
(with the cameras in the :f:`+Z` direction).
Additionally, the landmarks will be centered about the origin
and scaled to an approximate variance of :f:`1.0`.

:icon:`blank` Save Key Frames
Iterate through a video and save every key frame as an image file
in a subdirectory of the project directory.
Key frames are marked by the feature tracking algorithm.

:icon:`blank` Compute Single Depth Map
Estimate a dense depth map
and corresponding point cloud
for the current frame.
This requires a valid camera on the current frame
as well as cameras on other frames for triangulation.
It computes the solution within the active ROI
and shows an incremental visualization of how the solution evolves.

Compute Menu |rarrow| Options
-----------------------------

:icon:`blank` Ignore Metadata
Ignore the metadata fields (e.g. from KLV)
that are attached to the imagery.
If this option is set, metadata will not be used
in camera estimation or in geolocation.
The main reason to use this option
is when the metadata is known to be invalid.

:icon:`blank` Variable Lens
This option allows estimation of camera models
in which the lens model (e.g. focal length) can change over time.
If this option is off, all cameras will share the same lens model.
If the lens does not change between frames,
it is best to use a single model.

:icon:`blank` Fix Geo-Origin
If checked, this option will prevent TeleSculptor
from resetting the geospatial origin to a point centered on the data.
Normally, TeleSculptor automatically chooses an origin,
but there are use cases where it is helpful
to specify an origin and keep it fixed,
such as comparing results across different data of the same location.

:icon:`blank` Use GPU
If checked, this option will use a CUDA implementation
of some algorithms (currently, depth map fusion).
This requires an NVIDIA GPU with sufficient GPU RAM.
If unchecked, the CPU implementation will be used instead.
:icon:`blank` Options
Provides options to alter how the tools operate.
See :doc:`computeoptions` for details.

.. _view-menu:

Expand Down
57 changes: 49 additions & 8 deletions doc/processingsteps.rst
Original file line number Diff line number Diff line change
Expand Up @@ -339,20 +339,40 @@ tend to appear in areas with only a few views.
Colorize Mesh
==============

The fused mesh is provided initially in a solid grey color. To add color, use the drop down menu under the Volume Display button ( |volume_display_button| ). Check the
*Colorize surface* box to enable color. There two options to color the mesh. The *Current frame* option always projects the current frame onto the mesh and the color updates when
you play back the video. The *All frames* option estimates a static mesh coloring by projecting multiple images onto the surface and combining them. The *Frame sampling* combo box
allows configuration of how frequently to sample frames for coloring. Smaller sampling uses more frames for better color but more computation time. Press the Compute button to
compute mesh color (note: a progress bar is not yet implemented for this step). When complete, the *Color display* option can be changed without needing to recompute color. The
recommended color display option is *MedianColoration*, however, *MeanColoration* is often quite similar. There are also special colorization options to gain insight into the data.
The *Normals* option colors the mesh by surface normal direction, and the *NbProjectedDepthMap* option colors by the number of depth map views that observed each part of the surface.
The fused mesh is provided initially in a solid grey color.
To add color, use the drop down menu under the
Volume Display button ( |volume_display_button| ).
Use the left drop down menu to select between *No Color* and *Image Color*.
If, and only if, a mesh was loaded from a file,
a third option for *Original Color* is available.
*Original Color* displays the color loaded from the mesh file.

.. figure:: /images/mesh_colorization_menu.png
:align: center

The mesh colorization menu options.

.. figure:: /images/colored_fused_mesh.png
If *Image Color* is selected then the mesh vertices are colored by
projecting color from one or more images onto the mesh.
Additional settings on how the mesh is colored
are in the right drop down menu.
The *Current frame* option always projects the current frame onto the mesh
and the color updates when you play back the video.
The *All frames* option estimates a static mesh coloring by
projecting multiple images onto the surface and combining them.
The *Frame sampling* combo box allows configuration of
how frequently to sample frames for coloring.
Smaller sampling uses more frames for better color but more computation time.
Press the *Compute* button to compute mesh color.
When complete, the *Color display* option can be changed
without needing to recompute color.
The recommended color display option is *median*,
however, *mean* is often quite similar.
The *count* option colors by the number of depth map views
that observed each part of the surface.


.. figure:: /images/mesh_colored_by_mean.png
:align: center

A fused mesh colored by the mean of multiple frames.
Expand All @@ -363,6 +383,27 @@ The *Normals* option colors the mesh by surface normal direction, and the *NbPro
A fused mesh colored by the number of views
that see each part of the surface.

Additional check boxes are available to enable removal of color
on some surfaces that should not be colored.
The *Remove masked surface color* option uses a mask video loaded
into TeleSculptor and does not apply color at points under this mask.
The *Remove occluded surface color* option enables visibility checking
such that occluded surfaces will not receive color.
The *Occlusion threshold* varies the sensitivity of
the occlusion depth test.
Negative values cause larger occluded regions while
positive values cause smaller occluded regions.

.. figure:: /images/mesh_colored_by_single_frame.png
:align: center

A fused mesh colored by a single frame using occlusion detection.

.. figure:: /images/mesh_colored_by_single_frame_no_occ.png
:align: center

A fused mesh colored by a single frame without occlusion detection.

Export Data
============

Expand Down

0 comments on commit 9e7d98b

Please sign in to comment.