Merge pull request #516 from mwoehlke-kitware/update-manual

Overhaul camera calibration documentation and improve polish

doc/_static/css/custom.css

@@ -1,3 +1,7 @@
+p {
+  text-align: justify;
+}
+
 pre.wrap {
   white-space: pre-wrap !important;
 }
@@ -16,3 +20,53 @@ pre.literal-block,
   border: 1px solid #e1e4e5;
   padding: 2px 5px;
 }
+
+.menu {
+  background: #f8fbfc;
+  padding: 0.1em 0.3em;
+  border: 1px solid #f4f7f8;
+  font-style: italic;
+}
+
+.action {
+  background: #f4f4f4;
+  padding-left: 0.3em;
+  padding-right: 0.3em;
+  border: 1px solid #e1e4e5;
+  border-radius: 0.4em;
+  font-weight: 700;
+}
+
+img.icon-16 {
+  position: relative;
+  bottom: calc(0.5em - 7px);
+  height: 16px;
+  width: 16px;
+}
+
+.glyph {
+  position: relative;
+  bottom: calc(0.5em - 6px);
+  height: 1em;
+}
+
+.shortcut {
+  background: #eceff0;
+  padding: 0.1em 0.3em;
+  border: 1px solid;
+  border-color: #f4f7f8 #e1e4e5 #e1e4e5 #f4f7f8;
+  border-radius: 0.4em;
+  font-weight: 700;
+  font-size: 80%;
+  position: relative;
+  top: -0.05em;
+}
+
+.math {
+  font-family: math, serif;
+  font-style: italic;
+}
+
+.math.script {
+  font-family: cursive, math, serif;
+}

doc/advancedconfig.rst

@@ -1,7 +1,5 @@
 .. _advancedconfig:
 
-.. include:: version.rst
-
 ==============================
 Advanced Configuration Options
 ==============================

doc/advancedtools.rst

@@ -59,7 +59,8 @@ detect and correct for this ambiguity, so this the manual correction is rarely n
 .. figure:: /images/necker_reversal_after.png
    :align: center
 
-   *Effect of Necker Reversal on camera orbits.  Bottom cameras are flipped and upside down.*
+   Effect of Necker Reversal on camera orbits.
+   Bottom cameras are flipped and upside down.
 
 Align
 =======

doc/cameracalibration.rst

@@ -4,85 +4,147 @@
 Manual Camera Calibration
 =========================
 
-This section describes the functions for manaul camera calibration capabilities in TeleSculptor (TS)
-and steps to achieve, possibly correct, and export a camera calibration model.
+This section describes the functions
+for manual camera calibration capabilities in TeleSculptor (TS)
+and steps to achieve, possibly correct, and export
+a camera calibration model.
 
 Preparation
 ===========
 
-TeleSculptor can work with ffmpeg-readable videos (e.g. mpg, mp4, avi) or with image sequences.
-The latter can be specified by listing paths to individual images in a plain text file (e.g. images.txt)
-using one line per path/to/image/file. Standard image formats (e.g. bmp, png, jpg) are supported.
+TeleSculptor can work with ffmpeg-readable videos
+(e.g. mpg, mp4, avi) or with image sequences.
+The latter can be specified by listing paths to individual images
+in a plain text file (e.g. images.txt) using one line per path/to/image/file.
+Standard image formats (e.g. bmp, png, jpg) are supported.
 
-TeleSculptor has a capability to import PLY mesh or point cloud files via File/Import/Mesh.
+TeleSculptor has a capability to import PLY mesh or point cloud files
+via :menu:`File` |rarrow| :menu:`Import` |rarrow| :menu:`Mesh`.
 Point clouds are imported as faceless meshes.
-The imported mesh is used as a 3D target for calibration. RGB[A] color space is currently supported.
+The imported mesh is used as a 3D target for calibration.
+RGB[A] color space is currently supported.
 The alpha channel can be used to set mesh transparency levels.
 
-We recommend geo-registering the mesh before importing it to TS to have the output calibrated
-camera parameters in mesh metric units and also automatically geo-registered with the 3D model.
+We recommend geo-registering the mesh before importing it into TS
+to have the output calibrated camera parameters in mesh metric units
+and also automatically geo-registered with the 3D model.
 
 Calibration
 ===========
 
-Start the TeleSculptor GUI and proceed with the following steps in a new or an existing project.
-
-.. |gcp_button| image:: /../gui/icons/22x22/location.png
+Start the TeleSculptor GUI and proceed with the following steps
+in a new or an existing project.
 
 New project
 -----------
 
-Choose menu: File/New Project… (Ctrl+N) and select a folder for the new project,
-creating one if necessary. The project ini file name would have the same name by default.
-
-Choose menu: File/Import/Imagery… (Ctrl+O,I) and select a video or an image sequence text file.
-Large videos would imply a long search for metadata, which can be canceled via menu: Compute/Cancel.
-
-Choose menu: File/Import/Mesh… (Ctrl+O,M) and select an PLY file with a mesh or a point cloud,
-which should appear in the 3D viewer.
-
-Scroll to the desired video/image frame using the Camera Selection slider
-and observe it in the Camera View pane. Rotate the 3D model to the desired view,
-e.g. resembling the current image.
-
-To place/edit ground control points (GCP), click the 3D viewer GCP tool |gcp_button|
-to start GCP creation and editing. Use Ctrl+click to place GCP buttons.
-Drag them as needed to refine their locations.
-Place six or more GCPs on the 3D model at the feature points visible from the camera view,
-observe their IDs appear in the Ground Control Points list, name them, if desired.
-
-To build 3D-2D point correspondences, click camera registration points (CRP) tool |gcp_button|
-in the Camera View.
-For each GCP in the list (also highlighted in the 3D viewer):
-select a 3D point, then click on the Camera View image, where its 2D projection is expected to be.
-Use Ctrl+click to place a CRP, and drag it with the mouse to refine its location.
-Observe the target-like icon in the GCP list for the 3D point having a corresponding 2D point.
-Select another 3D GCP and create-click-move its 2D counterpart.
-Continue with the 3D-2D correspondence process until the count of 6 or more.
-
-In the Camera View, select the [Place/edit CRP] drop-down [Compute Camera] to invoke the calibration process.
-Check the Log Viewer for the re-projection error or for any error messages, e.g. not enough calibration points.
-Observe the 3D viewer for the image projection corresponding to the newly calibrated camera.
-Click the tree-expandable marker in the GCP list next to a GCP point to see the frame numbers
-this point was used in for camera calibration.
-Double-Click on the desired frame number to jump to that frame.
+Create a new project using :menu:`File` |rarrow| :menu:`New Project`
+(:shortcut:`Ctrl+N`)
+and select a folder for the new project, creating one if necessary.
+The project :path:`.ini` file name will have the same name by default.
+
+Select a video or image sequence text file
+using :menu:`File` |rarrow| :menu:`Import` |rarrow| :menu:`Imagery...`
+(:shortcut:`Ctrl+O, I`).
+Opening a large videos will initiate a potentially long search for metadata,
+which may be canceled via the menu action
+:menu:`Compute` |rarrow| :menu:`Cancel`.
+
+Select a PLY file with a mesh or a point cloud
+using :menu:`File` |rarrow| :menu:`Import` |rarrow| :menu:`Mesh...`
+(:shortcut:`Ctrl+O, M`).
+The loaded 3D geometry should appear in the 3D world view.
+
+Locate the desired video frame or image
+using the scrubber or spin box in the Camera Selection pane.
+Rotate the model in the world view as needed
+so the desired point correspondences are visible.
+
+Select :action:`location Edit Ground Control Points` (in the world view)
+to activate editing mode for ground control points in the world view.
+Place points using :shortcut:`Ctrl+Click`.
+Telesculptor attempts to place points "on" the loaded mesh,
+but it may be helpful to rotating the world view
+to verify that points are placed in the right 3D location.
+While editing, points may be dragged as needed.
+Newly placed points will appear in the Ground Control Points list,
+and may be given names if desired.
+
+To create 3D-to-2D point correspondences,
+select :action:`location Edit Registration Points` (in the camera view).
+Ensure that the desired 3D point is selected in the Ground Control Points list
+(the selected point is highlighted in a different color).
+Place a corresponding point using :shortcut:`Ctrl+Left Click`,
+positioned to match the location in the camera image
+to which the 3D ground control point would be projected.
+The Ground Control Points list will show a glyph (|glyph-registered|)
+to indicate that the point has a corresponding camera registration point
+associated with the active video frame or image.
+Once created, points may be dragged as needed
+to fine tune their locations.
+Select another 3D ground control point and repeat
+until at least six correspondences have been created
+for the active video frame or image.
+The relevant edit mode actions may be used at any time
+to switch between editing points in the camera or world views.
+
+Once six or more correspondences have been defined,
+the camera pose calibration process may be invoked
+from the :menu:`Compute Camera` action located in the drop-down
+associated with :action:`location Edit Registration Points`.
+Check the Log Viewer for the re-projection error
+or for any error messages (e.g. insufficient calibration points).
+Upon successful calibration,
+the world view will show the projected camera image
+(if :action:`image Show Camera Frame Image` is enabled),
+and the camera view will show the ground control points
+projected onto the camera image.
+
+.. TODO move to interface documentation
+
+    Click the tree-expandable marker in the GCP list next to a GCP point
+    to see the frame numbers this point was used in for camera calibration.
+    Double-Click on the desired frame number to jump to that frame.
 
 .. image:: /screenshot/telesculptor_screenshot_calibration.png
-   :alt: Manual Camera Calibration Screenshot
+  :alt: Manual Camera Calibration Screenshot
+
+Repeat this process on additional frames.
+Point locations may be adjusted as needed, as described above,
+to refine the camera pose calibration.
+Camera calibrations may be refined at any time;
+it is not necessary to "finalize" one camera pose
+before calibrating additional camera poses.
+
+When satisfied,
+:menu:`File` |rarrow| :menu:`Export` |rarrow| :menu:`KRTD Cameras...`
+may be used to save the parameters of all calibrated cameras
+to KRTD text files.
+Each file will capture
+the intrinsic matrix :var:`K`,
+the rotation matrix,
+the translation vector
+and the lens Distortion parameters.
+
+:menu:`File` |rarrow| :menu:`Export` |rarrow| :menu:`Ground Control Points...`
+will export all of the ground control points
+and their corresponding camera registration points
+as a GeoJSON_-like file.
+(Only the geodetic locations, if available, conform to GeoJSON;
+the world and camera locations are stored as extensions.)
+This will output all points,
+whether or not calibration was performed successfully.
 
-Move 3D and/or 2D points to adjust the camera calibration as needed.
-Zoom in and out, if necessary in both 3D and 2D views.
-Scroll to a different frame and repeat the camera calibration process.
+Existing project
+----------------
 
-Choose menu: File/Export/KRTD Cameras… to output the parameters of all calibrated cameras to KRTD text files,
-each capturing the intrinsic matrix K, Rotation matrix, Translation vector and lens Distortion parameters.
+Open an existing project using :menu:`File` |rarrow| :menu:`Open Project...`
+(:shortcut:`Ctrl+O, P`).
+The project's corresponding video or image sequence,
+and the 3D mesh or point cloud, will be loaded automatically.
+Follow the calibration procedure described for new projects, above.
 
-Choose menu: File/Export/Ground Control Points… to output all the GCPs
-and their corresponding 2D CRPs for each frame 3D-2D correspondence were built.
+.. |glyph-registered| image:: images/registered.svg
+  :class: glyph
 
-Existing project
-----------------
-Choose, menu: File/Open Project… (Ctrl+O,P) to open an existing project,
-its corresponding video/image sequence and the 3D mesh or point cloud,
-which are loaded automatically.
-Follow the calibration procedure described in the New project section.
+.. _GeoJSON: https://geojson.org/

doc/conf.py

@@ -4,7 +4,9 @@
 # list see the documentation:
 # https://www.sphinx-doc.org/en/master/usage/configuration.html
 
-from docutils import nodes
+import os.path
+
+from docutils import nodes, utils
 from docutils.parsers.rst.states import Struct
 
 # -- Path setup --------------------------------------------------------------
@@ -44,7 +46,11 @@
 # List of patterns, relative to source directory, that match files and
 # directories to ignore when looking for source files.
 # This pattern also affects html_static_path and html_extra_path.
-exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']
+exclude_patterns = [
+    'Thumbs.db',
+    '.DS_Store',
+    'replacements.rst',
+]
 
 rst_epilog = f'''
 .. include:: replacements.rst
@@ -86,6 +92,55 @@
 
 # -- Customizations ----------------------------------------------------------
 
+icon_root = '../gui/icons'
+icon_sizes = [22, 16]
+srcdir = ''
+
+def locate_icon(name, size):
+    for d in [f'{size}x{size}@2', f'{size}x{size}']:
+        p = os.path.join(icon_root, d, f'{name}.png')
+        print(p)
+        if os.path.isfile(os.path.join(srcdir, p)):
+            return p
+
+    return None
+
+def build_icon(rawtext, text, lineno, inliner, options={}, sizes=icon_sizes):
+    if text in ['-', 'blank']:
+        return None
+
+    options['alt'] = utils.unescape(text)
+    for s in sizes:
+        uri = locate_icon(text, s)
+        print(s, uri)
+        if uri is not None:
+            options['uri'] = uri
+            options['classes'] = [f'icon-{s}']
+            return nodes.image(rawtext, **options)
+
+    return None
+
+def action_role(name, rawtext, text, lineno, inliner, options={}, content=[]):
+    parts = text.split()
+    print(parts)
+
+    # Build icon element
+    icon = build_icon(rawtext, parts[0], lineno, inliner, sizes=[16, 22])
+
+    # Create node
+    options['classes'] = ['action']
+    node = nodes.inline(rawtext, '', **options)
+
+    if icon is not None:
+        node += icon
+        node += nodes.Text(' ', ' ')
+
+    title = ' '.join(parts[1:])
+    node += nodes.Text(title, title)
+
+    # Return resulting node
+    return [node], []
+
 def make_parsed_text_role(class_names=[], node_class=nodes.inline):
     def parsed_text_role(name, rawtext, text, lineno, inliner,
                          options={}, content=[]):
@@ -108,4 +163,13 @@ def parsed_text_role(name, rawtext, text, lineno, inliner,
     return parsed_text_role
 
 def setup(app):
+    global srcdir
+    srcdir = app.srcdir
+
+    app.add_role('var', make_parsed_text_role(class_names=['math', 'script']))
+    app.add_role('math', make_parsed_text_role(class_names=['math']))
     app.add_role('path', make_parsed_text_role(class_names=['filepath']))
+    app.add_role('menu', make_parsed_text_role(class_names=['menu']))
+    app.add_role('shortcut', make_parsed_text_role(class_names=['shortcut']))
+
+    app.add_role('action', action_role)