Tools for the Generation and Visualization of Large-scale Three-dimensional Reconstructions from Image Data.
C C++ HTML TeX Shell Python Other
Switch branches/tags
Nothing to show
Clone or download
mattjr IMPORTANT DEFAULT CHANGE: Now the variance images generate now produc…
…e var not on blending images but on raw pixels values you can restore previous results using the USE_BLENDING define in VipsSampler.cpp. Additionally the functionality to produce non blending mosaics has been restored omit the --blend flag to renderSquareAA
Latest commit 0cb4635 Jun 3, 2015
Permalink
Failed to load latest commit information.
CMVS-PMVS
RadialUndistort
SiftGPU
VirtualPlanetBuilder
ann big set of changes add ann and code for trinagle texture from bundler… Nov 6, 2013
gts-0.7.6 update with per predicates Jun 7, 2013
isam Added SE2 and SE2 headers from isam, rather than requiring the isam l… Sep 5, 2014
pba
re_vision Merge branch 'master' of https://github.com/mattjr/structured Jun 18, 2014
single_mesh_gen The jump from threadedStereo to Structured (ie Pavlopetri) Sep 12, 2012
spatialindex-src-1.6.1 new atlas mechnism for VT using seperate file clean up Sep 12, 2012
swrender IMPORTANT DEFAULT CHANGE: Now the variance images generate now produc… Jun 2, 2015
triangulation not sure about local changes Jan 19, 2014
trimesh2 trimesh code added and meshproc moved in Jun 4, 2013
ts-sonar Fixes for DIDSON images, now 3D mosaicing works Sep 22, 2014
vcgapps IMPORTANT DEFAULT CHANGE: Now the variance images generate now produc… Jun 2, 2015
vrip opencv threading issues May 6, 2015
vsfm
.gitignore added some sonar-related stuff. Sep 17, 2014
ANNWrapper.cpp big set of changes add ann and code for trinagle texture from bundler… Nov 6, 2013
ANNWrapper.h big set of changes add ann and code for trinagle texture from bundler… Nov 6, 2013
AddCompilerFlag.cmake MULTI-MODULE COMMIT fixed graphics context fail override and fixed Sep 12, 2012
BuildAtlas.cpp
BuildAtlas.h Added license info Jun 3, 2013
CMakeLists.txt update namespace conflict of ply file and add headers Nov 18, 2014
COPYING
CPUDetect.cpp amd fix ssse3 May 20, 2011
CPUDetect.h
CheckThreadPool.cpp Added license info Jun 3, 2013
CheckThreadPool.h Added license info Jun 3, 2013
Clean.cpp
Clean.h
Clipper.cpp return the number of faces clipped and output empty files Oct 19, 2012
Clipper.h return the number of faces clipped and output empty files Oct 19, 2012
DataStruct.h big set of changes add ann and code for trinagle texture from bundler… Nov 6, 2013
DeepZoom.cpp Added license info Jun 3, 2013
Depth.cpp Added license info Jun 3, 2013
Depth.h Added license info Jun 3, 2013
Extents.cpp overflow and div zero errors Jun 16, 2013
Extents.h
FindGlew.cmake
FindOSG.cmake switched to cmake Feb 7, 2008
FindPythonModule.cmake python check for osr gdal Sep 12, 2012
GIS_angle.png added gis image Jun 16, 2013
GLImaging.cpp Added license info Jun 3, 2013
GLImaging.h split GLimaging Sep 12, 2012
GraphicsWindowCarbon.cpp This commit was generated by cvs2svn to compensate for changes in r1865, Sep 18, 2007
GraphicsWindowCarbon.h
MSC.cpp
MSC.hpp Added license info Jun 3, 2013
MacroEnsureVersion.cmake
Makefile Make build take place in tmp directory. Sep 27, 2012
MemUtils.cpp Added license info Jun 3, 2013
MemUtils.h 13.04 compile and link fixes Jun 16, 2013
MemoryUsage.h memory usage Mar 10, 2009
OSGExport.cpp Added license info Jun 3, 2013
OSGExport.h Added license info Jun 3, 2013
OctreeBuilder.cpp Added license info Jun 3, 2013
OctreeBuilder.h Added license info Jun 3, 2013
OptimizeForArchitecture.cmake added ivybridge Sep 12, 2012
PLYWriterNodeVisitor.cc 14.04 compile changes Aug 6, 2014
PLYWriterNodeVisitor.h indentation and color write out for OSG node visitor Oct 1, 2012
PixelBufferCarbon.cpp This commit was generated by cvs2svn to compensate for changes in r1865, Sep 18, 2007
PixelBufferCarbon.h This commit was generated by cvs2svn to compensate for changes in r1865, Sep 18, 2007
PosterPrinter.cpp Added license info Jun 3, 2013
PosterPrinter.h
ProgressBar.cpp Added license info Jun 3, 2013
ProgressBar.h Added license info Jun 3, 2013
README.md Added manual_total_ply to replace vrip step with a-priori CAD mesh. F… Jul 22, 2014
RobustMatcher.h now stripped of auv deps need to check perf Jun 5, 2013
SeaBedIO.cpp Added license info Jun 3, 2013
SeaBedIO.h Added license info Jun 3, 2013
Semaphore.cpp Added license info Jun 3, 2013
Semaphore.h
ShellCmd.cpp Added manual_total_ply to replace vrip step with a-priori CAD mesh. F… Jul 22, 2014
ShellCmd.h
SpatialReference The jump from threadedStereo to Structured (ie Pavlopetri) Sep 12, 2012
SpatialReference.cpp The jump from threadedStereo to Structured (ie Pavlopetri) Sep 12, 2012
SplitBounds.cpp fixes bugs from sonar mode Nov 25, 2014
SplitBounds.h Added manual_total_ply to replace vrip step with a-priori CAD mesh. F… Jul 22, 2014
StereoEngine.cpp partial commit of sdense init Apr 25, 2014
StereoEngine.h stereo dense moved over and config file options moved into load mat m… Jun 5, 2013
TODO The jump from threadedStereo to Structured (ie Pavlopetri) Sep 12, 2012
TexPyrAtlas.cpp Added license info Jun 3, 2013
TexPyrAtlas.h 13.04 compile and link fixes Jun 16, 2013
TexturedSource.cpp
TexturedSource.h If desired, bounding box margin can be expanded only in {x,y,z} by pa… Jul 28, 2014
TexturingQuery.cpp added some sonar-related stuff. Sep 17, 2014
TexturingQuery.h added some sonar-related stuff. Sep 17, 2014
TexturingQueryDidson.cpp Fixes for DIDSON images, now 3D mosaicing works Sep 22, 2014
TexturingQueryDidson.h Fixes for DIDSON images, now 3D mosaicing works Sep 22, 2014
TightFit.cpp hole filling option added and zero sized images check Apr 14, 2014
TightFit.h process all thr way through still some bugs Nov 10, 2013
VPBInterface.cpp Added license info Jun 3, 2013
VPBInterface.hpp
ass.frag cleanup val boost Jan 31, 2008
ass.vert broken working ? plane dist Jan 31, 2008
atlasmesh.cpp
blend.frag still broken lighting for atlas Apr 28, 2011
blend.vert added ambient occulsion and color kdtree clipping Apr 20, 2011
blendAtlas.frag
borderClip.cpp Added license info Jun 3, 2013
calcTexCoord.cpp If desired, bounding box margin can be expanded only in {x,y,z} by pa… Jul 28, 2014
calcTexCoord.h Added license info Jun 3, 2013
calcTexCoordBundler.cpp lib ignored incorrectly Dec 5, 2013
calcTexCoordDidson.cpp added some sonar-related stuff. Sep 17, 2014
calibFile.cpp added some sonar-related stuff. Sep 17, 2014
calibFile.h
combineBaseTiles.py combine tiles scripts Apr 14, 2015
configFile.cpp Added license info Jun 3, 2013
configFile.h Added license info Jun 3, 2013
convertVirtualTextureTiles.py vt scripts Apr 4, 2015
cp_meshfld.sh
createSem.cpp Added license info Jun 3, 2013
create_main_feed.sh ipad data update Sep 12, 2012
create_masked.py Added license info Jun 3, 2013
create_mipmap.cpp Added license info Jun 3, 2013
debug.frag The jump from threadedStereo to Structured (ie Pavlopetri) Sep 12, 2012
deepzoom.py clear color support Jan 28, 2013
dicedImage.cpp
drawtext.frag semi working osgexport Jan 29, 2008
dumpPCD.cpp Added license info Jun 3, 2013
engine.cpp Added license info Jun 3, 2013
fply.h update namespace conflict of ply file and add headers Nov 18, 2014
gdal2tablet.sh updated scripts Apr 2, 2015
gdal_rastertotrn.py
gdalcopyproj.py
genOverlappingPoses.cpp Added license info Jun 3, 2013
genRangeImg.cpp Added license info Jun 3, 2013
gen_j2k.py working tiling Apr 26, 2014
generateOctreeFromObj.py added code to autogenerate ipad data MULTI-MODULE COMMIT Sep 12, 2012
generateVirtualTextureTiles.py PIL from Apr 6, 2015
getmeta.sh
hcolor.frag skip for downsampled images Jul 25, 2009
hcolor.vert
im_shrink_noblack.cpp Added license info Jun 3, 2013
im_shrink_noblack.py
imageNode.cpp
imageNode.h Added license info Jun 3, 2013
image_rect.cpp old opencv patch Apr 25, 2014
image_rect.hpp old opencv patch Apr 25, 2014
imath.h new create_mipmap code to use modified gimp dds plugin stuff to do do… Jan 20, 2013
kd.tar.bz2 jp2 proc now scripted Apr 25, 2014
keypoint_mesh.cpp
lerp.frag
lerp.m semi working osgexport Jan 29, 2008
lerp.vert fixed blend code Sep 13, 2010
mcd.cpp broken mem checking Nov 7, 2007
mcd.h broken mem checking Nov 7, 2007
mergeVirtualTextureTiles.py
mesh.cfg Merge branch 'master' of https://github.com/mattjr/structured Apr 25, 2014
mesh_proc.cpp
mesh_proc.hpp stereo dense moved over and config file options moved into load mat m… Jun 5, 2013
mipmap.cpp
mipmap.h more licensing Jun 3, 2013
monoDriver.cpp Added license info Jun 3, 2013
ng2_20070520_subset1.calib This commit was generated by cvs2svn to compensate for changes in r1865, Sep 18, 2007
ng2_stereo.cfg
novelty-live.frag added switch nodes Apr 29, 2009
novelty.frag less correct but simpler plane fit May 7, 2009
novelty.vert less correct but simpler plane fit May 7, 2009
opengl.supp memory leaks seem to be fixed if doesn't segfault not tested extensivly Sep 1, 2008
pass.frag still broken lighting for atlas Apr 28, 2011
plotpt.m
plyfile.c update namespace conflict of ply file and add headers Nov 18, 2014
progressbar.py
proj.frag
proj.vert working overlap texturing at lod0 working on atlasing higher lods May 25, 2010
projV.frag
projV.vert threading enabled seems to work Dec 23, 2010
projVatlas.vert
runtasks.py added cleaner markdown install commands and turn down debug level Jun 17, 2013
runtp.py Added license info Jun 3, 2013
runtp_dist.py
runts.sh cache meshtransforms locally Dec 10, 2007
screencap.cpp semi working reimage usiung faster method Feb 28, 2011
semi-dense.cfg This commit was generated by cvs2svn to compensate for changes in r1865, Sep 18, 2007
singleImageTex.cpp Added license info Jun 3, 2013
sparseJoin.cpp 13.04 compile and link fixes Jun 16, 2013
stereo_cells.cpp Added license info Jun 3, 2013
stereo_cells.hpp Added license info Jun 3, 2013
stereo_dense.cpp new q matrix Apr 25, 2014
stereo_dense.hpp 13.04 compile and link fixes Jun 16, 2013
stereo_mesh_gen.cpp printout updates Jun 7, 2013
svn-commit.tmp The jump from threadedStereo to Structured (ie Pavlopetri) Sep 12, 2012
taskworkers.py Added license info Jun 3, 2013
testImageModel.cpp
threadedStereo.cpp opencv threading issues May 6, 2015
threadpool.py Added license info Jun 3, 2013
threadpool_dist.py Added license info Jun 3, 2013
tlThread.cpp This commit was generated by cvs2svn to compensate for changes in r1865, Sep 18, 2007
tlThread.h
treeBBClip.cpp
typedefs.h
updatemodelxml.sh update jpg Sep 12, 2012
vecmath.py added code to autogenerate ipad data MULTI-MODULE COMMIT Sep 12, 2012
verasansmono.png semi working osgexport Jan 29, 2008
vertCheck.cpp
vertexData.cpp update namespace conflict of ply file and add headers Nov 18, 2014
vertexData.h added calc norm func Dec 13, 2012
vipsVTAtlas.cpp cap threads for vips to prevent internal segfault Nov 25, 2014

README.md

structured

Tools for the Generation and Visualization of Large-scale Three-dimensional Reconstructions from Image Data.

Installing

structured requires:

  • cmake (>=2.8)
  • openscenegraph (>=3.0)
  • OpenCV (>=2.2)
  • OpenGL
  • TCL/TK (8.4)
  • vips
  • gdal (including python bindings and binaries)
  • csh

For installing on Ubuntu the following apt-get install lines should pull in the requirements:

12.04

sudo apt-get install cmake libcv-dev libopenscenegraph-dev gdal-bin

sudo apt-get install libgdal1-dev python-gdal libvips-dev libopencv-dev

sudo apt-get install libhighgui-dev libcvaux-dev libopencv-gpu-dev

sudo apt-get installlibopencv-gpu2.3 tcl8.4-dev tk8.4-dev csh imagemagick

13.04

sudo apt-get install cmake libcv-dev libopencv-dev libopenscenegraph-dev

sudo apt-get install gdal-bin libgdal1-dev python-gdal libvips-dev

sudo apt-get install libhighgui-dev libcvaux-dev tcl8.4-dev

sudo apt-get install tk8.4-dev csh imagemagick

Running

Binary is not installed and must be called using the full path of the source dir.

For example:

/home/$USER/git/structured/threadedStereo .

Once data folder is setup as below.

Data folder file setup

Filename Desc
mesh.cfg Reconstruction config file. A default file is in the structured directory
img A link to the converted images
stereo.calib The stereo calibration file (this should be in the renav directory)
localiser.cfg The SLAM config file (this should be in the renav directory)
stereo_pose_est.data A link to the file stereo_pose_est.data produced by seabed_slam

Viewing the Meshes

The meshes can be viewed using the program benthicQT also avalaible on github.

The AUV Navigation Frame

The AUV estimates its position in a local navigation reference frame (not for example in degrees of latitude/longitude). Positions in the navigation estimate files are measured relative to the origin of the navigation frame.

The orientation of the local navigation frame is defined as follows:

  • Positive X is true (not magnetic) North

  • Positive Y is East

  • Positive Z is down

The origin of the local navigation frame is defined by latitude and longitude coordinates.

stereo_pose_est.data

Each line describes the pose of the stereo-vision system relative to the navigation frame at the time a pair of stereo images were acquired.

The stereo-rig reference frame is defined to be coincident with the left camera..

The orientation of the stereo-rig frame is defined as follows:

  • Positive X is aligned with the X-axis of an image (towards the right of an image)

  • Positive Y is aligned with the Y-axis of an image (towards the bottom of an image)

  • Positive Z is the direction in which the camera is looking

On each line of the file are 13 items:

Position Desc Type
1 Pose identifier integer value
2 Timestamp in seconds
3 X position (North) in meters, relative to nav frame
4 Y position (East) in meters, relative to nav frame
5 Z position (Down) in meters, relative to nav frame
6 X-axis Euler angle (Roll) in radians, relative to nav frame
7 Y-axis Euler angle (Pitch) in radians, relative to nav frame
8 Z-axis Euler angle (Yaw/Heading) in radians, relative to nav frame
9 Left image name local filename
10 Right image name local filename
11 Vehicle altitude in meters
12 Approximate bounding image radius in meters
13 Likely trajectory cross-over point 1 for true, 0 for false

Data items 12 and 13 are used within our 3D mesh building software, and can safely be ignored in other applications.

stereo.calib

  • First Line: number of cameras (1 number)

  • For each camera a line with camera parameters

    • image size (2): [x y]
    • intrinsic parameter matrix (9): [ fx 0 cx; 0 fy cy; 0 0 1 ]
    • distortion coefficients, 2 radial then 2 tangential (4): [k1 k2 p1 p2]
    • rotation matrix (9)
    • translation vector in meters (3)
  • Rectified image quadrangle for camera1 (8)

  • Rectified image quadrangle for camera2 (8)

  • Transformation Coefficients for camera1 (9)

  • Transformation Coefficients for camera2 (9)

Notes: for a stereo config, the number of cameras is two. for stefs calib, the left camera is camera1, the right is camera2



// Number of cameras
2


// Camera 1 - Image size
1360.0000000000 1024.0000000000 

// Camera 1 - Instrinsic parameters [ fx 0 cx; 0 fy cy; 0 0 1 ]
1742.4833984375    0.0000000000     702.2943725586    
0.0000000000       1744.5247802734  505.3277282715    
0.0000000000       0.0000000000     1.0000000000    

// Camera 1 - Distortion parameters [ k1 k2 p1 p2 ] 
0.1640239209    0.6399505734    0.0019153117    0.0084038246   

// Camera 1 - Rotation matrix
-0.9471259117   -0.3171964586   -0.0483626388   
-0.3193612993    0.9173640609    0.2375952154  
-0.0309982132    0.2404777408   -0.9701595306    
 
// Camera 1 - Translation vector 
0.1716919392   -0.0959866196    1.1181910038 






// Camera 2 - Image Size
1360.0000000000 1024.0000000000 

// Camera 2 - Instrinsic parameters [ fx 0 cx; 0 fy cy; 0 0 1 ] 
1739.3259277344    0.0000000000  678.7371215820    
0.0000000000 1740.5989990234  524.8011474609    
0.0000000000    0.0000000000    1.0000000000    

// Camera 2 - Distortion parameters [ k1 k2 p1 p2 ] 
0.1479872018    0.7572427392   -0.0015344607   -0.0037570531  

// Camera 2 - Rotation matrix
-0.9497462511   -0.3011870682   -0.0852552578   
-0.3128538132    0.9222519994    0.2270985991   
 0.0102276700    0.2423584610   -0.9701328874   
  
// Camera 2 - Translation vector 
0.1418329179   -0.0863670856    1.1126987934 





// Camera 1 - coordinates of destination quadrangle after epipolar geometry rectification
-63.7492790222  -17.5878181458 1363.6937255859    0.0580182485 1360.0000000000  983.3759155273  -63.3137931824  990.2045288086 

// Camera 2 - coordinates of destination quadrangle after epipolar geometry rectification
-3.4339661598   35.0206069946 1416.6680908203   25.5735092163 1423.8615722656 1022.9803466797    0.0000000000 1023.4026489258 





// Camera 1 - Transformation coefficients
1.0745683575    0.0002419059  -63.7492790280    0.0129759424    0.9870401045  -17.5878181530    0.0000183164    0.0000028963    1.0000000000 

// Camera 2 - Transformation coefficients
1.0347662652    0.0033534826   -3.4339661738   -0.0071165600    0.9626237039   35.0206069774   -0.0000066539   -0.0000025338    1.0000000000 

Converting the Stereo-rig Z-axis Euler Angle to an Image Rotation for GIS Programs

The stereo-rig Z-axis Euler angle (shown as φ in the figure below) is the angle between the X-axis of the navigation frame (north) and the X-axis of the camera or image. The image rotation angle (shown as α) is the angle between East and the camera/image X-axis.

The image rotation angle can therefore be obtained from the Z-axis Euler angle by subtracting 90 degrees:

α = φ - 90°

In the figure below, the stereo-rig's Z-axis Euler angle is 135°, and the image rotation angle is 45°.

Gis Angle

Using Manually-aligned CAD model as mesh

  1. Align the CAD model to a the same frame as the stereo_pose_est file.
  2. Ensure that the face normals are pointed towards the exterior of the surface.
  3. Remesh the surface to have ~7cm vertex resolution. For instance, use the midpoint resampling method in meshlab.
  4. Pass the '--manual_total_ply ' option to threadedStereo to use this instead of vrip-ing the individual stereo-derived surfaces