Project

LiDAR & 3D Visualization Practice 🏢 🏞️ 🛰️

📖 Task description

Task and problem

Work was inspired by various rayshader projects in particular, and trends in 3D geodataviz in general, where rayshader stays for raytracing + terrain shading/hillshading (which cover both 'smart' light rendering and 'smart' terrain-relief shading above known topography). Usually, people in DataViz community have been using either over-specialized tools (f.e. blender, aeroid, 3DMax, plugins for the family of professional *GIS software) or R programming language ecosystem (f.e. rayshader, rayvista, R-lang APIs) for these purposes. This project is trying to accelerate a more-python-based pipeline.

Target goal of the project was to catalyze OSM datasets (polygons and their feature-attributes) acquired over the Downtowns, Parks, Parkings project with 3D rendered plots (meshes above Digital Surface Models) to color selected assets across dozens of urban areas in a fancy way in an automated manner. However, I've met some restrictions still working under trying to match datasets (LiDARs with OSM; different LiDARs to desired logics) and scale the pipeline (many different datasets sources; computational complex scripts). Might note, the major restriction was caused by the limited availability of high-quality (~1m/<3ft) and time-actual (>2019) point cloud open datasets. The minor restrictions were caused by relative complexity of high-level packages themselves.

Secondary sources

• "Penn MUSA Masterclass: 3D Mapping and Visualization with R and Rayshader" by Tyler Morgan-Wall (@tylermorganwall)
• "Python Libraries for Mesh, Point Cloud, and Data Visualization (Part 1)" by Ivan Nikolov
• "Atlas of oblique maps: A collection of landform portrayals of selected areas of the world" by USGS, 1988
• USGS viewers [f.e. USGS / Entwine - (Potree) (Cesium)]
• Maxar [Precision3D Data Suite | The Globe in 3D] and Airbus [Airbus Constellation]
  • Satellite surveillance may be less of a privacy concern than you think - for now
• Additional sources - click to expand

  • "#30DayMapChallenge" project
  • "Topography - 3D Rendered" by longitude.one
  • "Shaded maps (3D, LiDAR, Terrain Shading)" by Visual Wall Maps
  • "3D printed maps / tactile maps" project and "Touch-Mapper" project
  • "Making Mountains - Manufacturing: 3D Printing + Bronze Casting" by Greg Kamradt
  • Various Etsy stores with 3D printed/tactile/wooden maps (i.e. "Colberd", "3DCityframes" "WoodShoreWater")
  • Various digital agencies or shops with full 3D models of cities/game assets (i.e. "TurboCG 3d models")
• List of secondary sources from "Downtowns, Parks, Parkings" project

Architecture and report

Pipeline scheme - click to view example

Data flow (logic)

Preprocessing logic

Current raw report available, see [html] | [ipynb]

📊 Selected charts

Selected charts are provided. See below:

Charts - click to expand

isometric	orthographic
↑ Miami Beach, FL. Data comes from Interactive Hurricane Impact Assessment LiDAR Project, 2007 - IHRC at FIU (also accessible via State of Florida Division of Emergency Management web cite - LiDAR and Digital Elevation Data). 1ft aerial LiDAR. Tiled Lidar point cloud. [Dataset Link] | [Dataset Link alternative] | [Satellite View]
isometric	orthographic
↑ Dallas, TX. Data comes from USGS 2019 (Texas Pecos Dallas). Retrived via Texas Natural Resources Information System. 70cm aerial LiDAR. Tiled Compressed Lidar Point Cloud. [Dataset Link] | [Satellite View]
isometric	orthographic
↑ Bishkek city, Kyrgyzstan. Data comes from Pleiades satellites tristereo optical imagery, 2013 (retrived via OpenTopography). 0.5m satellite remote sensing photogrammetry point cloud. [Dataset Link] | [Satellite View]

📁 Structure of repository

Project structure:
+--data                       <- folder for datasets
¦  L--raw                       <- ... 1. raw data
¦  L--interim                   <- ... 2. auxiliary, generated, temporary, preprocessed data
¦  L--processed                 <- ... 3. final, ready-to-analysis data
¦  L--external                  <- ... +. additional datasets
¦  
+--notebooks                  <- folder for *.ipynb files
¦  L--*.ipynb 1                 <- ... file 1
¦
+--src                        <- folder for .py scripts
¦  L--*.py 1                    <- ... file 1
¦  L--*.py 2                    <- ... file 2
¦  L--*.py 3                    <- ... file 3
¦
+--figures                    <- folder for charts and images to reports
¦  L--external                  <- ... 1. external images
¦  L--internal                  <- ... 2. internal images
¦
+--reports                    <- folder for reports (i.e. *.pptx, *.html, *.ipynb)
¦
+--docs                       <- folder for documentation files
¦
+--README.md                  <- the top-level README for developers using this project

Cookiecutter Data Science

🧰 Installation

1. Check the requirements.txt
2. Check wget and unzip (or OS analogues)
3. Check backend for whitebox tools (whitebox geo)

a. https://www.whiteboxgeo.com/manual/wbt_book/install.html
b. https://giswqs.github.io/whitebox-frontends/#python

4. Check backend for pyvista (VTK)

a. https://docs.pyvista.org/getting-started/installation.html

5. If needed, check backend for rasterio (GDAL)

https://rasterio.readthedocs.io/en/latest/installation.html

📌 Links

Placeholder

🐉 License and legals

Ask before use.