mesh2audio

This project continues to be developed here.

Generate axisymmetric 3D models, or import existing 3D models, and transform them into real-time playable physical audio models!

Supports fast DSP generation of physical audio models that sound decently realistic, as well as blazing fast 2D axisymmetric model generation, at the expense of some fidelity.

The generated audio model can be played in real-time by "striking" (clicking) on mesh vertices in the 3D mesh viewer, or by using an audio input device (such as a microphone) to excite the vertices.

This is the final project for Karl Hiner and Ben Wilfong for CSE-6730 Modeling & Simulation, Spring 2023.

Project video: https://youtu.be/RwxgOHVBDvc

Example audio output is in the sound_samples directory.

Custom 2D axisymmetric FEM model designed and implemented by Ben Wilfong. (Ben implemented everything under the fem directory.)

The rest of the code is basically a GUI wrapper around Ben's axysimmetric FEM, and the O.G., mesh2faust by Romain Michon, Sara R Martin, and Julius O Smith, with some performance improvements like using tetgen instead of VegaFEM for converting triangular meshes to tetrahedral, and using Eigen/Spectra instead of MKL/Pardiso to find the eigenvalues of the mass/stiffness matrices, for compatibility with non-Intel processors such as ARM.

Build app

Install dependencies

Mac

$ brew install glew llvm eigen
$ brew link llvm --force

Linux

(Only tested on Ubuntu.)

$ sudo apt install llvm libc++-dev libc++abi-dev libeigen3-dev
$ ln -s llvm-config-15 llvm-config
$ export PATH="$(llvm-config --bindir):$PATH"

Install GTK (for native file dialogs), and OpenGL dependencies:

$ sudo apt install build-essential libgtk-3-dev libglu1-mesa-dev freeglut3-dev mesa-common-dev libglew-dev

Clone, clean and build app

$ git clone --recurse-submodules git@github.com:GATech-CSE-6730-Spring-2023-Project/mesh2audio.git
$ cd mesh2audio/app

• Clean:
  • Clean up everything: ./script/Clean
  • Clean debug build only: ./script/Clean -d [--debug]
  • Clean release build only: ./script/Clean -r [--release]
• Build:
  • Debug build (default): ./script/Build
  • Release build: ./script/Build -r [--release]

Debug build is generated in the ./app/build directory relative to project (repo) root. Release build is generated in ./app/build-release.

On Linux, you will also need to build the fem executable (since the one that's committed to this repo was build on Mac):

$ sudo apt install gfortran
$ ./script/Build

To run the freshly built application:

# The application assumes it's being run from the build directory when locating its resource files.
$ cd app/build # or build-release
$ ./mesh2audio

Stack

• App: UI/UX/mesh/audio generation & editing
  • ImGui + SDL3: Immediate-mode UI/UX, supporting many environments & backends.
  • Faust: Render the mesh to an audio graph, with real-time interactive vertex excitation.
  • miniaudio: Continuously render the modal physical model of the input 3D volumetric mesh to audio.
  • tetgen: Convert triangular 3D meshes into tetrahedral meshes.
  • CDT: Constrained Delaunay Triangulation, the default 2D polygon triangulation method
  • earcut: Simpler 2D polygon triangulation, provided as an option.
  • glm: Frontend maths.
  • nativefiledialog-extended: Native file dialogs.
  • nanosvg: Read path vertices from SVG files.
  • ImPlot: Plotting
  • ImGuizmo: Mesh transform and camera rotation gizmos.
  • ImSpinner: Wicked cool loading spinners for ImGui.
• FEM:
  • 2D FEM: Custom Fortran implementation by Ben Wilfong
  • 3D FEM: VegaFEM