glTF to USDZ

Proof of concept of converting glTF to USDZ for AR Quick Look (iOS 12+).

Reasoning

Even though I think the intensions of Apple / Pixar are great with the open source USD pipeline I think we as an industry should be relying more on truly open formats that are not controlled by a single entity. Installing USD is cumbersome, requires a lot of disk space and is completely overkill for most situations (if your goal is to convert some 3D models to USDZ and show them using AR Quick Look). I'm hoping that as we get closer to the public release of iOS 12 more tools will pop up that directly convert to USDZ so that we can get cross-platform support.

In order to move away from using the USD pipeline solution offered by Pixar I think it would be wise to try and manipulate the intermediary readeable USDA format. Unfortunately there are very little examples available of USDA files.

My idea is to dynamically generate / manipulate the intermediary a general USDA file-structure and pass that to the usdz-converter to handle the further conversion to USDZ.

After some work, debugging and friendly help from @domenicopanacea and @virakri I got it to work. Most of the findings on OBJ support come from trayser who posted details regarding OBJ to USDZ conversion on developers.apple.com. The OBJ files that are generated by gltf-to-usdz only use the supported tags as mentioned by trayser.

Please note that this is just an experimental setup and should be seen as a proof of concept.

Live demo

Live demo

Limitations

• .glb files are not yet accepted. Please use glTF files with seperate textures.

• Only the first mesh in the glTF file will be parsed and exported. Make sure you merge all meshes.

• Only glTF files with external textures are currently handled (as opposed to embedded base64 encoded textures).

• Animations from the glTF are not yet transferred, it seems possible to transfer skeleton animation and regular transformations. Unfortunately due to she shadow being baked on the first frame of the animation you can't really do large moving animations. Besides that the filesize increases quite dramatically. It is not recommended to have animations that move a character away from the origin as it results in tracking problems. Your best bet are micro-animations that enhance a character or model instead of large animations.

To do

• Convert the example USDZ examples to USDA structures by converting USDC to USDA. Unfortunately I think this requires the installation of the USD pipeline and the use of usdcat.

  Thanks to @wave-electron the USDC's of the examples of the AR Quick Look gallery have been converted to USDA. Due to copyright concerns I unfortunately won't be able to upload them to this repo. I've had a look at them and for the most part look very similar compared to the USDA file generated by gltf-to-usdz.

Installation

• Make sure you have Node.js installed

• Upgrade your operating system to macOS High Sierra 10.13.4 or newer

• Download Xcode 10 beta and put it in /Applications/

https://developer.apple.com/download/

• Link to the beta version instead of the normal version

sudo xcode-select --switch /Applications/Xcode-beta.app

• Construct a new .usda using gltf-to-usdz and run it through the usdz_converter

node ./bin/gltf-to-usdz.js -i ./assets/DamagedHelmet/DamagedHelmet.gltf -o ./assets/DamagedHelmet.usda -s 10.0 && xcrun usdz_converter ./assets/DamagedHelmet.usda ./assets/DamagedHelmet-`date +"%H-%M-%S"`.usdz

• On succes the following should be outputted to the console

2018-06-20 17:21:23.364 usdz_converter[82749:13335731]


Converting asset file 'DamagedHelmet.usda' ...

In order to see the contents of the outputted USDZ change the extension to .zip and unzip it. You will see a .usdc and several textures (if the original glTF file had textures).

Development

In order to install USD on MacOS please follow the following instructions:

• Upgrade your operating system to macOS High Sierra 10.13.4 or newer

• Download Xcode 10 beta and put it in /Applications/

https://developer.apple.com/download/

• Link to the beta version instead of the normal version

sudo xcode-select --switch /Applications/Xcode-beta.app

• Install Cmake and QT. If you get any issues during compiling or installing you might have to install QT5.

brew install cmake
brew install qt@4

• Install PyOpenGL

pip install PyOpenGL

• Install PySide2

pip install --index-url=http://download.qt.io/snapshots/ci/pyside/5.9/latest/ pyside2 --trusted-host download.qt.io

• Update OpenImageIO release version from Release-1.7.14.zip to Release-1.8.12.zip in build_scripts/build_usd.py.

• Run python USD/build_scripts/build_usd.py BUILD, it will take roughly 1 hour, resulting in the following output if succesfully installed:

➜  pixar python USD/build_scripts/build_usd.py BUILD

Building with settings:
  USD source directory          /Users/timvanscherpenzeel/Projects/pixar/USD
  USD install directory         /Users/timvanscherpenzeel/Projects/pixar/BUILD
  3rd-party source directory    /Users/timvanscherpenzeel/Projects/pixar/BUILD/src
  3rd-party install directory   /Users/timvanscherpenzeel/Projects/pixar/BUILD
  Build directory               /Users/timvanscherpenzeel/Projects/pixar/BUILD/build
  CMake generator               Default
  Downloader                    curl

  Building                      Shared libraries
    Imaging                     On
      Ptex support:             Off
    UsdImaging                  On
    Python support              On
    Documentation               Off
    Tests                       Off
    Alembic Plugin              Off
      HDF5 support:             Off
    Maya Plugin                 Off
    Katana Plugin               Off
    Houdini Plugin              Off

    Dependencies                zlib, boost, TBB, JPEG, TIFF, PNG, OpenEXR, GLEW, OpenImageIO, OpenSubdiv

STATUS: Installing zlib...
STATUS: Installing boost...
STATUS: Installing TBB...
STATUS: Installing JPEG...
STATUS: Installing TIFF...
STATUS: Installing PNG...
STATUS: Installing OpenEXR...
STATUS: Installing GLEW...
STATUS: Installing OpenImageIO...
STATUS: Installing OpenSubdiv...
STATUS: Installing USD...

Success! To use USD, please ensure that you have:

    The following in your PYTHONPATH environment variable:
    /Users/timvanscherpenzeel/Projects/pixar/BUILD/lib/python

    The following in your PATH environment variable:
    /Users/timvanscherpenzeel/Projects/pixar/BUILD/bin

Unfortunately I have the following error when running usdcat:

➜  cupandsaucer usdcat CupAndSaucer.usdc -o CupAndSaucer.usda

------------------------ 'Python' is dying ------------------------
Python crashed. FATAL ERROR: Failed axiom: ' Py_IsInitialized() '
in operator() at line 148 of /Users/timvanscherpenzeel/Projects/pixar/USD/pxr/base/lib/tf/pyTracing.cpp

The stack can be found in Tims-MacBook-Pro.local:/var/folders/s9/rnmbn61120s2hwww26_gk3k40000gn/T//st_Python.53393
done.
------------------------------------------------------------------
[1]    53393 abort      usdcat CupAndSaucer.usdc -o CupAndSaucer.usda

I've reported the issue here #19. @wave-electron has been able to correctly compile and run USD on a very similar setup. My issues likely arise from having multiple Python versions installed.

Resources

• https://graphics.pixar.com/usd/docs/Converting-Between-Layer-Formats.html

• https://forums.developer.apple.com/thread/104042

• https://graphics.pixar.com/usd/docs/UsdPreviewSurface-Proposal.html

• https://developer.apple.com/videos/play/wwdc2018/603/

• https://devstreaming-cdn.apple.com/videos/wwdc/2018/603augiuv41xoowslk8/603/603_integrating_apps_and_content_with_ar_quick_look.pdf?dl=1

Contributing

As it is my goal to create a community project I welcome you to submit any pull requests or open a ticket if you spot an issue. Please be aware that as of now the project is in a very experimental state and only serves as a proof of concept so there are probably lots of things wrong and the code is far from clean.

Licence

gltf-to-usdz is released under the MIT licence.