Skip to content

Commit

Permalink
docs: add build3d a generally better debug notes
Browse files Browse the repository at this point in the history
  • Loading branch information
mwfarb committed May 8, 2024
1 parent 496a064 commit 84390b9
Show file tree
Hide file tree
Showing 4 changed files with 89 additions and 18 deletions.
5 changes: 3 additions & 2 deletions content/examples.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ Thumbnail images can be clicked for a larger view. Multiple links are provided t
| Box Collision | [![][i-box-collision]][i-box-collision] | Listen for bounding-box collisions with user camera and hands (`box-collision-listener`) | [Attribute:][obja] [Schema][s-box-collision] [Python][p-box-collision] |
| Buffer Geometry | [![][i-buffer]][i-buffer] | Reduce geometry memory usage while being harder to manipulate (`buffer`) | [Attribute:][obja] [Schema][s-buffer] |
| Build | [![][i-build]][i-build] | Build is a JSON editor for the persisted scene graph. | [Application][a-build] [Tutorial][t-build] |
| Build 3D | [![][i-build3d]][i-build3d] | Build3D is a visual 3D editor for the persisted scene graph (based on Inspector) | Application Tutorial forthcoming |
| Build 3D | [![][i-build3d]][i-build3d] | Build3D is a visual 3D editor for the persisted scene graph (based on Inspector) | Application [Tutorial][t-build3d] |
| Callbacks | | A handler in a program to receive an ARENA Event. | Concept [Tutorial][t-callbacks] |
| Camera | [![][i-camera]][i-camera] | Camera is the pose and component data representing a user avatar | [Object:][obj3] [Schema][s-camera] [Message][m-camera] [Python][p-camera] |
| Capsule | [![][i-capsule]][i-capsule] | Capsule Geometry | [Object:][obj3] [Schema][s-capsule] [Python][p-capsule] |
Expand Down Expand Up @@ -643,14 +643,15 @@ These effects are enabled in desktop and XR views. For reference, a list of [all
[t-audio]: /content/interface/user-presence#user-audio
[t-box]: /content/overview/dev-guide#create-a-box-and-observe
[t-build]: /content/overview/build
[t-build3d]: /content/interface/debug#build-3d
[t-callbacks]: /content/overview/dev-guide#clients-and-scene-callbacks
[t-chat]: /content/overview/user-guide#chat-find-people-and-places
[t-displayname]: /content/interface/user-presence#user-display-name
[t-face]: /content/interface/user-presence#facial-recognition-avatar
[t-filestore]: /content/interface/filestore
[t-gltf-lod]: /content/3d-content/gltf-files#gltf-model-lod-level-of-detail
[t-gltf]: /content/overview/build#add-new-objects
[t-inspector]: /content/overview/debug#debug-your-scene-with-a-frame-scene-inspector
[t-inspector]: /content/interface/debug#a-frame-scene-inspector
[t-landmark]: /content/overview/build#add-landmarks
[t-landmark]: /content/overview/user-guide#chat-find-people-and-places
[t-localization]: /content/overview/localization
Expand Down
46 changes: 46 additions & 0 deletions content/interface/debug.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
---
title: Debug A-Frame
nav_order: 9
layout: tutorial
parent: Web Interface
---

# Debug A-Frame

Our ARENA deployment of A-Frame includes the A-Frame Inspector, which is a **local-only** scene editor, and useful to locate, observe, and test objects in ARENA scenes. We have added a wrapper application around A-Frame Inspector called Build 3D, which adds the ability to become a **remote distributed** scene editor.

Examine the list of elements on the left side. Each element or object you select will show it's details and attributes on the right side. You may edit any attributes here you wish, however, remember that the A-Frame Scene Inspector will not persist any changes to the persistence database. We do have a way to visually manipulate objects and save changes that we will share next.

## Build 3D

Build 3D is for editing persisted scene objects and configuration in 3D just as they would be on the 3D browser view. Build 3D leverages the A-Frame Inspector, with some critical differences, namely that changes are shared to other users.

Usage:

1. Enter on your example scene by clicking: expand settings (`v`), then the `3D Editor` link.
1. Optionally, you can enter from the JSON Build Page, clicking on the `Edit 3D` button to the right of any object in your list of scene objects.
1. Be sure to "Play" the scene to allow your ability to publish MQTT changes by clicking the Play (``) button.
1. From this point, any changes you make, including add/remove objects/components, should be reflected in the bottom logging panel of MQTT publish events.
1. If you want to edit JSON from this view, an easy way back to JSON Build is to select the object, open the `build3d-mqtt-object` component, and toggle on `openJsonEditor`.
1. Exit by clicking the `Back to Scene` button. **Note: This will reload a new scene page.**

<img src="/assets/img/examples/build3d.png" width="50%" />

## A-Frame Scene Inspector

Since the ARENA's rendering uses the A-Frame web 3D rendering engine, you can open the [A-Frame Scene Inspector](https://aframe.io/docs/1.5.0/introduction/visual-inspector-and-dev-tools.html) on any scene to examine and manipulate any of the A-Frame elements in your scene.

Usage:

1. Enter on your example scene by typing `<ctrl> + <alt> + i` on most systems.
1. Exit by clicking the `Back to Scene` button.

<img src="/assets/img/overview/inspector.png" width="50%" />

## Help Commands

{% include alert type="tip" content="
While in the Build 3D or A-Frame Inspector views, press the `H` key to pull up a list of super-useful A-Frame Inspector commands.
"%}

<img src="/assets/img/overview/inspector-help.png" width="50%" />
54 changes: 39 additions & 15 deletions content/overview/debug.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,31 +5,55 @@ layout: tutorial
parent: Tutorial
---

# Debugging ARENA Web Browser Visually

# Python Console Interface
When running programs in Python or Unity, you have a plethora of debug tools at your disposal to debug lines of code. For debugging the visual elements of rendered scenes, here are some other tools to help, and some guidance of where they are useful. Each tool has an expansive or limited capacity to display object messages (**Read Objects**), and add/change/delete object messages (**Write Objects**), as noted

There is a [console interface](/content/programs/debug) to the `arena-py` library. This is designed to have a way to inspect the program from the console, without having to send a heartbeat or write your own command/response interface.
You can also interact with it from [`/programs`](https://arenaxr.org/programs):
| Visual Object Editors | Format | Read Objects | Write Objects |
| ----------------------------------------------------- | ------ | -------------------------------------------------- | -------------------------------------------------- |
| [Python Console Interface](#python-console-interface) | MQTT | <span style="color:green">All</span> | <span style="color:red">None</span> |
| [Build JSON](#build-json-page) | JSON | <span style="color:green">All</span> | <span style="color:green">All</span> |
| [Build 3D](#build-3d) | 3D | <span style="color:green">All</span> | <span style="color:green">All</span> |
| [A-Frame Inspector](#a-frame-inspector) | 3D | <span style="color:green">All</span> | <span style="color:red">None</span> |
| [Unity Editor](#unity-editor) | 3D | <span style="color:orange">Most</span><sup>1</sup> | <span style="color:green">All</span> |
| [WebXR API Emulator](#webxr-api-emulator) | 3D | <span style="color:orange">Camera/Hands</span> | <span style="color:orange">Camera/Hands</span> |
| [AR Builder](#ar-builder) | 3D | <span style="color:green">All</span> | <span style="color:orange">Primitives/GLTFs</span> |

![](/assets/img/programs/cli-interpreter.png)
<sup>1</sup> The `arena-unity` library is still in development. Objects currently rendered are all Primitives, GLTFs, Lights, Lines, Text, Images. Others [still to be rendered](/content/schemas/render-support#rendering-support) are: PCD, Threejs, URDF, UI Panels, Gaussian Splat.<br/>

# Debugging ARENA Web Browser Visually
## Python Console Interface

There is a [console interface tutorial](/content/programs/debug) to the `arena-py` Python library. This is designed to have a way to inspect the program from the console, without having to send a heartbeat or write your own command/response interface.

<img src="/assets/img/programs/cli-interpreter.png" width="50%" />

## Build (JSON) Page

When running programs in Python or Unity, you have a plethora of debug tools at your disposal to debug lines of code. For debugging the visual elements of the Browser view, here are some other tools to help.
There is a [JSON scene build tutorial](/content/overview/build) for ARENA scenes. This is for editing persisted scene objects and configuration in JSON format with full support of all objects and their components.

## Debug your scene with A-Frame Scene Inspector
<img src="/assets/img/overview/builder.png" width="50%" />

Since the ARENA's rendering uses the A-Frame web 3D rendering engine, you can open the [A-Frame Scene Inspector](https://aframe.io/docs/1.5.0/introduction/visual-inspector-and-dev-tools.html) on any scene to examine and manipulate any of the A-Frame elements in your scene. Try this now from your example scene by typing `<ctrl> + <alt> + i` on most systems.
## Build 3D

Examine the list of elements on the left side. Each element or object you select will show it's details and attributes on the right side. You may edit any attributes here you wish, however, remember that the A-Frame Scene Inspector will not persist any changes to the persistence database. We do have a way to visually manipulate objects and save changes that we will share next.
There is a [Build 3D tutorial](/content/interface/debug#build-3d) for ARENA scenes using Build 3D. This is for editing persisted scene objects and configuration in 3D just as they would be on the 3D browser view. Build 3D leverages the A-Frame Inspector, with some critical differences, namely that changes are shared to other users.

![A-Frame Inspector](/assets/img/overview/inspector.png)
<img src="/assets/img/examples/build3d.png" width="50%" />

{% include alert type="tip" content="
While in the A-Frame Inspector view, press the `H` key to pull up a list of super-useful A-Frame Inspector commands.
## A-Frame Inspector

There is a [A-Frame Inspector tutorial](/content/interface/debug#a-frame-scene-inspector) for ARENA scenes using A-Frame Inspector. **NOTE: The default A-Frame Inspector does not publish MQTT updates. Any changes you make are local only.**

{% include alert type="warning" content="
A-Frame Inspector and Build 3D are similar, but operate critically differently. Learn to tell them apart: The A-Frame Inspector has **solid black** control panels, and Build 3D has **opaque black** control panels.
"%}

![A-Frame Inspector Help](/assets/img/overview/inspector-help.png)
<img src="/assets/img/overview/inspector.png" width="50%" />

## Unity Editor

There is a [Unity 3D scene editing tutorial](/content/unity/editor) for ARENA scenes. This is for editing persisted scene objects and configuration in 3D just as they would be on the 3D browser view. **NOTE: See links above for which .**

<img src="/assets/img/unity/unity-desktop.png" width="50%" />

## WebXR API Emulator

Expand All @@ -39,9 +63,9 @@ Since ARENA's web view runs on WebXR, we can use the WebXR API Emulator to test
1. Open your ARENA scene web view and the ARENA detects that you have a XR device (emulated) and it will let you enter the immersive (VR、AR) mode.
1. Open the `WebXR` tab in the browser developer tool ([Firefox](https://developer.mozilla.org/en-US/docs/Tools), [Chrome](https://developers.google.com/web/tools/chrome-devtools/)) to control the emulated devices. You can move the headset and controllers, and trigger the controller buttons.

![WebXR Browser Tab](/assets/img/overview/webxr-vr-emulator.png)
<img src="/assets/img/overview/webxr-vr-emulator.png" width="50%" />

## AR Builder, visual content authoring
## AR Builder

We also have a Python program, [AR Builder (ARB)](/content/tools/authoring), which you can use to create and edit objects for your scene. You can use it in VR (virtual reality) as a way to edit your scene and save changes to the persistence database. Importantly, you can use it in AR (augmented reality) in combination with supported browsers and localization techniques to anchor scene objects in physical space. See our [section on miXed Reality (XR)](/content/xr) for details.

Expand Down
2 changes: 1 addition & 1 deletion content/programs/debug.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,4 +74,4 @@ Exiting...

You can also interact with it from [`/programs`](https://arenaxr.org/programs):

![](/assets/img/programs/cli-interpreter.png)
<img src="/assets/img/programs/cli-interpreter.png" width="50%" />

0 comments on commit 84390b9

Please sign in to comment.