How to put an object in the air?

[YangShengLiQQ]

It seems HitResult only gives us intersection with a surface (plane) or a point cloud. How can I get a point in the middle of air with my click, and thus put an object floating in the air?

[eugenelozada]

Create an anchor from a session and create a world coordinate. For example, the below will spawn a renderable a meter away from the camera. It's debatable, but I think for phone AR, it's much more intuitive to use the camera as a ' gaze pointer' rather than transforming taps into world space.

                val anchor = arFragment.arSceneView.session.createAnchor(camera.displayOrientedPose)
                val anchorNode = AnchorNode(anchor)
                anchorNode.setParent(arFragment.arSceneView.scene)

                val node = Node()
                node.setParent(anchorNode)
                node.localPosition = Vector3(0f,0f,-1f)
                node.renderable = arCardRenderable

[YangShengLiQQ]

Cannot resolve symbol 'camera'
Could you teach me how to solve the problem?

[claywilkinson]

@YangShengLiQQ - Can you describe what you are trying to achieve? There are a couple things to consider in your approach. First, since a tap on the screen is in 2D, the Z coordinate of your point needs to be determined. A HitResult contains the point where the ray from the tap meets a 3D object.

The second thing to consider is placing an anchor that is not associated with a Trackable object is usually not a good experience. The trackable object (planes, augmented images, oriented points) are update by ARCore to represent the connections between the real world image and the augmented, virtual images. If you place an anchor in the "air", it will drift and move relative to the real world.

[YangShengLiQQ]

@claywilkinson - Could you tell me how to achieve it with the first one?

[eugenelozada]

@claywilkinson I'm doing something similar right now, but placing AR based 'cards' to represent POIs in an area that's not attached to any particular plane, but rather created from the session and placed in an arbitrary world space coordinate like what's discussed here:

https://developers.google.com/ar/develop/developer-guides/anchors#anchor_one_or_more_objects

The tracking is actually pretty good, even if i leave a particular room where I place the objects, go around the office, go into different rooms and go back. There is drift, maybe .5 to 2 meters sometimes, but still, pretty impressive tracking considering it's not attached to any particular physical feature point. So kudos to the arcore/sceneform team for achieving that.

[niusounds]

I created a demo for putting Fukidashi in the air.
https://github.com/niusounds/AR-Fukidashi

[mistrydarshan99]

I have used transformable node for load 3D object in the air. But application gonna to be crashed while I have move node and zoom in and zoom out node. Below is code that I have used to load a 3D model in the air.

 ArSceneView arSceneView = arFragment.getArSceneView();
 Scene scene = arSceneView.getScene();

 Vector3 forward = scene.getCamera().getForward();
 Vector3 worldPosition = scene.getCamera().getWorldPosition();
 Vector3 positon = Vector3.add(forward, worldPosition);

 Vector3 direction = Vector3.subtract(worldPosition, positon);
 direction.y = positon.y;

 TransformableNode node = new TransformableNode(arFragment.getTransformationSystem());
 node.setWorldPosition(positon);
 node.setLookDirection(direction);
 node.setParent(scene);

 Node andy = new Node();
 andy.setParent(node);
 andy.setRenderable(andyRenderable);

[dsternfeld7]

Can you share with us what the crash is? TransformableNode is built for manipulating a node with gesture controls that sits on an ArCore plane. For objects that are floating in mid-air, I suggest just using a regular Node and adding your own UX controls for manipulating it.

Also, I suggest creating an AnchorNode that you add the node as a child of of. The AnchorNode uses an ARCore anchor to make sure that the position is tracked correctly in space.

[dsternfeld7]

Closing due to inactivity.

[mistrydarshan99]

@dsternfeld7 I have used below code. Using below code 3D model is render fine in the air but while I have to zoom in or zoom out 3D model or move 3D model over the air then application crash.

/*
 * Copyright 2018 Google LLC. All Rights Reserved.
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *      http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
package com.google.ar.sceneform.samples.hellosceneform;

import android.app.Activity;
import android.app.ActivityManager;
import android.content.Context;
import android.os.Build;
import android.os.Build.VERSION_CODES;
import android.os.Bundle;
import android.support.v7.app.AppCompatActivity;
import android.util.Log;
import android.view.Gravity;
import android.widget.Button;
import android.widget.Toast;
import com.google.ar.core.Anchor;
import com.google.ar.core.Pose;
import com.google.ar.core.Session;
import com.google.ar.core.TrackingState;
import com.google.ar.sceneform.AnchorNode;
import com.google.ar.sceneform.ArSceneView;
import com.google.ar.sceneform.FrameTime;
import com.google.ar.sceneform.Node;
import com.google.ar.sceneform.Scene;
import com.google.ar.sceneform.math.Vector3;
import com.google.ar.sceneform.rendering.ModelRenderable;
import com.google.ar.sceneform.ux.ArFragment;
import com.google.ar.sceneform.ux.TransformableNode;

/**
 * This is an example activity that uses the Sceneform UX package to make common AR tasks easier.
 */
public class HelloSceneformActivity extends AppCompatActivity {
  private static final String TAG = HelloSceneformActivity.class.getSimpleName();
  private static final double MIN_OPENGL_VERSION = 3.1;

  private ArFragment arFragment;
  private ModelRenderable andyRenderable;
  private Button btnLine;
  private AnchorNode anchorNode;
  private AnchorNode mAnchorNode;

  @Override @SuppressWarnings({ "AndroidApiChecker", "FutureReturnValueIgnored" })
  // CompletableFuture requires api level 24
  // FutureReturnValueIgnored is not valid
  protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);

    if (!checkIsSupportedDeviceOrFinish(this)) {
      return;
    }

    setContentView(R.layout.activity_ux);
    btnLine = findViewById(R.id.btnLine);
    btnLine.setOnClickListener(v -> {
      onUpdateRender();
    });

    arFragment = (ArFragment) getSupportFragmentManager().findFragmentById(R.id.ux_fragment);

    // When you build a Renderable, Sceneform loads its resources in the background while returning
    // a CompletableFuture. Call thenAccept(), handle(), or check isDone() before calling get().
    ModelRenderable.builder()
        .setSource(this, R.raw.andy)
        .build()
        .thenAccept(renderable -> andyRenderable = renderable)
        .exceptionally(throwable -> {
          Toast toast = Toast.makeText(this, "Unable to load andy renderable", Toast.LENGTH_LONG);
          toast.setGravity(Gravity.CENTER, 0, 0);
          toast.show();
          return null;
        });
  }

  /**
   * Returns false and displays an error message if Sceneform can not run, true if Sceneform can run
   * on this device.
   *
   * <p>Sceneform requires Android N on the device as well as OpenGL 3.1 capabilities.
   *
   * <p>Finishes the activity if Sceneform can not run
   */
  public static boolean checkIsSupportedDeviceOrFinish(final Activity activity) {
    if (Build.VERSION.SDK_INT < VERSION_CODES.N) {
      Log.e(TAG, "Sceneform requires Android N or later");
      Toast.makeText(activity, "Sceneform requires Android N or later", Toast.LENGTH_LONG).show();
      activity.finish();
      return false;
    }
    String openGlVersionString = ((ActivityManager) activity.getSystemService(
        Context.ACTIVITY_SERVICE)).getDeviceConfigurationInfo().getGlEsVersion();
    if (Double.parseDouble(openGlVersionString) < MIN_OPENGL_VERSION) {
      Log.e(TAG, "Sceneform requires OpenGL ES 3.1 later");
      Toast.makeText(activity, "Sceneform requires OpenGL ES 3.1 or later", Toast.LENGTH_LONG)
          .show();
      activity.finish();
      return false;
    }
    return true;
  }
 
  private void onUpdateRender() {
    arFragment.getArSceneView().getScene().setOnUpdateListener(this::onUpdate);
  }

  public void onUpdate(FrameTime frameTime) {
    if (arFragment.getArSceneView().getArFrame() == null) {
      Log.d(TAG, "onUpdate: No frame available");
      // No frame available
      return;
    }

    arFragment.getPlaneDiscoveryController().hide();
    arFragment.getPlaneDiscoveryController().setInstructionView(null);

    if (arFragment.getArSceneView().getArFrame().getCamera().getTrackingState()
        != TrackingState.TRACKING) {
      Log.d(TAG, "onUpdate: Tracking not started yet");
      // Tracking not started yet
      return;
    }

    if (this.mAnchorNode == null && andyRenderable != null) {
      Log.d(TAG, "onUpdate: mAnchorNode is null");
      Session session = arFragment.getArSceneView().getSession();

            /*float[] position = {0, 0, -1};
            float[] rotation = {0, 0, 0, 1};
            Anchor anchor = session.createAnchor(new Pose(position, rotation));*/

      Vector3 cameraPos = arFragment.getArSceneView().getScene().getCamera().getWorldPosition();
      Vector3 cameraForward = arFragment.getArSceneView().getScene().getCamera().getForward();
      Vector3 position = Vector3.add(cameraPos, cameraForward.scaled(1.0f));

      // Create an ARCore Anchor at the position.
      Pose pose = Pose.makeTranslation(position.x, position.y, position.z);
      Anchor anchor = arFragment.getArSceneView().getSession().createAnchor(pose);

      mAnchorNode = new AnchorNode(anchor);
      mAnchorNode.setParent(arFragment.getArSceneView().getScene());

     /* Node node = new Node();
      node.setRenderable(andyRenderable);
      node.setParent(mAnchorNode);
      node.setOnTapListener((hitTestResult, motionEvent) -> {
      });*/

      TransformableNode transformableNode = new TransformableNode(arFragment.getTransformationSystem());
      transformableNode.setRenderable(andyRenderable);
      transformableNode.setParent(mAnchorNode);
    }
  }
}

[mistrydarshan99]

@dsternfeld7 What's wrong in above code?

[meetvora-ecosmob]

@mistrydarshan99 I've tried above code by updating ARCore v1.5.0 and sceneform v1.5.1. And your code is working fine. But the added object in the air keeps moving slightly. I think this is because of that plane detection process. The actions on that added object are working fine like zoom and scale.
One more thing to mention that I am not able to drag that object.

Is there any way that I can just hover that rendered object over the camera view and whenever the library detects any plane, I can stick that object to that anchor?