How to make the plane renderer have a single non-repeating texture on it's material?

[lokiiarora]

Something along the lines of the image below:

The idea is that every time the user moves the phone, the texture's center should be focused on the focus point (like this does for spotlight) of the frame, and this should ideally be the same height / width of the current model that is downloaded.

Questions:

1. Is it possible with the current .filamat for the plane? If not, how do I go forward with moving the texture to the focus point?
2. How to go forward with the texture's XY scaling with respect to the size of the model?

[grassydragon]

Hi!
Instead of modifying the plane material you can display it as an object in the scene. The easiest way to do that would be to use a ViewRenderable with a FixedWidthViewSizer and set the width of the ViewSizer and the aspect ratio of the displayed view according to the downloaded model.

[RGregat]

A very basic implementation of what @grassydragon suggested

Layout.xml

<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="wrap_content"
    android:layout_height="wrap_content"
    android:id="@+id/linearLayout">

    <ImageView
        android:id="@+id/image_view_ar"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:background="@drawable/ic_tracking_plane"
        tools:layout_conversion_absoluteHeight="50dp"
        tools:layout_editor_absoluteY="0dp"
        tools:layout_editor_absoluteX="0dp"
        tools:layout_conversion_absoluteWidth="50dp" />
</LinearLayout>

ic_tracking_plane

<vector xmlns:android="http://schemas.android.com/apk/res/android"
    android:width="60dp"
    android:height="60dp"
    android:viewportWidth="60"
    android:viewportHeight="60">
    <path
        android:pathData="M2.575659303251037,2.306419295191382 L57.829926365885946,2.296878830805386 L57.90754413020447,57.40356772363032 L2.488460406775956,57.248990756189016 L2.575659303251037,2.306419295191382 z"
        android:strokeWidth="2"
        android:strokeColor="@color/colorTrackingPlane" />
</vector>

Create a ViewRenderable

View view = ((LayoutInflater) context
   .getSystemService(Service.LAYOUT_INFLATER_SERVICE))
   .inflate(configuration.getLayoutId(), null);


CompletableFuture<ViewRenderable> viewRenderable = ViewRenderable
   .builder()
   .setView(context, view)
   .build();

handle the first HitResult to create a Anchor

viewRenderable.thenAccept(renderable -> {
   anchor = session.createAnchor(hit.getHitPose());

   tackingAnchorNode = new AnchorNode(anchor);
   fragment.getArSceneView().getScene().addChild(trackingAnchorNode); 
   node = new Node();
   node.setRenderable(renderable); 
   node.setParent(tackingAnchorNode);
}

update position of the view based on a HitResult

if (!node.isEnabled())
   return;

Pose hitPose = hit.getHitPose();
Vector3 oldPosition = node.getWorldPosition();
Vector3 desiredPosition = new Vector3(
   hitPose.tx(),
   hitPose.ty(),
   hitPose.tz());

node.setWorldPosition(desiredPosition);
                            
Quaternion rotation1 = Quaternion.axisAngle(new Vector3(1.0f, 0.0f, 0.0f), 90);
node.setWorldRotation(rotation1);

The posted code might not work by simple copy & paste, but it gives you a general idea how to implement what you want.

[RGregat]

Maybe not, but isn't it not necessary to anchor Nodes somewhere?

[grassydragon]

After we place the model and want to fix its position in the real world we can create an anchor. But since we are repositioning the object every frame using the camera and recognized planes it is not a problem if the coordinate space updates.

[lokiiarora]

Can we utilize the hit test performed on PlaneRenderer at this line

[ThomasGorisse]

You should better use your own hit test like this:

// Update each frame (
arSceneView.scene.addOnUpdateListener { frameTime -> // You can reduce frame rate using frameTime to skip bellow
    arSceneView.arFrame?.let { frame ->
        // Check ARCore tracking state
        if (frame.camera?.trackingState == TrackingState.TRACKING) {
            // Hit test at the view/screen center
            for (hitResult in frame.hitTest(
                arSceneView.width / 2.0f,
                arSceneView.height / 2.0f
            )) {
                // Do whatever you want with the center hitResult depending on its type and on your need
                when (val trackable = hitResult.trackable!!) {
                    is Plane -> if(trackable.isPoseInPolygon(hitResult.hitPose)) { }
                    is DepthPoint -> {}
                    is Point -> if(trackable.orientationMode == Point.OrientationMode.ESTIMATED_SURFACE_NORMAL) {}
                    is InstantPlacementPoint -> {}
                }
            }
        }
    }
}

It should not be very consuming compared to how Sceneform internal frame HiTests duplicates originally (before we cleaned it) was.

[ThomasGorisse]

Extracted from the upcoming sceneview-android push:

/**
 * ### Performs a ray cast to retrieve the hit trackables
 *
 * Performs a ray cast from the user's device in the direction of the given location in the
 * camera view. Intersections with detected scene geometry are returned, sorted by distance from
 * the device; the nearest intersection is returned first.
 *
 * When using:
 * - **Plane and/or Depth:** Significant geometric leeway is given when returning hit results.
 * For example, a plane hit may be generated if the ray came close, but did not actually hit
 * within the plane extents or plane bounds
 * [Plane.isPoseInExtents][com.google.ar.core.Plane.isPoseInExtents] and
 * [Plane.isPoseInPolygon][com.google.ar.core.Plane.isPoseInPolygon] can be used to determine
 * these cases). A point (point cloud) hit is generated when a point is roughly within one
 * finger-width of the provided screen coordinates.
 *
 * - **Instant Placement:** Ray cast can return a result before ARCore establishes full
 * tracking. The pose and apparent scale of attached objects depends on the
 * [InstantPlacementPoint][com.google.ar.core.InstantPlacementPoint] tracking method and the
 * provided approximateDistanceMeters. A discussion of the different tracking methods and the
 * effects of apparent object scale are described in
 * [InstantPlacementPoint][com.google.ar.core.InstantPlacementPoint].
 * This function will succeed only if
 * [Config.InstantPlacementMode][com.google.ar.core.Config.InstantPlacementMode] is
 * [com.google.ar.core.Config.InstantPlacementMode.LOCAL_Y_UP] in the ARCore session
 * configuration, the ARCore session tracking state is
 * [TrackingState.TRACKING][com.google.ar.core.TrackingState.TRACKING] }, and there are
 * sufficient feature points to track the point in screen space.
 *
 * - **[Session.Feature.FRONT_CAMERA][com.google.ar.core.Session.Feature.FRONT_CAMERA]:**
 * The returned hit result list will always be empty, as the camera is not
 * [TrackingState.TRACKING][com.google.ar.core.TrackingState.TRACKING]. Hit testing against
 * tracked faces is not currently supported.
 *
 * Note: In ARCore 1.24.0 or later on supported devices, if depth is enabled by calling
 * [com.google.ar.core.Config.setDepthMode] with the value
 * [Config.DepthMode.AUTOMATIC][com.google.ar.core.Config.DepthMode.AUTOMATIC], the returned
 * list includes [com.google.ar.core.DepthPoint] values sampled from the latest computed depth
 * image.
 *
 * @param xPx x view coordinate in pixels
 * *Default: The view width center*
 * @param yPx y view coordinate in pixelsls
 * *Default: The view height center*
 * @param approximateDistanceMeters the distance at which to create an
 * [InstantPlacementPoint][com.google.ar.core.InstantPlacementPoint]. This is only used while
 * the tracking method for the returned point is
 * [InstantPlacementPoint.TrackingMethod.SCREENSPACE_WITH_APPROXIMATE_DISTANCE]
 * [com.google.ar.core.InstantPlacementPoint.TrackingMethod.SCREENSPACE_WITH_APPROXIMATE_DISTANCE]
 * *Default: [ArSession.approximateDistanceMeters]*
 * @param plane enable plane results
 * *Default: [ArSession.planeFindingEnabled]*
 * @param depth enable depth results
 * *Default: [ArSession.depthEnabled]*
 * @param instantPlacement enable instant placement results
 * *Default: [ArSession.instantPlacementEnabled]*
 *
 * @return an ordered list of intersections with scene geometry, nearest hit first
 * In case of instant placement result, if successful a list containing a single
 * [HitResult][com.google.ar.core.HitResult], otherwise an empty list.
 * The [HitResult][com.google.ar.core.HitResult] will have a trackable
 * of type [InstantPlacementPoint][com.google.ar.core.InstantPlacementPoint]
 */
@JvmOverloads
fun hitTests(
    xPx: Float = session.displayWidth / 2.0f,
    yPx: Float = session.displayHeight / 2.0f,
    approximateDistanceMeters: Float = session.approximateDistanceMeters,
    plane: Boolean = session.planeFindingEnabled,
    depth: Boolean = session.depthEnabled,
    instantPlacement: Boolean = session.instantPlacementEnabled
): List<HitResult> {
    if(camera.isTracking) {
        if (plane || depth) {
            frame.hitTest(xPx, yPx).takeIf { it.isNotEmpty() }?.let {
                return it
            }
        }
        if (instantPlacement) {
            return frame.hitTestInstantPlacement(xPx, yPx, approximateDistanceMeters)
        }
    }
    return listOf()
}

/**
 * @see hitTests
 * @see firstValid
 */
@JvmOverloads
fun hitTest(
    xPx: Float = session.displayWidth / 2.0f,
    yPx: Float = session.displayHeight / 2.0f,
    approximateDistanceMeters: Float = session.approximateDistanceMeters,
    plane: Boolean = session.planeFindingEnabled,
    depth: Boolean = session.depthEnabled,
    instantPlacement: Boolean = session.instantPlacementEnabled
): HitResult? {
    return hitTests(
        xPx = xPx,
        yPx = yPx,
        approximateDistanceMeters = approximateDistanceMeters,
        plane = plane,
        depth = depth,
        instantPlacement = instantPlacement
    ).firstValid(camera)
}

/**
 * ### Best HitResult within the list in precision terms
 *
 * Hits are sorted by depth. Consider only closest hit on a plane, Oriented Point, Depth Point,
 * or Instant Placement Point.
 */
fun Collection<HitResult>.firstValid(camera: Camera): HitResult? {
    return firstOrNull { hitResult ->
        when (val trackable = hitResult.trackable!!) {
            is Plane -> trackable.isPoseInPolygon(hitResult.hitPose) &&
                    hitResult.hitPose.calculateDistanceToPlane(camera.pose) > 0.0f
            is DepthPoint -> trackable.isTracking
            is Point -> trackable.orientationMode == Point.OrientationMode.ESTIMATED_SURFACE_NORMAL
            is InstantPlacementPoint -> true
            else -> false
        }
    }
}

[ThomasGorisse]

May I suggest moving to sceneView-android?
The result you will have is nothing compared since you will have the depth and soon instant (already done but waiting for commit) placement.
What you describe is actually what the CursorNode is doing:
https://sceneview.github.io/assets/videos/screenrecords/04.mp4

https://github.com/SceneView/sceneview-android/blob/0a1d36c8a097f1e1fd9e43c59226789147544836/arsceneview/src/main/kotlin/io/github/sceneview/ar/node/CursorNode.kt#L17-L71
Simply override your own cursor node without the clicking behavior stuffs and instead of using a Glb renderable, use a View Renderable.

[lokiiarora]

We have already performed a lot of customisations on the Sceneform repo.
After the MVP, we do wish to move to it.

[dhaval-android]

Is it possible to implement the same thing as above in scenview-android?

[dhaval-android]

@grassydragon
Do you have or other have idea how to draw simple ring like above video at bottom of node, Before we can use footVisulizer but in this sdk how we can do it.

[grassydragon]

Is it possible to use a model with a texture in your case?

[dhavalenstead]

yes, But how we can adjust bottom of one model with size of top model?

[grassydragon]

You can use the dimensions from the bounding box of the top model.

[dhavalenstead]

Thanks I Alread did it.