Receipt_AR Waste Management

AR Cooking Book: Interactive Augmented Reality Experience

Overview

The AR Cooking Book is an interactive Augmented Reality (AR) experience that allows users to place, move, rotate, and scale virtual kitchen objects while following a step-by-step cooking guide. Using Unity’s AR Foundation and XR Interaction Toolkit, users can engage in a realistic AR cooking simulation, interacting with utensils and ingredients in their real-world space.

---

Table of Contents

1. Setting Up the Project
2. Deploying to Android
3. Placing and Manipulating Objects in AR
4. Gesture Controls: Move, Rotate, Scale
5. Implementing Cooking Steps in AR
6. Testing and Debugging
7. Future Enhancements

---

Setting Up the Project

A. Install Unity and Required Packages

1. Open Unity Hub and create a new 3D project.
2. Install AR Foundation & XR Interaction Toolkit:
   • Go to "Window → Package Manager".
   • Install:
     • AR Foundation
     • XR Interaction Toolkit
     • ARCore XR Plugin (Android) / ARKit XR Plugin (iOS)
3. Enable XR Plugin Management:
   • Open "Edit → Project Settings → XR Plug-in Management".
   • Under Android, enable ARCore.
   • Under iOS, enable ARKit.

** B. Create AR Components in Unity**

Main Steps in the Video: "Deploying an AR App with Unity AR Foundation"

---

1. Introduction to AR Foundation

• Overview of Unity's AR Foundation package.
• AR Foundation supports Android, iOS (ARKit), and HoloLens.
• Features include plane detection, image tracking, object tracking, face tracking, body tracking, and more.

---

2. Understanding AR Foundation Structure

• Subsystems and Providers:
  • Unity provides subsystems (interfaces with required functions).
  • Providers (e.g., ARCore for Android, ARKit for iOS) implement these functions.
• Managers:
  • Help Unity access subsystem data via game objects.
  • Example: Plane Manager creates a game object when a plane is detected.

---

3. Setting Up Unity for AR Development

• Start a new project with Unity's AR Template (pre-installed AR packages & sample scene).
• Installing Necessary Modules:
  • For Android: Install Android Build Support.
  • For iOS: Install iOS Build Support.
• Adding AR Foundation in an Existing Project:
  • Open Package Manager → Download AR Foundation.
  • Install ARCore XR Plugin (for Android) or ARKit XR Plugin (for iOS).
• Enabling Input System:
  • Helps with better control and interaction.

---

4. Configuring Unity Build Settings

• Switch to the Target Platform (Android/iOS).
• Enable Development Build (for easy testing).
• Adjust Player Settings:
  • iOS: Set Minimum iOS Version to 11.0, enable ARKit Support, add Bundle Identifier.
  • Android: Set Minimum API Level (24 for AR apps, 19 for non-AR apps).
  • Remove Vulkan Graphics API (not supported in AR).
  • Set IL2CPP as the scripting backend for Android Play Store builds.

---

5. Enabling XR Plugin Management

• Go to XR Plugin Management in Player Settings.
• Enable ARCore (Android) or ARKit (iOS).
• Check settings such as face tracking if needed.

---

6. Setting Up AR Scene in Unity

• Create AR Session Game Object:
  • Add AR Session Component (Manages AR lifecycle).
  • Add AR Input Manager (Handles user interactions).
• Create AR Session Origin:
  • Converts AR session space to Unity world coordinates.
  • Attach an AR Camera (with AR Camera Manager & Tracked Pose Driver).
• Adding Objects in AR Space:
  • Place a 3D object (Cube) in front of the camera for proper positioning.

---

7. Building and Deploying to Mobile

• Go to Build Settings → Select correct platform → Add open scenes.
• Click Build, create a Builds Folder, and export the project.
• For iOS:
  • Open in Xcode, enable Automatic Signing, and deploy to a connected iPhone.
• For Android:
  • Build directly to phone.
  • Ensure Developer Mode is enabled on the device.

---

8. Running the AR App on Your Phone

• Launch the AR app on the phone.
• Grant camera access (iOS).
• The placed 3D Cube appears in AR space.

---

9. Exploring Unity's AR Foundation Samples

• Unity provides sample AR projects on GitHub.
• Download the sample scenes (e.g., image tracking, plane detection) for reference.
• Future tutorials will cover advanced AR features.

--

1. Add an AR Camera:
   • Hierarchy → XR → AR Session Origin.
   • Inside AR Session Origin, add:
     • AR Camera Manager
     • AR Raycast Manager
     • AR Pose Driver
2. Enable Plane Detection:
   • Hierarchy → Create Empty → Rename it "AR Plane Manager".
   • Add:
     • AR Plane Manager -AR Raycast Manager

Now, Unity is ready for AR development! 🎉

---

Deploying to Android

A. Enable Developer Mode on Android

1. Open Settings → About Phone → Tap "Build Number" 7 times.
2. Enable USB Debugging in Developer Options.
3. Connect your phone to your PC via USB.

B. Set Up Unity for Android Deployment

1. In Unity, go to "File → Build Settings".
2. Select Android and click Switch Platform.
3. Install Android Build Support & SDK via Unity Hub.
4. Set Player Settings:
   • Minimum API Level: Android 7.0 or higher.
   • Scripting Backend: IL2CPP.
5. Click Build & Run to install the app on your Android device.

---

Placing and Manipulating Objects in AR

A. Implement AR Object Placement

1. Create an "AR Placement Manager" GameObject.
2. Attach the following script to enable object placement:

using UnityEngine;
using UnityEngine.XR.ARFoundation;
using UnityEngine.XR.ARSubsystems;
using System.Collections.Generic;

public class ARPlacement : MonoBehaviour
{
    public GameObject objectToPlace;
    public ARRaycastManager raycastManager;
    private List<ARRaycastHit> hits = new List<ARRaycastHit>();

    void Update()
    {
        if (Input.touchCount > 0 && Input.GetTouch(0).phase == TouchPhase.Began)
        {
            Touch touch = Input.GetTouch(0);
            if (raycastManager.Raycast(touch.position, hits, TrackableType.PlaneWithinPolygon))
            {
                Pose hitPose = hits[0].pose;
                Instantiate(objectToPlace, hitPose.position, hitPose.rotation);
            }
        }
    }
}

3. Assign a prefab (e.g., pot, knife, or ingredient) to objectToPlace.

Now, users can tap on surfaces to place kitchen items!

---

Gesture Controls: Move, Rotate, Scale Objects

A. Enable Object Interaction

1. Open your cooking object prefab (e.g., pot).
2. Add:
   • XR Grab Interactable
   • Rigidbody (Use Gravity = False)
   • Collider

B. Implement Pinch-to-Scale (Resize Objects)

1. Attach this script to all interactable objects:

using UnityEngine;

public class PinchToScale : MonoBehaviour
{
    private float initialDistance;
    private Vector3 initialScale;

    void Update()
    {
        if (Input.touchCount == 2)
        {
            Touch touch0 = Input.GetTouch(0);
            Touch touch1 = Input.GetTouch(1);

            if (touch0.phase == TouchPhase.Moved || touch1.phase == TouchPhase.Moved)
            {
                float currentDistance = Vector2.Distance(touch0.position, touch1.position);
                if (initialDistance == 0)
                {
                    initialDistance = currentDistance;
                    initialScale = transform.localScale;
                }
                else
                {
                    float factor = currentDistance / initialDistance;
                    transform.localScale = initialScale * factor;
                }
            }
        }
        else
        {
            initialDistance = 0;
        }
    }
}

---

Implementing Cooking Steps in AR

1. Display step-by-step instructions in a UI overlay.
2. Implement voice commands (e.g., "Next Step") to move through the recipe.
3. Add animations for ingredient mixing and food preparation.

---

Student Name: Rawan Gomaa