🎬 Unity Scene Capture API

Production-ready scene capture system with multi-buffer support, batch processing, and 360° rotation capabilities for machine learning dataset generation.

📸 Features

🚀 Core Capabilities

• Multi-Channel Capture - Color, Depth, and Normal buffers
• Batch Processing - Capture multiple frames with rotation
• Random Scene Generation - Procedural object spawning for datasets
• Flexible Camera Setup - Use existing camera or create new one
• Custom Inspector - User-friendly Unity Editor integration

🎯 Use Cases

• 🤖 ML Dataset Generation
• 📊 Computer Vision Research
• 🎮 Synthetic Data Creation
• 🔬 Scene Analysis Tools

🏗️ Architecture

API Design Pattern

┌─────────────────────────────────────────────────┐
│          CaptureController (MonoBehaviour)      │
│  - User-facing component                        │
│  - Inspector UI customization                   │
│  - Multi-capture orchestration                  │
└──────────────────┬──────────────────────────────┘
                   │ uses
                   ▼
┌─────────────────────────────────────────────────┐
│          CaptureSession (IDisposable)           │
│  - Core API logic                               │
│  - Factory pattern (3 Create overloads)         │
│  - Channel management (bitwise flags)           │
│  - Resource lifecycle management                │
└──────────────────┬──────────────────────────────┘
                   │ uses
                   ▼
┌─────────────────────────────────────────────────┐
│              Replacement Shaders                │
│  - DepthGrayscale: Eye-space depth → grayscale  │
│  - NormalsRGB: World-space normals → RGB        │
└─────────────────────────────────────────────────┘

Namespace Structure

namespace SceneCapture
{
    public class CaptureSession : IDisposable
    {
        // Factory methods
        public static CaptureSession Create(Camera camera, ...);
        public static CaptureSession Create(Transform transform, ...);
        public static CaptureSession Create(Vector3 pos, Quaternion rot, ...);

        // Properties
        public CaptureChannels Channels { get; set; }
        public Vector2Int Resolution { get; set; }
        public float FieldOfView { get; set; }

        // Methods
        public void Capture(string name = null);
        public void SetPosition(Vector3 pos, Quaternion rot);
        public void Dispose();
    }
}

🚀 Quick Start

Basic Usage

using SceneCapture;

// Create session from existing camera
using (var session = CaptureSession.Create(Camera.main, "Assets/Captures"))
{
    session.Channels = CaptureSession.CaptureChannels.All;
    session.Resolution = new Vector2Int(1920, 1080);
    session.Capture("MyScene");
}

Multi-Capture with Rotation

// Attach CaptureController to GameObject
// Configure in Inspector:
// - Multi Capture Mode: ✓
// - Capture Count: 36
// - Rotate Around Target: ✓
// - Target Object: [Your Object]
// - Rotation Radius: 5

// Press Space or call from code:
captureController.TakeCapture();

Random Scene Generation

// Enable in Inspector:
// - Generate Random Objects: ✓
// - Object Count: 10
// - Spawn Area: 5
// - Size Range: 0.5 - 2.0

// Scene auto-generates on capture
// Press 'R' to regenerate scene manually

📋 Component Reference

CaptureController (MonoBehaviour)

Inspector Properties:

Section	Property	Description
Camera	Source Camera	Reference camera (optional)
Output	Save Path	Directory for captures
	Base Name	Filename prefix
	Use Timestamp	Add timestamp to filenames
	Width/Height	Output resolution
Channels	Capture Color	Enable color buffer
	Capture Depth	Enable depth buffer
	Capture Normals	Enable normal buffer
Controls	Capture Key	Trigger key (default: Space)
	Capture On Start	Auto-capture on play
Multi-Capture	Multi Capture Mode	Enable batch processing
	Capture Count	Number of captures
Rotation	Rotate Around Target	360° rotation mode
	Target Object	Object to orbit
	Rotation Radius	Orbit distance
	Rotation Height	Camera elevation
Random Scene	Generate Random Objects	Procedural scene
	Object Count	Number of objects (1-50)
	Spawn Area	Scene size
	Size Range	Object scale range

CaptureSession API

Factory Methods:

// Use existing camera
CaptureSession.Create(Camera camera, string path, Vector2Int resolution);

// Use transform for position/rotation
CaptureSession.Create(Transform transform, string path, Vector2Int resolution);

// Create new camera at position
CaptureSession.Create(Vector3 pos, Quaternion rot, string path, Vector2Int resolution);

Properties:

session.SavePath = "Assets/MyDataset";
session.Resolution = new Vector2Int(2048, 2048);
session.BaseName = "Frame";
session.UseTimestamp = false;
session.FieldOfView = 60f;

// Bitwise channel selection
session.Channels = CaptureChannels.Color | CaptureChannels.Depth;

Methods:

// Capture single frame
session.Capture("custom_name");

// Set camera position
session.SetPosition(new Vector3(0, 5, -10), Quaternion.identity);
session.SetPosition(myTransform);

// Reset frame counter
session.ResetIndex();

🎨 Channel Details

Color Buffer

• Standard scene rendering
• ARGB32 format
• Includes lighting and materials

Depth Buffer

• Eye-space depth normalized by max scene depth
• White = near objects
• Black = far objects
• Grayscale PNG output
• Custom _MaxDepth parameter (auto-calculated)

Shader Implementation:

// Shader: Custom/DepthGrayscale
COMPUTE_EYEDEPTH(o.depth);
float normalizedDepth = i.depth / _MaxDepth;
float depthValue = 1.0 - normalizedDepth; // Invert for better visualization

Normal Buffer

• World-space surface normals
• RGB encoding: R=X, G=Y, B=Z
• Range mapped from [-1,1] to [0,1]

Shader Implementation:

// Shader: Custom/NormalsRGB
float3 worldNormal = UnityObjectToWorldNormal(v.normal);
float3 encodedNormal = normal * 0.5 + 0.5;

📊 Output Structure

Single Capture

Assets/Captures/
├── Capture_2025-01-17_14-23-45-123_Color.png
├── Capture_2025-01-17_14-23-45-123_Depth.png
└── Capture_2025-01-17_14-23-45-123_Normals.png

Multi-Capture (Batch Mode)

Assets/Captures/Batch_2025-01-17_14-25-30/
├── Frame_0000_Color.png
├── Frame_0000_Depth.png
├── Frame_0000_Normals.png
├── Frame_0001_Color.png
├── Frame_0001_Depth.png
├── ...
└── Frame_0035_Normals.png

🔧 Advanced Usage

Custom Capture Loop

using (var session = CaptureSession.Create(Camera.main, "Dataset"))
{
    session.UseTimestamp = false;
    session.Channels = CaptureChannels.Depth | CaptureChannels.Normals;

    for (int i = 0; i < 100; i++)
    {
        // Position camera
        float angle = i * Mathf.PI * 2f / 100f;
        Vector3 pos = new Vector3(Mathf.Cos(angle) * 10f, 2f, Mathf.Sin(angle) * 10f);
        session.SetPosition(pos, Quaternion.LookRotation(-pos));

        // Capture
        session.Capture($"orbit_{i:D3}");

        // Modify scene between captures
        RandomizeObjectPositions();
    }
}

Programmatic Channel Selection

// Using bitwise flags
var channels = CaptureSession.CaptureChannels.None;

if (needsColor) channels |= CaptureSession.CaptureChannels.Color;
if (needsDepth) channels |= CaptureSession.CaptureChannels.Depth;
if (needsNormals) channels |= CaptureSession.CaptureChannels.Normals;

session.Channels = channels;

Dynamic Resolution Adjustment

// Start with low res for testing
session.Resolution = new Vector2Int(512, 512);
session.Capture("test_lowres");

// Switch to high res for final dataset
session.Resolution = new Vector2Int(4096, 4096);
session.Capture("final_highres");

🎓 Code Patterns Demonstrated

1. Factory Pattern

Three overloaded Create() methods for flexible instantiation:

Create(Camera camera, ...)      // Use existing camera
Create(Transform transform, ...) // Use transform position
Create(Vector3 pos, ...)         // Create new camera

2. IDisposable Pattern

Proper resource cleanup:

public void Dispose()
{
    if (_ownsCamera && _cameraObj != null)
        Object.Destroy(_cameraObj);

    _camera = null;
    _cameraObj = null;
}

3. Bitwise Enum Flags

Flexible channel selection:

[Flags]
public enum CaptureChannels
{
    None = 0,
    Color = 1,
    Depth = 2,
    Normals = 4,
    All = Color | Depth | Normals
}

4. Custom Editor

Enhanced Inspector UI:

#if UNITY_EDITOR
[CustomEditor(typeof(CaptureController))]
public class CaptureControllerEditor : Editor
{
    public override void OnInspectorGUI()
    {
        // Custom folder picker button
        // Runtime capture buttons
        // Organized property groups
    }
}
#endif

5. Coroutine-Based Batching

Frame-by-frame multi-capture:

private IEnumerator MultiCapture()
{
    for (int i = 0; i < captureCount; i++)
    {
        // Setup scene
        yield return new WaitForEndOfFrame();

        // Capture frame
        session.Capture($"{baseName}_{i:D4}");

        yield return new WaitForEndOfFrame();
    }
}

🛠️ Requirements

• Unity: 6.2 or higher (compatible with Unity 2020+)
• Render Pipeline: Built-in RP
• Dependencies: None (self-contained)

📈 Performance

Aspect	Value
Single Capture	<50ms @ 1920x1080
Multi-Capture (36 frames)	~2-3 seconds
Memory Usage	~30MB temporary RT memory
Depth Calculation	O(n) renderers in scene

🐛 Troubleshooting

Shaders Not Found

Error: Missing shader: Custom/DepthGrayscale

Solution: Ensure shaders are in Assets/Shaders/ and named correctly.

Blank Output Images

Common causes:

• Camera culling mask excludes scene objects
• Objects outside camera frustum
• Shaders not assigned to materials

Multi-Capture Rotation Issues

Check:

• Target Object is assigned
• Rotation Radius > 0
• Target Object is centered correctly

🎯 Best Practices

For ML Datasets

session.Resolution = new Vector2Int(512, 512);  // Standard dataset size
session.UseTimestamp = false;                    // Sequential naming
session.Channels = CaptureChannels.All;          // Capture all channels

For High-Quality Renders

session.Resolution = new Vector2Int(4096, 4096);
session.FieldOfView = 45f;  // Narrower FOV for less distortion
camera.farClipPlane = 100f;  // Limit depth range

Memory Management

// Always use 'using' statement for automatic cleanup
using (var session = CaptureSession.Create(...))
{
    // Captures here
} // Automatically disposes resources

📚 Learning Resources

Concepts Covered

• Factory design pattern
• IDisposable pattern
• Bitwise enum flags
• Custom Editor scripting
• RenderTexture management
• Replacement shader rendering
• Coroutine-based workflows

Unity APIs Used

• Camera.RenderWithShader()
• RenderTexture.GetTemporary()
• FindObjectsByType<Renderer>()
• CustomEditor attribute
• EditorUtility.OpenFolderPanel()

🤝 Contributing

Suggestions for improvements:

• Add segmentation mask channel
• Support URP/HDRP pipelines
• Export metadata (camera poses, scene info)
• Real-time preview window
• Video sequence capture
• Custom shader injection

📄 License

MIT License - Free for personal and commercial use

📧 Contact

For questions or collaboration, please open a GitHub issue.

---

Production-ready tool for computer vision and graphics research 🎯