Aurellia Depth

This work presents Aurellia Depth. It is trained using the teacher-student model where the teacher model was the already available DINO-V2 transformer model and the student model is our model.

Key Achievements

• Successfully trained a smaller model for depth estimation without compromising accuracy too much.
• achieved a better frame rate on video processing.

Pre-trained Models

You can download the models from here:

Model	Params	Checkpoint
vit-b	97.5M	Download
vit-l	335.3M	Download

Usage

Prepraration

git clone https://github.com/Parvm1102/depth_prediction.git
cd depth_prediction
pip install -r requirements.txt

Download the checkpoints listed here and put them under the checkpoints directory.

Running script on images

python run.py \
  --encoder <vits | vitb | vitl | vitg> \
  --img-path <path> --outdir <outdir> \
  [--input-size <size>] [--pred-only] [--grayscale]

Options:

• --img-path: You can either 1) point it to an image directory storing all interested images, 2) point it to a single image, or 3) point it to a text file storing all image paths.
• --input-size (optional): By default, we use input size 518 for model inference. You can increase the size for even more fine-grained results.
• --pred-only (optional): Only save the predicted depth map, without raw image.
• --grayscale (optional): Save the grayscale depth map, without applying color palette.

For example:

python run.py --encoder vitl --img-path assets/examples --outdir depth_vis

Running script on videos

python run_video.py \
  --encoder <vits | vitb | vitl | vitg> \
  --video-path assets/examples_video --outdir video_depth_vis \
  [--input-size <size>] [--pred-only] [--grayscale]

Run Real time

python run_realtime_camera.py --encoder vitb --camera-id 0

Faster Real Time

source venv/bin/activate && python3 run_realtime_camera_optimized.py --encoder vitb --input-size 392 --skip-frames 1 --camera-width 640 --camera-height 480

Our larger model has better temporal consistency on videos.