NOTE: THE README CONSIST OF IN DETAIL PROJECTS , RESUME IS IN "TarunPandianM_BTech_CSE_AIML.pdf"
I am a Computer Vision Engineer with a research-oriented approach to building vision systems, from problem formulation through algorithm design to deployment and runtime optimization. I avoid treating models or frameworks as black boxes; I prefer to understand the underlying mechanisms and failure modes of a system before relying on it in practice. This drives me toward systematic investigation, evaluating multiple approaches, studying relevant literature and documentation, and reasoning from first principles rather than defaulting to established conventions. The same methodology carries across every stage of my work, from research and algorithm development to the engineering trade-offs required to make a system reliable in production. I am currently extending this approach beyond computer vision as well, investing time in understanding multimodal AI and audio reasoning through the redesign of an ongoing personal project.
Bengaluru, India • March 2026 – July 2026
During my internship at Tonbo Imaging, I worked on research and development projects covering EO/IR image processing, object detection model deployment, dataset generation, and computer vision infrastructure. My work involved developing image processing algorithms, optimizing deep learning models for embedded hardware, building software tools to simplify experimentation and deployment, and supporting end-to-end computer vision workflows from data preparation to inference. A good part of the internship also involved reading research papers to understand the reasoning behind existing approaches before building on or replacing them, along with regular chats with my colleagues who walked me through the reasoning behind design decisions and helped me work through new ideas. It was one of the most useful parts of my internship, since it shaped how I approached the technical problems that followed.
Deployed a 70.5M-parameter Cascade R-CNN (ResNet-50-FPN) trained for aerial tiny object detection to NVIDIA Jetson Orin NX for real-time embedded inference. The project centered on building a reliable, production-grade deployment pipeline — diagnosing and resolving TensorRT-specific correctness issues, redesigning post-processing, and validating performance trade-offs — while preserving detection accuracy across the PyTorch > ONNX > TensorRT conversion.
Technical Contributions & Results
- Converted the trained PyTorch model into ONNX and TensorRT engines, resolving MMDeploy-generated graph incompatibilities and custom TensorRT plugin/operator constraints to enable successful engine generation on NVIDIA Jetson Orin NX.
- Identified a correctness bug in TensorRT's TRTBatchedNMS plugin where padded output values corrupted low-confidence detections; fixed by modifying the ONNX graph to expose raw bounding box/confidence outputs and moving NMS outside the TensorRT engine.
- Designed a custom Hard NMS implementation and integrated it into a production C++/CUDA inference pipeline, restoring correct detection behavior with no latency regression (138.46 ms vs. 141 ms baseline).
- Benchmarked deployment configurations across proposal counts and confidence thresholds to study the latency-vs-accuracy trade-off; increasing proposal limits improved AP50 and AR@100 recall.
- Built a standalone C++ inference application supporting image/video inference, performance logging, and quantitative on-device evaluation.
- Final deployment: FP16 TensorRT engine (~135 MiB) for the 70,468,546-parameter model, achieving 0.3217 mAP@0.5 on the target validation set, representing a 23.6% relative reduction from the 0.421 mAP@0.5 PyTorch baseline while investigating and quantifying the accuracy-latency trade-offs introduced by FP16 quantization and TensorRT-specific deployment constraints.
Tech Stack: TensorRT • ONNX • MMDeploy • CUDA • C++ • NVIDIA Jetson Orin NX • PyTorch • MMDetection
Developed a fully classical computer vision pipeline to remove HUD and UI overlays from Electro-Optical (EO) and Infrared (IR) imagery while preserving underlying scene structure and sensor characteristics for downstream machine learning. The project began by evaluating multiple reconstruction strategies, including PDE-based inpainting, deep learning approaches, watershed segmentation, wavefront propagation, and progressively refined ray-guided methods, before converging on a custom unified reconstruction pipeline designed specifically for EO/IR imagery and operational constraints.
Technical Contributions & Results
- Evaluated and iteratively refined multiple inpainting strategies, analysing the structural and statistical limitations of each approach before designing the final reconstruction algorithm.
- Designed a unified wavefront-based reconstruction algorithm combining distance transform propagation, multi-directional ray casting, adaptive drift validation, and sequential boundary-guided filling, preserving scene geometry across large overlay regions where prior methods failed.
- Developed a sensor-aware noise reconstruction pipeline using residual donor sampling, adaptive sensor noise estimation, read and shot noise modelling, and fractional Brownian motion synthesis, reproducing EO and IR sensor characteristics after reconstruction.
- Built an automated UI mask extraction pipeline capable of detecting anti-aliased overlays and generating reconstruction masks for large scale image and video processing.
- Systematically diagnosed reconstruction artefacts including texture collapse, structural drift, colour bleeding, edge degradation, and noise mismatch, feeding each finding into successive algorithm revisions to reach a stable pipeline.
- Delivered a stable, unified reconstruction framework integrating structural reconstruction and sensor-aware noise synthesis into a single processing pipeline, with a scalable batch workflow supporting automated processing of image collections and video sequences for dataset generation.
Tech Stack: Python • OpenCV • NumPy • Classical Computer Vision • Image Processing • EO/IR Imaging • Wavefront Algorithms • Directional Ray Casting • Fractional Brownian Motion
Developed an internal platform for evaluating and testing computer vision models across multiple deep learning frameworks through a unified interface. The project addressed inconsistent execution pipelines, framework specific dependencies, and environment conflicts by providing isolated runtime environments, standardized inference workflows, and a common interface for deploying and previewing models without manual framework setup.
Technical Contributions & Results
- Designed a unified inference interface capable of executing models from PyTorch, MMDetection, TorchVision, and Ultralytics YOLO without requiring framework specific execution logic from the user, reducing new model integration to a standardized configuration step.
- Built Docker isolated runtime environments to eliminate dependency conflicts, enabling independent execution of different computer vision frameworks on shared GPU infrastructure and removing the need for per framework manual environment setup.
- Standardized model execution by abstracting framework specific preprocessing, inference, and post-processing into a common execution pipeline, producing consistent behaviour across all supported frameworks.
- Developed asynchronous task execution using Celery and Redis to manage long running inference jobs, improving platform responsiveness and supporting concurrent workloads across multiple users.
- Implemented model caching, runtime lifecycle management, and automated validation workflows, reducing repeated initialization overhead during experimentation.
- Delivered a centralized platform unifying model upload, execution, visualization, and inference testing across PyTorch, MMDetection, TorchVision, and Ultralytics YOLO, reducing the manual setup effort previously required to evaluate a new model or framework.
Tech Stack: Python • Django • FastAPI • Docker • Celery • Redis • PyTorch • MMDetection • TorchVision • Ultralytics YOLO
Designed a semi automated annotation pipeline for generating object detection labels on aerial imagery by combining multiple complementary detection models instead of relying on a single detector. The project evaluated detector behaviour for tiny objects, open vocabulary categories, rooftops, and segmentation based recovery, then integrated the strongest performing approaches into a priority based annotation workflow producing standardized COCO annotations.
Technical Contributions & Results
- Evaluated seven detection approaches, including Cascade R-CNN, YOLO, Detic, GroundingDINO, LocateAnything, RT-DETR, and Grounded SAM, to identify the strongest performing models for aerial and small object detection tasks.
- Built the primary annotation layer using a tiny object detector for small aerial targets and a YOLO based detector for people and vehicles, enriched with Detic and LocateAnything to recover text defined categories outside the primary detectors' coverage.
- Iteratively refined open vocabulary ontologies for Detic, restructuring prompts and object descriptions to improve detection quality and reduce false positives across diverse aerial scenes.
- Investigated SAHI based tiled inference across multiple slice sizes and overlap strategies, improving detection performance for small rooftops and densely packed objects in high resolution imagery.
- Designed a priority based annotation merger that resolves detections from multiple models using detector priority ranking, geometric filtering, overlap resolution, and non maximum suppression, producing a single unified COCO annotation dataset.
- Evaluated RT-DETR for rooftop localization and Grounded SAM for segmentation assisted recovery as experimental proposal generators for future pipeline extension.
Tech Stack: Python • OpenCV • PyTorch • MMDetection • YOLO • Detic • GroundingDINO • Grounded SAM • RT-DETR • SAHI • COCO Dataset Format
(Undergoing Redesign)
Developing a multimodal forensic audio analysis system for tactical speech and acoustic event understanding. Current work focuses on building a reliable end-to-end training pipeline under limited GPU resources by redesigning data preparation, preprocessing, and training workflow to support stable fine-tuning of a large audio language model while improving the quality of supervision provided to it.
Technical Contributions & Results
- Built a complete data preparation pipeline, converting raw forensic audio samples into a structured multimodal training dataset with synchronized audio and semantic annotations.
- Redesigned the annotation schema to capture meaningful semantic information instead of simple labels, reducing metadata shortcuts and forcing the model to learn from audio content rather than label patterns.
- Developed a custom data collator and preprocessing pipeline to correctly handle variable length audio samples, resolving preprocessing failures encountered during training.
- Optimized the training workflow for resource constrained GPUs using parameter-efficient fine-tuning, mixed precision training, gradient checkpointing, and memory-aware data loading, enabling stable training of a large multimodal model on commodity hardware.
- Reworked dataset validation, feature generation, batching, preprocessing, and training configuration through iterative debugging to establish a reliable end to end training pipeline.
- Reduced trainable parameters to approximately 82.8 million, roughly 1.68% of the full model, using parameter efficient fine tuning, producing a reusable training framework adaptable to additional forensic audio tasks.
Tech Stack: Python • PyTorch • Hugging Face Transformers • PEFT • QLoRA • Librosa • NumPy • Gradio
Feel free to check out my other projects where some i started and then learnt a lot from it but didnt quite optimise it,
so there will be few projects which is not production ready type, these are my personal projects i used them to learn the language,
the process and i primarily work in kaggle notebook for their gpu access, so not all projects are here - -__-
November 2025 • 30-Hour Offline Hackathon
Built an autonomous AI security agent for verifying Real-World Assets tokenized on the Cardano blockchain. Contributed to the AI fraud detection pipeline, integrated anomaly detection with the blockchain workflow, and connected AI decisions to on-chain smart contract actions for an end-to-end verification system.
Tech Stack: Python • Node.js • Docker • Cardano • Hydra • Aiken • Masumi Protocol • MeshSDK
October 2025 • Online Shortlisting Round
Developed an AI-powered career guidance platform using Google Gemini to analyze resumes, recommend career paths, and generate personalized roadmaps. Built the FastAPI backend, integrated Gemini, designed the Firestore data model, implemented Firebase Authentication, and deployed the backend on Google Cloud Run.
Tech Stack: Python • FastAPI • React • Google Gemini API • Firebase • Firestore • Google Cloud Run
- Computer Vision & Perception:
Object Detection•Tiny Object Detection•Segmentation•Tracking•Classical Image Inpainting•Sensor Fusion (EO/IR) - Deep Learning & Model Deployment:
PyTorch•MMDetection•ONNX•TensorRT•MMDeploy•Model Optimization (Quantization, FP16)•Parameter-Efficient Fine-Tuning (PEFT, QLoRA) - Systems & Infrastructure:
Docker•Celery•Redis•C++/CUDA Inference Pipelines•Django•FastAPI - Languages & Tools:
Python•C++•OpenCV•NumPy•Git•Linux
B.Tech Computer Science & Technology (Artificial Intelligence & Machine Learning)
2022 – 2026 • CGPA: 8.23 / 10
Relevant Coursework: Computer Vision • Deep Learning • Machine Learning • Artificial Intelligence • Data Structures & Algorithms • Operating Systems • Database Management Systems • Computer Networks
- Google Cloud Skills Boost — Classify Images with TensorFlow on Google Cloud (Deprecated)
- Google Cloud Skills Boost — Explore Generative AI in Agent Platform
- Google Cloud Skills Boost — Future Ready Skills
- Google Cloud Skills Boost — Machine Learning Operations (MLOps) for Generative AI
- Google Cloud Skills Boost — Inspect Rich Documents with Gemini Multimodality and Multimodal RAG
- Google Cloud Skills Boost — Build Real World AI Applications with Gemini and Imagen
- Google Cloud Skills Boost — Prompt Design in Agent Platform
- NPTEL (IIT Madras) — Deep Learning for Natural Language Processing (Elite)
- Simplilearn — Machine Learning using Python: Supervised & Unsupervised Learning
Tarun Pandian M
Bengaluru, Karnataka, India
- Mail: mtarunp@gmail.com
- LinkedIn: linkedin.com/in/tarun-pandian-m
- GitHub: github.com/Krishna4311