Universal Object Tracking & Digitization
Automated digital twin pipeline with high-accuracy object detection for Boston Dynamics Spot.
The Problem
I engineered the solution using ROS 2, Docker, Python, PyTorch, OptiTrack, and YOLOv8 to ensure robust multi-agent coordination. My pipeline begins with scanning the object and attaching IR markers for OptiTrack localization. I built a bridge that consumes NatNet packets from OptiTrack and publishes them to ROS 2 as TF frames and JSON metadata, effectively creating a real-time digital twin for any arbitrary object. To ensure accessibility for other researchers, I containerized the entire pipeline using Docker, enabling a single-command installation that handles all dependencies automatically. Extending this capability, I enabled the lab's Boston Dynamics Spot robot to autonomously identify these custom objects by fine-tuning Meta's SAM2 and YOLOv8 models, achieving 92.8% detection accuracy on lab-specific datasets.
Approach
The core of the solution involves a custom pipeline built with ROS 2, Docker, Python, PyTorch, OptiTrack, YOLOv8. I prioritized modularity and performance, ensuring the system can run in real-time constraints.
Results
