- To the best of our knowledge, UniDrive presents the first comprehensive framework designed to generalize vision-centric 3D perception models across diverse camera configurations.
- We introduce a novel strategy that transforms images into a unified virtual camera space, enhancing robustness to camera parameter variations.
- We propose a virtual configuration optimization strategy that minimizes projection error, improving model generalization with minimal performance degradation.
- We contribute a systematic data generation platform along with a 160,000 frames multi-camera dataset, and benchmark evaluating perception models across varying camera configurations.
UniDrive : Towards Universal Driving Perception Across Camera Configurations
|
|
1University of Michigan, Ann Arbor 2UC Berkeley |
*Work done while visiting UC Berkeley, ✉Corresponding author |
Overview
UniDrive is a novel framework designed to address the challenge of generalizing perception models across multi-camera configurations.
Motivation
Existing driving perception models susceptible to sensor configuration (e.g., camera intrinsics and extrinsics) variations. However, generalizing across camera configurations is important for deploying autonomous driving models on different car models.
Our Pipeline
We deploy a set of unified virtual cameras and propose a ground-aware projection method to effectively transform the original images into these unified virtual views. We further propose a virtual configuration optimization method by minimizing the expected projection error between original cameras and virtual cameras. The proposed virtual camera projection can be applied to existing 3D perception methods as a plug-and-play module to mitigate the challenges posed by camera parameter variability, resulting in more adaptable and reliable driving perception models.
Experiment Results
Visualizations of Different Camera Configurations
BEVFusion Fails to Generalize Across Different Configurations
UniDrive is Robust to Sensor Variations
Virtual Configuration Optimization Further Improves Performance
Effect of Different Configuration Factors
