ULDepth: Unpaired Multi-Domain Learning for Robust Self-Supervised Depth Estimation
Monocular depth estimation (MDE), which aims to predict pixel-wise depth maps from images only, plays a crucial role in 3D scene understanding, robotics, autonomous driving, and augmented reality. Self-supervised MDE has significantly reduced the reliance on costly labeled depth datasets by learning from unlabeled consecutive images, while maintaining competitive performance compared with supervised methods.
However, for real-world deployment, depth estimation models need to generalize across multiple domains, such as varying lighting conditions and weather scenarios. Current models often fall short in this regard. Single-domain models are typically optimized for specific conditions, such as day or night only, which limits their applicability.
Meanwhile, existing multi-domain approaches often rely on paired images from different domains, for example, images of the same scene captured in both day and night. Since such paired data are generally unavailable in real datasets and real-world scenarios, they are often synthesized using style transfer, which can introduce unexpected artifacts and restrict performance. Figure 1 summarizes this motivation and illustrates the transition from single-domain or paired-input self-supervised MDE to the proposed unpaired multi-domain setting.
To address these limitations, ULDepth proposes a general plug-in framework that transforms existing self-supervised MDE models into an unpaired multi-domain depth estimation framework. As shown in Figure 2, ULDepth can be applied to both single-domain depth estimation baselines and paired-domain depth estimation baselines, enabling them to learn from unpaired real-world images collected across diverse scenes and domains. This design eliminates the reliance on image style transfer for synthesizing paired cross-domain images and avoids the associated artifacts.
The key idea of ULDepth is to use daytime images as a reference to guide the model in learning consistent depth distributions across diverse domains through adversarial training. The architecture introduces three plug-in modules: a Dynamic Normalization Module, a Pseudo-Depth Network, and a Discriminator. The Pseudo-Depth Network provides high-quality daytime depth estimation as a reliable reference, while the Discriminator aligns the depth distributions between daytime images and challenging domains such as nighttime, rain, and snow.
Moreover, the Dynamic Normalization Module supports shared feature extraction across different domains, removing the need for separate domain-specific encoders. ULDepth also augments the discriminator with positional encoding combined with predicted depth maps, allowing the model to refine regions prone to artifacts. Through this architecture, ULDepth enables existing self-supervised MDE baselines to learn robust depth representations from unpaired real-world images across multiple domains without relying on synthetically generated paired data.
Figure 3 visualizes the qualitative results on the Oxford RobotCar [1] multi-domain dataset, covering diverse driving conditions including day, day rain, snow, sunny, night, and night rain. Among these domains, night and night rain are particularly challenging due to severe lighting variations, rain reflections, and nighttime noise. Despite these difficulties, ULDepth produces high-quality and consistent depth predictions, demonstrating its robustness and strong generalization ability across diverse real-world conditions.
