UFO2: A Unified Framework towards Omni-supervised Object Detection

Zhongzheng Ren    Zhiding Yu    Xiaodong Yang   
Ming-Yu Liu    Alexander G. Schwing    Jan Kautz   

UIUC       NVIDIA     

European Conference on Computer Vision (ECCV), 2020

Paper | Slides | Bibtex


Existing work on object detection often relies on a single form of annotation: the model is trained using either accurate yet costly bounding boxes or cheaper but less expressive image-level tags. However, real-world annotations are often diverse in form, which challenges these existing works. In this paper, we present UFO2, a unified object detection framework that can handle different forms of supervision simultaneously. Specifically, UFO2 incorporates strong supervision (e.g., boxes), various forms of partial supervision (e.g., class tags, points, and scribbles), and unlabeled data. Through rigorous evaluations, we demonstrate that each form of label can be utilized to either train a model from scratch or to further improve a pre-trained model. We also use UFO2 to investigate budget-aware omni-supervised learning, i.e., various annotation policies are studied under a fixed annotation budget: we show that competitive performance needs no strong labels for all data. Finally, we demonstrate the generalization of UFO2, detecting more than 1,000 different objects without bounding box annotations.

paper thumbnail




Z. Ren, Z. Yu, X. Yang, M.-Y. Liu, A. G. Schwing, J. Kautz.
UFO2: A Unified Framework towards Omni-supervised Object Detection.
ECCV 2020. Bibtex

ECCV video

Related Work

Z. Ren, Z. Yu, X. Yang, M.-Y. Liu, Y. J. Lee, A. G. Schwing, J. Kautz. Instance-aware, Context-focused, and Memory-efficient Weakly Supervised Object Detection. CVPR 2020.


ZR is supported by Yunni & Maxine Pao Memorial Fellowship. This work is supported in part by NSF under Grant No. 1718221 and MRI #1725729, UIUC, Samsung, 3M, Cisco Systems Inc. (Gift Award CG 1377144) and Adobe.

Thanks the SPADE folks for this beautiful webpage.