Towards Open World Object Detection

Joseph, K J and Khan, Salman and Khan, Fahad Shahbaz and Balasubramanian, Vineeth N (2021) Towards Open World Object Detection. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2021, 19 June 2021through 25 June 2021, Virtual, Online.

[img] Text
Proceedings.pdf - Published Version
Available under License Creative Commons Attribution.

Download (2MB)


Humans have a natural instinct to identify unknown object instances in their environments. The intrinsic curiosity about these unknown instances aids in learning about them, when the corresponding knowledge is eventually available. This motivates us to propose a novel computer vision problem called: 'Open World Object Detection', where a model is tasked to: 1) identify objects that have not been introduced to it as 'unknown', without explicit supervision to do so, and 2) incrementally learn these identified unknown categories without forgetting previously learned classes, when the corresponding labels are progressively received. We formulate the problem, introduce a strong evaluation protocol and provide a novel solution, which we call ORE: Open World Object Detector, based on contrastive clustering and energy based unknown identification. Our experimental evaluation and ablation studies analyse the efficacy of ORE in achieving Open World objectives. As an interesting by-product, we find that identifying and characterising unknown instances helps to reduce confusion in an incremental object detection setting, where we achieve state-of-the-art performance, with no extra methodological effort. We hope that our work will attract further research into this newly identified, yet crucial research direction. © 2021 IEEE

[error in script]
IITH Creators:
IITH CreatorsORCiD
Balasubramanian, Vineeth N
Item Type: Conference or Workshop Item (Paper)
Additional Information: The vibrant object detection community has pushed the performance benchmarks on standard datasets by a large margin. The closed-set nature of these datasets and evaluation protocols, hampers further progress. We introduce Open World Object Detection, where the object detector is able to label an unknown object as unknown and gradually learn the unknown as the model gets exposed to new labels. Our key novelties include an energy-based classifier for unknown detection and a contrastive clustering approach for open world learning. We hope that our work will kindle further research along this important and open direction. Acknowledgements We thank TCS for supporting KJJ through its PhD fellowship; MBZUAI for a start-up grant; VR starting grant (2016-05543) and DST, Govt of India, for partly supporting this work through IMPRINT program (IMP/2019/000250). We thank our anonymous reviewers for their valuable feedback.
Uncontrolled Keywords: Clusterings; Computer vision problems; Energy-based; Evaluation protocol; Learn+; Novel solutions; Object detectors; Objects detection; Open world; Unknown objects
Subjects: Computer science
Divisions: Department of Computer Science & Engineering
Depositing User: . LibTrainee 2021
Date Deposited: 06 Oct 2022 10:16
Last Modified: 06 Oct 2022 10:16
Publisher URL:
Related URLs:

Actions (login required)

View Item View Item
Statistics for RAIITH ePrint 10814 Statistics for this ePrint Item