LMC : Large model collaboration with cross-assessment for training-free open-set object recognition

Qu, Haoxuan and Hui, Xiaofei and Cai, Yujun and Liu, Jun (2023) LMC : Large model collaboration with cross-assessment for training-free open-set object recognition. Advances in Neural Information Processing Systems, 36. ISSN 1049-5258

Text (NeurIPS-2023-lmc-large-model-collaboration-with-cross-assessment-for-training-free-open-set-object-recognition-Paper-Conference)
NeurIPS-2023-lmc-large-model-collaboration-with-cross-assessment-for-training-free-open-set-object-recognition-Paper-Conference.pdf - Accepted Version
Available under License Creative Commons Attribution.
Download (12MB)

Abstract

Open-set object recognition aims to identify if an object is from a class that has been encountered during training or not. To perform open-set object recognition accurately, a key challenge is how to reduce the reliance on spurious-discriminative features. In this paper, motivated by that different large models pre-trained through different paradigms can possess very rich while distinct implicit knowledge, we propose a novel framework named Large Model Collaboration (LMC) to tackle the above challenge via collaborating different off-the-shelf large models in a training-free manner. Moreover, we also incorporate the proposed framework with several novel designs to effectively extract implicit knowledge from large models. Extensive experiments demonstrate the efficacy of our proposed framework. Code is available \href{https://github.com/Harryqu123/LMC}{here}.