Qu, Haoxuan and Li, Yanchao and Foo, Lin Geng and Kuen, Jason and Gu, Jiuxiang and Liu, Jun (2022) Improving the Reliability for Confidence Estimation. In: Computer Vision – ECCV 2022 - 17th European Conference, 2022, Proceedings :. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) . Springer Science and Business Media Deutschland GmbH, ISR, pp. 391-408. ISBN 9783031198113
Full text not available from this repository.Abstract
Confidence estimation, a task that aims to evaluate the trustworthiness of the model’s prediction output during deployment, has received lots of research attention recently, due to its importance for the safe deployment of deep models. Previous works have outlined two important qualities that a reliable confidence estimation model should possess, i.e., the ability to perform well under label imbalance and the ability to handle various out-of-distribution data inputs. In this work, we propose a meta-learning framework that can simultaneously improve upon both qualities in a confidence estimation model. Specifically, we first construct virtual training and testing sets with some intentionally designed distribution differences between them. Our framework then uses the constructed sets to train the confidence estimation model through a virtual training and testing scheme leading it to learn knowledge that generalizes to diverse distributions. We show the effectiveness of our framework on both monocular depth estimation and image classification.