To Trust Or Not To Trust Your Vision-Language Model's Prediction - View it on GitHub