We often talk about interpretability as if it were an output: press a button, receive a saliency map, declare the model explained. But an explanation only becomes useful when someone can do something with it.
For a clinician, that might mean checking whether the model noticed the same anatomical structure they did. For a researcher, it might mean finding a shortcut hidden in the training data. The picture is not the explanation. The conversation it enables is.
An explanation should create a useful disagreement, not end the discussion.
What I now look for
I have started judging interpretability methods with three simple questions: Who is this for? What decision changes after seeing it? And can the person challenge what the model appears to believe?
Margin note: confidence without a mechanism for disagreement is just a more polished black box.
This makes the work less tidy, but more honest. A trustworthy system is not one that always looks right. It is one that gives people enough leverage to notice when it is wrong.