Explanations for black-box models help us to understand model decisions as well as provide information on model biases and inconsistencies. Most of the current post-hoc explainability techniques provide a single level of explanation, often in terms of feature importance scores or feature attention maps in the input space. The explanations provided by these methods are also sensitive to perturbations in the input space. Our focus is on explaining deep discriminative models for images at multiple levels of abstraction, from fine-grained to fully abstract explanations. We use the natural properties of hyperbolic geometry to more efficiently model a hierarchical relationship of symbolic features with decreased distortion to generate robust hierarchical explanations. Specifically, we distill the underpinning knowledge in an image classifier by quantising the continuous latent space to form hyperbolic symbols and learn the relations between these symbols in a hierarchical manner to induce a knowledge tree. We traverse the tree to extract hierarchical explanations in terms of chains of symbols and their corresponding visual semantics.