What’s in the box? Explaining the black-box model through an evaluation
  of its interpretable features