Would you have allowed the batter to hit the ball that caromed off the backstop?
Take this play. Pitch is as you described, way high and wide, hits the backstop. Runner takes off from 3B trying to score on the wild pitch. As it caroms off the backstop, catcher is trying to pick it up to tag the runner. The batter, still in the box, takes a swing at the now rolling ball, whacking the catcher, and the runner scores. Are you going to call catcher's obstruction (this is FED, catcher's interference everywhere else)?
Once the ball hits the backstop, I'm no longer considering it a pitch.