I agree with the call. A.R 9-1-16-VI says that when it again becomes obvious that the ball will be kicked the team B players must avoid him. Here the team B player makes no effort to lessen the contact which occurs well after the punter has recovered the ball and showed that he will kick the ball.
Absolutely agree. The defender (apparently in the excitement of the moment) clearly went directly after the kicker, rather than the kick. The "Kicker" has long been recognized as the MOST vulnerable player in the game and is appropriately provided the maximum protection to avoid injury. Had the defender focused attention on the ball, and been able to "touch" the kicked ball, the call may have produced a completely different judgment.
The Referee was in the perfect position to observe the action and render his judgment.