The NCAA rule for A stepping out of bounds is logical. A player of A who steps out on his own (i.e. is not forced out) cannot legally catch a pass, but is otherwise allowed to keep playing. If he does catch the pass, the penalty is a loss of down from the previous spot.
On a kick play, the penalty for going out and coming in is 5 yards from the previous spot, or a 5-yard penalty from the "spot of the subsequent dead ball" at the option of B.