Until the opponent reaches the same yard line as the blocker, the opponent is to be considered a potential blocker. As such, in addition to the blocking action described above, the defender may use his hands/arms to grasp/encircle the opponent IN AN EFFORT TO AVOID THE BLOCK OR MOVE THE OPPONENT OUT OF THE WAY. However, once the opponent has reached the same yard line as the opponent, the defender may NOT use his hands/arms to grasp/encircle the opponent and obstruct/restrict him.
I don't think this is quite as cut and dry as you're making it out to be. In general, I agree that we should err on the side of "A is a potential blocker until on the same yard line as the defender", but as noted in the thread linked earlier, Case Play 9.2.3.A points out that if A is not attempting to block or is moving away from the defender, he is also not a potential blocker and contact by B would be a foul.
If receiver A is running a straight route down field and B comes across the field (say B was a linebacker and not head up on A at the snap) to block A and just knocks him down while A was not running towards B or looking like he was trying to block B, I would call that a foul even if A wasn't "even up" on B by yard line.
If B was directly in front of A at the snap and A just runs in B's general direction, then I'd definitely say A is a potential blocker even if he doesn't make a blocking motion with his hands. B is allowed to exist on the field which forces A to alter his route, including using his hands to push A out of bounds.