Imagine situation. 1st and goal near goal-line of defense.
Offense runs toss play (backward pass). Ball is muffed by halfback.
Defensive player penetrates from the edge and tries to gain possession of loose ball.
While struggling to recover ball defensive player accidently gives impetus to it going into own end-zone and out of bounds.
What is the call?
The key here is impetus, and “muffing” the ball, as defined, does not impart impetus, UNLESS the ball had come to complete rest. By your description, it sounds like the ball was still moving, thus, no new impetus. So, the impetus is still from the backward pass. If the ball travels out of bounds behind B’s goal line with no new impetus on it, the result is a touchback, B, 1/10, B-20.
Now, if the ball had, in fact come to complete rest, then is muffed by a Team B player, the muff imparts the impetus on the ball, and, if it travels OB behind B’s goal, the result is a safety, 2-points for Team A, with a free kick by B to follow.