Not to convolute the scenario, but the opposite situation would be a foul:
Designed pass play, the O-line is supposed to "stay at home", but A56 accidentally goes 5 yards downfield during the scramble, realizes his mistake and retreats behind the LOS before the ball is thrown. Once an ineligible is downfield, this will be a foul if a pass crosses the LOS regardless of their position at the time of the pass.
However, if the pass did not cross the LOS, it's still not a foul for the same reason as the original play.