Like most things, I will let the game dictate what I do, but most of the time, I will stand over the ball until the offense breaks the huddle. If the tempo of the game dictates, I will leave earlier- when the R blows RFP, etc.
As Jason said, I also don't turn my back on the ball, but back out.