I judge this similar to a false start - it depends on how the action occurs.
If before the snap, a player on the end of the line rises slowly, moves to the backfield and goes in motion - did he do this slowly enough, and did he not simulate movement like he does on a snap?
The same with the defense. If they're moving around, barking out defensive signals, fine. But if they are doing something (I know it when I see it type of 'something') which is meant to cause an offensive player to false start, the defense is getting a penalty.
Moving up to the line quickly as if to blitz is fine, one player on the line suddenly stomping his foot is NOT.
Barking out defensive calls is fine, but barking out sudden noises like 'hut...hut...hut.hut.hut' at the offensive line is NOT.
Youth games? I'm talking to the defense about what is/is not acceptable and giving a warning. Friday night? I'm throwing a flag - they know what they're doing, and I'm putting an end to it quickly.