NFL Rule 9-1-3-2:
Item 2: Field Goal or Try Kick Formation. When Team A presents a field goal or Try Kick formation:
(1) A Team B player, who is within one yard of the line of scrimmage, must have his entire body outside the snapper’s shoulder
pads at the snap.
(2) No more than six Team B players may be on the line of scrimmage on either side of the snapper at the snap.
Penalty: For illegal formation by the defense: Loss of five yards.
Not sure which subsection was the violation on that particular play though, but the NFL does have some specific defensive formation requirements on kicks (free and scrimmage).