I use one all the time. It's way faster than trying to whip out a card and pen, find the box to write it down, then put everything back in my pocket, all while making sure the yardage was walked off correctly, and all before beginning the presnap routine.
"First quarter 9:56, white, offense, holding, 77, accepted, line judge."
Each recording gets saved to a separate file number, so I can just click forward and backward to jump between them. Always say the quarter and time first, so you know right away what part of the game you've jumped to.
Important feature: Mine has a sliding switch that acts like a safety, so I'm not constantly toggling the record button in my pocket while running.
My voice recorder also has four "folders" that you can jump between. I use folder 1 for first half fouls, folder 2 for second half fouls. Folders 3 and 4 are for anything I might want to remember to note on the postgame sheet, or things my supervisor might come asking about the next day.
My recorder is several years old so I don't have a specific model recommendation. Nothing on Amazon's first page of results seems to match the one I grabbed from Staples back then. Don't pay too much and don't worry about storage capacity - I don't even fill up 1% of mine. These things are designed for hours of recording, not twenty sentences.
I record every foul, not just my own. You never know when it will save the day filling out the postgame sheet.
I don't delete anything until the next week's pregame, just in case.