First off, I can't stress enough about saving as you go as well as saving in layers (xcf, psd). As a reminder, I always use the word "edit" in my saved title and sometimes I add a number to designate the edited version, so I know that this image isn't finished and should not be shared. Saving these can also help prove yourself in a copyright case, as I had to once before. Some projects you'll find you save more edits than others. Now that digital storage (thumb drives, CD's, DVD's, external hard drives) is getting cheaper and cheaper, it doesn't hurt to keep these around.
When should you save an edited version? That depends on you and your project. If you think you'll want to create another version of this but with certain changes, save just before you go in one direction, such as erasing and recoloring. I always keep the bottom layer as my unedited base image in case I need a quick copy. It's usually called "background" by default.
When you first install Gimp, it has a default setting for undo levels. This setting is inadequate, especially once you apply a script-fu. So if you have ever wanted to undo something but can only go so far, there's a fix for this. Go into the main Gimp box and click on File> Preferences> Environment. I have my settings at 99 undo levels and 100 megabytes of undo memory. This seems to be fine for what I do and for my system resources. Even when I was running W98, 1 gig ram with 14 gigs of memory, it seemed to work just fine.
Now when you make a mistake you should be able to back yourself up through your undo levels or simply by opening up a saved edit.