Fonts are fun and addictive. I don't think anyone can have too many of them, but that's just me. I can't give you any advice on Mac fonts since I've never owned a Mac. With PC fonts, and newer versions of Windows, you do not need to download a font viewer or organizer. ACDSee and Windows will preview your fonts for you. To install them, it's easy, after you download your fonts, unzip them and copy and paste font file into your Windows font folder. If you do this while your programs that use fonts are running, you'll need to refresh it's font database from the program or restart it (Gimp has a refresh button, Photoshop Elements it's easier to just restart it as occasionally a few fonts won't appear right away).
If a font you found isn't exactly what you wanted, keep in mind you might be able to alter it to work - by scaling the layer it's on to be taller or wider or using brushes to grunge it up or add flourishes. I've addressed this a bit on this blog already. As you learn to use your program, you'll find out that you are not limited by what fonts you are given.
Another option, some font sites have a feature where they will show you similar fonts so you find the one you are looking for or one that comes closest.
If you want to learn about font file types and other details, most font sites have a section available, and of course, there's Google.
[EDIT TO ADD: Be careful of stretching and scaling fonts as you might end up pixelating them or making them too distorted. To prevent this, make your font much larger than you need, edit, then scale down. See if that helps.]