Icon fonts are awesome. Other than the fact that they have to be single color, they are superior to using images as icons in every way. But which do you choose? There are loads of different sets out there. I'm going to attempt to round them all up here and keep this updated (this post has already been updated several times).
Pictos is the icon font that really popularized this whole idea. In additional to the sets, they also offer Pictos Server, a service which allows you to build your own font set from its library of icons and will host/serve it to you. This allows for very small (quick loading) font files and a nice UI for updating your current sets.
IcoMoon is a set of open source icons. But more importantly, IcoMoon is a web app for customizing and downloading optimized icon fonts. You choose just the icons you want and download that minimized set. You can easily add font from different sets or add icons from SVG files via simple drag-and-drop.
The icons as graphics only are priced differently, but the fonts are broken up into 8 sets. Seems like kind of a pain in the ass to buy just the right sets you need since they are just chopped up to be even numbers. If you love the set, you're probably better off buying one of the bundle packs.
The Noun Project is probably the largest collection of icons out there. It's not really a "set" but rather a huge archive of symbols. You can download them one-by-one for free if you agree to attribute the author, or buy the symbol to get it royalty-free. You'll have to combine them all together to use as a font. You can submit your own.