Remember that anything vector you can get into SVG. Adobe software is great at this. There could be vector stuff in a PDF - just open the PDF and you'll be able to get your hands on the vector parts. As we saw, stock photography sites often have things in EPS format, no problem, open up in Illustrator and get those vectors and save as SVG.
Icon sets are a great source here. Whether they are "icon fonts" or not, all that matters if you are going to be using your own custom SVG is that they come in a vector format, and they pretty much all do. Here's a huge list.
Even if the icon set only comes as a font, remember you can just install and activate that font. Then in Illustrator, type the character with the symbol you want, make sure the font is set to that icon font, and you'll get that symbol. Then "Convert to Outlines" and you got the vector.