The root of the trick is that borders on elements meet each other at angles. So if one is transparent and the other isn't, it appears as if the whole shape is a triangle. You can turn "on and off" different borders and use different heights/widths to get different triangle shapes pointing different directions.
Of course you can get all kinds of crazy with CSS and only one element. The triangle is just always particularly compelling because it's so practical.