I assume you are checking it in Internet Explorer, where it is a little off.
The reason is that it’s positioned there, is because that’s where it’s positioned if you wouldn’t rotate it. So, what actually happens is that you first position it all the way to the right, and then you rotate it 90 degrees using the button’s LEFT side as the rotation point.
Other browsers, who support full CSS rotation, rotate with the center as the rotation point.
I would advise to not use the rotation thingie, but just create a vertical button and position it without rotating. That way you won’t have to deal with the different ways browsers rotate elements.