I hear what you’re saying, and you do have a point, a valid one at that, but i still think it’s worth it in most cases (when it’s done well). Take this site for example.. would you rather scroll horizontally when browsing the forums? What if you want to resize your browser window to half the width, or a third of the width of your screen, and occupy the remaining space with something else? I think rearranging content to fit is far superior than having to scroll horizontally to read a paragraph.
When it comes to less technically inclined users and their possible frustration with a change in layout I think we can’t really win no matter what we do. There will be cases both for and against, but I think those who are for might be in bigger numbers then you might think. If my mom was to visit some site on say, an android device, I think she’d have an easier time navigating it if everything just fell exactly “into place” as opposed to having to scroll around the site, zoom in here, zoom in there etc.
I think responsive designs are here to stay. When they are done right, things should be apparent to the user. Navigation should be easy and the same content should be available just as it would be on a larger screen.