Cascade Framework 2 is under development and involves a rewrite of the codebase to Sass. This will allow you to use the framework in a completely semantic way and select on a per-feature basis what you need.
Still, for a project with a custom look-and-feel, you shouldn’t be using more than 5kb (Gzipped) of Cascade Framework’s CSS code today. That includes IE6+ support, responsiveness and default typography. And then there’s always Cascade Framework Light, which is only 2kb (Gzipped). I would hardly call that bloated.
Given that (nearly?) all browsers now resize the page as a whole, rather than just the text, previous issues with PX vs. % vs EM in terms of accessible font resizing are rather moot. I definitely don’t see any impact on using PX for media queries. Zooming in an out works fine on all browsers I tested.
With regards to font sizes, I decided to use a fixed font size for the body and % based font sizes elsewhere. Old IE doesn’t like % or EM for the body’s font size. Using % everywhere else offers about the same advantages as EM.