Perhaps you've heard about the WebP image format? And how it's a pretty good performance win, for the browsers that support it? Well that's only for Blink-based browsers, at the moment. Estelle Weyl's article Image Optimization explains the best image format for each browser:
Optimal image format
IE 9+ / Edge
And you can serve these formats through the <picture><source type=""> syntax.
Couple that complexity with the complexity of responsive images, and it really seems like outsourcing image delivery to a dedicated service seems like the way to go. At least above a certain scale.