>I am using nested Divs here to try and make the effect
To be honest this sounds more like a structure problem than anything else.
Is there a reason you couldn’t make the large image a background image instead of being inline. I think this would simplify things greatly.
I’d like to see an link to the site or a reduced case in Codepen