Got it fixed. Was rather simple. Just put the separates SVG’s on top with absolute position on the img tag. Then applied the same dimensions to make sure it would stay relative when the parent would scale.
Now I only need to animate the color of the area.