I figured it out, Lol! My brain hurts though. Anyways the big clue came from Paulie_D. Adding and removing the elements was definitely not the best option. That got me on the right track to just manipulate the color changes through a function and then call it with
setInterval. Now the code does exactly what I wanted. Thx for the tips everyone.