First off: this is actually a neat little plugin you’re working on ;-)
Now, on topic:
Maybe an easy way is to put the little menu inside the anchor, this will at least fix the issue for “regular” browsers, as you’ll still be hovering over the anchor while you’re hovering over the menu.
Now, when it comes to other devices that use touch, it might be better to use jQuery and listen for ‘click’ events (I don’t think you have to listen to native touch events, afaik most (all?) mobile browsers send a click event as well).
So a little mockup:
It’s really rough and you’d probably want it to behave a little different, but what it does is: when you click the link (you can also hover it on desktop) it will show the menu, then when you click the link again, it will go to that link regardless.
Again: you’d need to change the code here and there and do some decent testing, but it should get you on the right track (hopefully) ;-)