It is. Unless your code is doing something to the cursor that is making it not the default when you need it to be, adding that extra bit of code is pointless. It slows down the the loading of the page. It’s best to use as little code as possible. Don’t use any code that is unnecessary when you can help it.
Most of the time you will only need to set a cursor when you need to give more user feedback when creating your websites user interface. If you are doing something in jQuery for example, and your listening for something to be clicked that isn’t by default a link its always good to change the cursor to show it has a "click-able" action. Changing the cursor otherwise is a little (no pun intended) pointless. So don’t do it as a gimmick, because that’s what people will see.
As for loading speed, it will make no difference. Unless your machine is a 486 windows 3.11 beast with 2mb of ram – and even then I doubt you would notice, so the speed issue is redundant.