We dug into overriding default buttons styles not long ago here on CSS-Tricks. With garden-variety fully cross-browser-supported styles, you're looking at 6-10 CSS rules to tear down anything you need to off a button and then put in place your own styles. Hardly a big deal if you ask me, especially since it's extremely likely you'll be styling buttons anyway.
Scott O'Hara has taken a look as well including a couple of interesting other CSS explorations, neither of which stacked up for different reasons:
display: contents; - some semantics-based accessibility problems.
all: unset; - doesn't reset display value, not good enough browser support.
It took me a little bit to wrap my head around this, including having a little chat with Scott over DMs. I was so confused that the correct way to do what he was laying out was <span role="button" tabindex="0" onClick="...">. What?! I thought. It's almost never the correct thing to do to force a non-button element to behave like a button and have to replicate all the button functionality in other ways. And indeed, if you just have like a word-or-two kinda button, you probably don't need to, and can just un-do button styles. The trouble comes in here: you cannot un-do inline-block on a <button>. Apparently, it's just impossible. You can set it to display: inline; and it will take it, but it won't behave like it.