Never tried it tbh, but it sounds like an interesting idea.
Depending on the size of the hexes I guess you could try and do some sort of equivalence where 1 hex = 3", and then everything goes from there. The whole board would be 16x16 hexes with deployment zones being 4 hexes deep, movement being 2 hexes for Advance actions and 4 hexes for Rush actions, an assault rifle having an 8 hex range, etc.
If you are playing with much bigger hexes everything changes. What mat do you have? Pictures?
I have some experience with hex and square games. I love the idea of clearing up LoS issues. Math can tell you the line of sight from one hex (or square) to another and you don't have the issues of arguing over how terrain impacts the shot.
I'd go with hexes for sure (squares create this weird either square or diamond firing arc) and I'd go with 2" flat-to-flat. 40mm bases will still fit in that space, and you'd know that anyone in the area surrounding a model could still reach them for close combat. Also, weapon ranges could be pretty easily halved.
As far as terrain goes, look for BattleTech terrain. That game is classically played on a hex mat and there are lots of cool looking terrain options for it.