Height is king.
Noise floor seems to be less of a problem in UHF, even at <1 mile from a big 1400' angry RF firebreathing tower, the measured RSSI noise floor in the GMRS channels is always in the high -120dBm range.. that's pretty good.
The issue with UHF is the 10dB attenuation loss over VHF in long range distances, and the fact that anything it touches attenuates the signal like its going out of style... rain? boom, range goes to crap, snow? boom, range goes to crap, vegetation nearby? boom, there goes some more range... The only thing, like you've stated, that will will fix this is more height. And a good antenna setup.
G.