I'm going along the same lines as Joatmon. It seems more as if your blower motor is intermittent? If the motor stops working and you can smack the dash and it comes on, it's the motor on it's way out. If your motor only works on high speed, it's more likely the resistor. Based on what you said though, I'd lean towards the motor being on it's way out.
It's an easy fix, take the glove box out, and the motor has three 5.5mm screws holding it into place, plus the drain hose. Make sure you reattach the drain hose when you replace it.