It was the FUSE! I took a quick look at it before and didn't see anything burnt or broken. Just got a test light to it and yup, it was broken. I do hear a sound from the blower motor that sounds like it's blowing on paper though. Never noticed it before.