weather generally shouldn't affect your DSL... it runs through a phone line... however if the insulation is breached (tree root hitting it, worm or something chewing through it) then rain can cause an issue ....
I'm guessing you've gotten DSL within the past 3 years or so? (so it's a bundled loop... piggy backs w/your voice/phone line)
Do you have all the filters installed? when you notice the slower response do you hear any static on your voice line (could indicate a breach in the insulation for the phone line)