If you want to get an inexpensive Mustang GT then get the 99-04 New Edge ones like you already have planned. As previously said, they run the PI (performance improved) heads. The 96-98 GT's came with 215hp whereas the 99-04's came with 260hp in stock form. The stock bottom end of the 96-04 GT's will reliably hold about 450rwhp before giving up the ghost depending on the tune. Most people will usually make around 400rwhp-425rwhp with them using a blower with supporting mods at around 9psi. There are many blower and turbo kit options for those cars ranging from Kenne Bell, Whipple, Vortech, Procharger, Paxton, Hellion, THP, HP, etc, etc, etc... Nitrous is also a valid option as well. Usually with full boltons those cars run bottom 13's and with cams will run middle to high 12's on average with a solid tune. There are allot of Tom, Dick, and Harry's running around making crap for power and ET's with those cars because they usually haven't taken the time to really set them up right; whether that be actually degreeing the cams when installing, getting a proper tune, learning how to drive and so on. The cars are overall pretty reliable and drivetrain can take a little power. The stock M5 transmission is a bit of a weak link but unless you start throwing forced induction of some form then you are unlikely to encounter those issues.
The popular initial bolton mods for these cars are:
CAI
Exhaust
Tune
Shifter (MGW and Pro5.0 are popular ones amongst others)
Underdrive pullies
Popular intermediate mods for these cars are:
LT headers
Upper intake elbow
Gears (usually 3.73's or 4.10's, with most going 4.10)
Stickier tires
Clutch
Cams
Subframe connectors
Torque arm
They are pretty decent cars and with the proper exhaust can sound pretty mean too. If you want to locate some good deals on used parts for those cars then SVTPerformance, Modularfords, and Corral are some good forums for that. I have scored good deals on used parts on those sites for years.