I find that the 4870 seems to prefer running at high temperatures (normally around 77 degrees if I leave the fan on auto) and the fact that it is this high in turn makes the rest of the inside of my case rather warm so I just manually ramp up the fan in the driver options and everything is fine (if a little noisy).
You shouldn't need to.
By biggest point would be unlike the crap cooler on the 8800GT Ati's stock coolers will come in and do their job if they need to. They use a little more power as a result but I'd say it is worth it. There is also very little benefit to pumping the GPU fan up, it shouldn't be running that hot, the fact that it is makes me think that the rest of the cooling in your case is inefficient or not doing it's job correctly.
If you were to go to water cooling extents you'd need a new case a new at least and a spare £250+ to do it well. Good Air is better than mediocre and ok water. So don't even consider it.
Good air cooling comes down to a few basic principles. The first is case layout. You need a good airflow, if you look at any modern cases the thing they push more than anything is the airflow and the way the case is laid out. Ideally you want to separate component areas so that you can remove heat in certain areas allowing you control how the air is moving.
Now assuming you are keeping your beige monstrosity this isn't very helpful but I thought while making the post it'd be useful to write it all up for you.
The next thing is picking the right fans and fan direction. You could spend £60 on a decent CPU heatsink but if there is no airflow in and out then it is largely redundant. The heatsink while just circulate hot air. At a guess with your case you'll probably have an exhaust at the back and possibly one inlet in the front, whether or not there is a fan there is a different matter.
My advice would be to get a couple of decent case fans one as an in and one as an exhaust. Then also get a non stock CPU cooler.
Another important thing to look at is dust and to ensure that the case is a clear and clean as it can be. A build up of dust in CPU coolers or GPU coolers especially in small scale coolers can cause DEATH to components. As well as keeping everything clean/clear is also important get some bog standard cable ties and keep your case as clear as you can.
A few other things for general cooling magic. Good mobos will have the ability to control CPU and other case fans automagically. It can also be worth getting a fan controller if you have more fans or a better case that allows you more flexibility with your fan layouts/fan speeds.
Right I meant to add a section on things I'd suggest and then forgot to do so. As Haven has said below. For CPU coolers good thermal paste is a must. You are more than likely to have 120mm or 92mm fan mountings. As Haven says big and slow is better than small and fast. 120mm are a good size however and there are some very good low noise/good throughput fans. Sharkoon, Scythe and Noctua are all good fans, for good throughput good RPM and not too loud.
I'd not bother with a GPU fan replacement, the stock coolers should be fine as long as you don't OC it and the rest of the case cooling is good.
CPU cooler wise for a cheap but not stock alternative the arctic cooling freezer 7 at ~£15 are very good for the money.
Otherwise for a fair bit more the larger scale coolers such as the Noctuas, the tuniq towers and the thermalrights are very good but not worth the money if you are leaving components at stock speeds.
