I recently bought an XFX GeForce 6600GT AGP video card for my computer. It seems to be running alright, and handles anything I can throw at it just fine. However, according to NVIDIA's temperature monitoring software, the GPU core temperature is around 50 - 55°C idle and around 75 - 80°C in Half-Life 2 and Halo. Is this too hot? Am I at risk of damaging the card / computer at these temperatures?

In my case right now, I have one exhaust fan on the side of the case (although I don't think it's doing much since it's blocked by a lot of power cables) and the PSU fans. I am thinking I might buy another case fan for the rear to improve airflow in the system. If I do buy one, should I set it up for exhaust or air intake? Which would give the best results?

That's normal temp for a powerful videocard. Nvidia cards are always quite hot since the Fx5800 days. ;) My 6800NU runs at 56-60C and maybe even at 65-75C when doing intense gaming. The Nv40 core can handle max of 135C so don't worry about it. :D

Yep, those temps are indeed just fine for a GPU. Because of the architectural differences between CPU's and GPU's/VPU's, the vid chips can handle higher temps.

However, for a better cooling solution, add a rear fan for exhaust, and turn your side fan around for intake. And try to at least tame the cables so they're not directly blocking the side fan, even if you have to tie them to the floor of the case or something.