Briefly:
When I press the on button, the PSU starts working, and for a second the CPU fan and
VGA fan start to work too but then before any beeps or anything, all the fans go off.
So nothing except for the PSU fan stays on. And it can't be a problem with the PSU since I tried replacing the existing one with a stronger new PSU and still no luck.
What are the possible causes for this?

Details:
I was working with my system two days ago when all of a sudden it just went off.
As if the power suddenly failed. So I turned it on again and this time it got to
windows and my temperature monitor started warning me about the voltage to CPU being too low
(around 10.7 V) and then after a few seconds the graphics got all messed up.
I restart and this time POST tells me that overclocking has failed and that I
should reload default BIOS settings. I do that and restart and the computer is turned off before
it shows anything.
I thought that it's a problem with the Video Card fan.
So i took the VGA card out and cleaned the fan and put it back in and try to start it again
and this time I got the problem mentioned above. The fans run for less than a second and
before any beeps, everything except for the PSU goes off.

I thought it's a faulty PSU problem so I changed the power supply and installed a new fan on the CPU, but still the exact same problem.

What are the possible causes for this?

Should I try changing the MB and CPU? I'm clueless....

SYSTEM:
MB: Asus P5GD2
CPU: Intel 3.2Ghz P4
RAM: 1024MB DDR (2x512)
PSU: 470 Watts high quality

It could be that the psu isn't working right and isn't putting out the right voltage or the cpu is dead (if you were over clocking I would guess it's the cpu). You could try swapping out the psu (easer than swapping out the cpu) and then the cpu and see were that gets you.

Sylvander’s Diagnostic Flowcharts
Download a copy of my diagnostic flowcharts from here
www.erniek.eclipse.co.uk/downloads/sylvanderdiags.zip
and print them to leaf through.
Begin on the STARTUP chart.
-----------------------------------------------
NO POST, NO BEEP, NO VIDEO
www.pcguide.com/vb/showthread.php?t=39685
-----------------------------------------------
The CPU is responsible for switching on the PSU and then [having done that] the BIOS decides whether all's well with the hardware and it's ok to keep the PSU switched on and supplying power, and continue with the POST routine.
Looks like the BIOS decided you have a problem and switched off the PSU.
The fact that the PSU fan is still running means the system isn't "dead".
Try a bare-bones boot as above in NO POST, NO BEEP, NO VIDEO.