You need to unplug your computer before you take the panels off or mess around with the back of it.
However, I'm curious about this too. All three of my ATX systems do this as well, even when the ATX power switch on the back is off. I assumed it was a grounding problem with the local electricity here. All electrical outlets here have only the 2 holes, no ground.