That PSU should be upgradable. It will screw out of the case. I have never seen one as a permanent attachment. Does the PC have any grapics card at all?
I would suggest a PCI version of the FX5200 or fx5600 as anything more is probably overkill on a PCI system such as yours.