If the battery is on its way out, it just wouldnt retain info such as date/time, but should still boot.
To try and narrow it down a bit, disconnect everything none essential, such as cd drives etc, leaving just mobo, ram, gfx card, hdd, fdd etc.
I noticed that you said fdd light came on (permanently? if so then the ribbon cable is probably on the wrong way round.)
It does sound like as you suspect, overheating or psu. But just to be on the safe side, remove and reseat everything, cpu, ram, gfx card etc. From personal experience I know that ram if its not perfectly seated can have you banging your head on the desk for hours, until you discover the ram wasn't quite seated properly.