The power switch only gives a momentary connection while it is being pressed then the power supply should receive a 'power good' signal from the motherboard which keeps it running.
Have you tried disconnecting all periferals that aren't needed initially? Try booting with just the graphics card connected and 1 memory stick in the first blue slot. Disconnect hard drives, CD/DVD drives, floppy drive and especially any usb devices. You don't even need a keyboard or mouse initially. If it now starts to boot consistently, add things back in one by one starting with keyboard, then mouse, then floppy, then hard drive, etc. If it fails after an item is connected then you will know the cause.
My ASUS P5P800 failed to boot initially due to a USB card reader.