Possibly a failing power suppy component or ditto mother board.
The reason I conclude this is.
The power button on the front of a PC[This is the one you refere to is it not?]
Ctivates an already 'live system' by sending a trigger voltage to the PSU to 'wake up'
that is the main power is already in the PSU standing by.
So I conclude that the time lage colud be due to a capacitor or similar is taking its to 'come up'
All of this is supposition with out haviing the machine infront of me.
What can you do?
Remove the power cord from the back of the machine.
Wait 10 minutes.
Remove the cover.
Then with one hand of the metal chassis to equalize your self and touching no metal part with the other do as much as you can to ensure all connectors are firmly in place
espicially the 20 line connector from the power supply. but also all others for disks, memnory.