This has to be a Power supply problem.
Not necessarily the PSU itself, it could be the switch itself, since you have replaced the PSU.
These days, the on/off switch like the reboot one merely uses a low voltage to toggle a transistorised switch which governs another circuit.
It is possible that the PSU is not quite there!... So what can you do?....... Like all intermittent faults they are a pain to diagnose.
If the old PSU has a chance of further life you could try it for a few days, or better still try a substitute On/Off switch.
See what others say.