This could be caused by any number of things, keyboard and or mouse. Try another KB/Mouse. Power supply unit could be failing. These PC's use a very cheap PSU and will fail fairly easily, so could try another PSU. It could be a motherboard problem, which would mean, a new PC,probably. Could be a memory (RAM) issue.
Some things to try:
Power off, pull out power cable and the CMOS battery for 5 minutes, or short across the battery contacts.
Try with only one RAM stick in and see if it loads quicker.
Disconnect all drives except the OS, all USB devices except keyboard, mouse. In fact disconnect the kybrd and mouse too since we are testing to see if it loads quicker.