Most computers will give a short beep at startup to indicate that the motherboard Power On Self Test (POST) has completed satisfactory with no errors found. However as your computer goes on to load Windows I would guess that there is no problem and your computer is one of the exceptions. The fact that your computer restarts OK makes me wonder if this is a hard drive slow to get up to speed. You can test this if your computer BIOS has an option to disable "Quick Power On Self Test", probably in a "Boot" sub menu.
Another possibility that occurs to me is that your keyboard and mouse aren't PS/2, and the drivers aren't loading quickly enough.