Turn off both computers and then take out the power supply to the router. Leave for a minute and re-attach the power supply to the router. Wait 3 minutes and check that all the LED`s on the router are static and working.
Reboot each computer in turn, and make sure that each one reconnects to the Internet before rebooting the next one. Post back with any problems.