Yes, I use a modem router with 4 ports which allow me to connect up to 4 comps simultaneousely. Typically, the Netgear DG814 is very good. If you also want to share files between the comps you'll have to network.For the web it is just a matter of plugging them in.
If you only have two computers you can network them with an ethernet card (pci slot) in each of them and a cross-over cable connecting them. One of your machines will then be connected to the ADSL modem and be on line. The other will use that one as an internet gateway. Only drawback is that the gateway pc has to be switched on for the other to go on line. If that would be a problem then you would need to use a router as suggested by rickf.