that depends on several things. Firstly that the hard drive actually works. Secondly,,if the hard drive is formatted as NTFS and your machine is running 98 or ME then you wont be able to read it.
It none of the above are a concern then you have no problems. However, if you are using XP on your machine, there is a big chance it will ask if you want to create a dual boot menu so you can boot from either hard drive. If you say YES and boot from the old one, you will have to install all the hardware drivers for your system.