You've probably accepted a toolbar as a passenger in some other software installation. Some perfectly respectable software like CCleaner comes with addons, but you can usually deselect them when installing.
Try right clicking the IE taskbar, you will get a drop down list of active bars that should include the new arrival. You can untick it to make it disappear, but it'll still be installed. have a look in Add or Remove programs and if the relevant toolbar is listed there run the uninstall.