I use Outlook and my ISP mail, so don't need OE. But I would not remove the software, instead just delete the shortcuts in your Program lists and on your desktop, then forget about it. Never know you may decide you want it later and u can restore the shortcuts.
Outlook Express is packaged with Internet Explorer, and thus to remove one may require the removal of the other (in versions 5.5 and upwards). Windows XP home and Pro come with Internet Explorer and Outlook Express installed and these cannot be removed without ripping half the system apart or with great difficulty.
The best advice is to ignore Outlook Express, delete all shortcuts on the Desktop, Quick Launch bar and Start Menu, then forget it, as JaßîsFaß ˜ states.
Webmail services use Internet Explorer, not Outlook Express, so if you're happy to use this instead, then use it. Forget Outlook Express. It won't harm your computer being on there.
Just remember to open Internet Explorer, go to Tools, Internet Options, the Programs tab, and select items other than Outlook Express for all of the lists on this page.
As Dorsai mentions I am sure it can be done without causing any other programs to have errors, but I would ask why remove software if it can easily be ignored. I would recommend you simply delete the shortcuts, ignore it, unless you desperately need to clear some memory space.
I have explored the unticking method, but the description of this action is to remove it from the Start Menu, not remove the whole program. This can be seen by doing this, but highlighting "Outlook Express". The description (at the bottom somewhere) says it will remove the shortcut. It doesn't mention removing the program.