You WILL need something like "IconPacker" to change Icons for programs, drives, My Computer, etc.. (click here). Without "Third Party" software, you can only change the shortcut icons for programs and drives/special folders (like My Computer)
From Microsoft web site instrucions on how to make Icons. you can use PSP aswell.
Make your own icons
It's shockingly easy to create your own icons in Windows XP. Click Start, click All Programs, click Accessories, and then click Paint. On the Image menu, click Attributes. Type 32 for both the Width and Height of the document, and make sure that Pixels is selected under Units. Click OK to create a new 32x32-pixel document: the size of an icon.
Now add type, colour, or do whatever you'd like to your image. I like to shrink photos (headshots work best) to 32x32 and simply paste them into my Paint document. When you're finished, open the File menu and click Save As. Use the dialog box to choose where you want to save your file, then give it a name followed by ".ico" (without the quotes), and click Save. (The extension ".ico" tells Windows that it's an icon file.) You just created an icon! Now you can change any shortcut or folder to your own icon — just browse to it on your hard drive.