Open paint. For desktop icons use a canvas of 128x128px save the file aswhatever name. then use winows to change the extension to .ico
for favicons the little things in the corner like the PCA logo on the corner of the address bar use a size of 16x16 for standard viewing or 32x32. Alternately use 128 x 128 for everything and windows will resize it.