Fit a HDD is very easy, but you need to check if your motherboard support it. It maybe difficult to get memory for your PC, place to look click here, very very good site to find right memory.
I would say, not worth to upgrade.
As an example:
IBM PIII 1GHz PC 256Mb RAM, 20Gig, Floppy with CD (Full size bay) (2nd hand) £169 click here
If you can DIY, I can recommend some kit for you to make a better PC base unit less than £169