"For example on one newsletter from Radio4 I get squares where there should be inverted commas"
This is not a keyboard issue. Squares are a default display character when the system doesn't know what symbol to use.
"whenever I use the pound symbol it comes ou as a capital A"
Do you mean when you type into Notepad - some of the symbols show wrongly - if so could you give a list of what happens please.
If you're using XP - then go to the Control Panel - Regional and Language Options. Look through all the settings on all the tabs, and ensure they're appropriate for (presumably) English (UK).