The Oxford English Dictionary confirms that pence is the plural of penny.
It goes on to say "The two plural forms pence and pennies tend to be used for different purposes: pence refers to sums of money (five pounds and sixty-nine pence) while pennies refers to the coins themselves (I left two pennies on the table). It is a mistake to use pence rather than penny in the singular to refer to one penny: the chancellor will put one pence on income tax."
That is a straight cut and paste from the concise OED
And do you all remember that when the Chancellor announced a one penny rise in the price of petrol from 6pm on Budget day there would be long queues at the forecourts to get a few gallons at the old price. And that was when the penny was on a gallon not a litre so most people would save literally just a few pence