Start by checking the following -
Have You UNCHECKED 'Leave Copy Of Message On Server' ?
Leaving them on the server for any length of time can cause such duplicated mail receipts.
Open Outlook, click on "Tools", choose "Email Accounts", choose "Change or edit and existing account", highlight your POP3 account, then click on the "Change" button.. When that opens, click on the "More Settings" button, then the "Advanced" button. UNCHECK the box that says: "Leave a copy of message on the server". OK and Apply your way out.
Hope this helps.