you can easily delete folders in Outlook. Make sure you have the folder view selected in the 'View' menu, and then right click the folder you want to delete. Select 'Delete' from the menu that appears.
Similarly with contacts - go into the contacts view and simply right-click on the contact you want to delete - once more select 'Delete' from the menu.
Or do you mean something else?