If you store your e-mails on your PC you will not lose them by changing your ISP. They are on your HDD and changing ISP will not affect that. If you store them as web mail, ie you need to go to your ISP mail page to read them, you should put them in your Inbox at the ISP site and mark them as unread to download them. They should then appear in your Inbox in OE, or whichever mail program you use.
You should also advise all those who you have e-mail contact with of your new address as soon as possible on changing ISP.
Most ISPs will allow a short period for any odd bits of mail that appear in your web account with them to be collected after the change over. You would need to ask how long they will leave the account active for this.
I am unsure of what other "information" you may wish to save from the old ISP unless it is confirmation of payments to them. If it is these then simply download details of payments from your account.