It's a while since I've done this, so the following is a bit hazy, I'm afraid.
To create a reminder you need to set up a task (there should be a shortcut for this in your Outlook shortcut list). Once you've created the task, double-click on it and its window will open. You'll see that you have the opportunity to set up a reminder for the task, which will prompt you at whatever date/time you set.
If I remember correctly, any reminders that pass their allotted time while the computer is switched off will open up as soon as you boot up.