Download and install PicaView32, an excellent program for viewing and manipulating all your pictures. You can rotate the thumbnails so they are the right way up, and view them Full Screen - and do much more.
But I dont know if those will actually recognise that the photos are in portrait as soon as i open windows explorer. I had adobe album starter 3 (whatever that was!), but it became so slow to open a photo in it that i uninstalled it. Before I uninstalled it, if i opened a folder with pics in it, they would all be correctly orientated.
Bit annoying. Might have to reinstall it (and not use it,just for that feature!).