"Will turning it off at night cause my ISP (AOL) to assume I get disconnects, thus lowering my bandwidth to get a stable connection."
Yes, that's my understanding of how it works. As a test however, leave your router connected for a couple of weeks to see what happens. If your speeds don't improve then you may as well go back to turning it off at night.
"Any way I can get them to up my bandwidth as well as ignore the so called unstable connection?"
I doubt that you'll be able to get AOL to do that.