...you are liable to be capped down to 1Mb download speed. This applies if you download a lot of large files.
The other option may be that the modem is suffering from heat. You didn't say which one you have. You may need to interrogate the modem's setup pages to check the frequencies. These should be supplied by VirginMedia, together with the method off correcting them.
If this is the problem and they continue to got out of synch, you may need a new modem.