It certainly sounds like a graphics card issue. Have you double checked your VGA cable isn't loose? Did you have similar graphics issues with Vista? if so, then I would opt for a new graphics card. If no graphics issue occurred with Vista, then I suspect that the graphic card may require an up to date driver to run properly on windows 8 (driver for an older graphics card may not yet be available) or it's possible that your graphic card is not quite powerful enough to run windows 8 (I don't know what the minimum graphic requirement is for W8), best to double check.
Alienware 17 R4 2017 review