Not a lot, really.
Out in the real world, where servers are often over-worked at peak times and have finite bandwidth, there is only so much that one can expect from them. Even on a humble 8Mb connection, I often see a lot of variations between servers. One will fly along, while another will plainly not be using my connection speed to the fullest.
And what are you downloading?
Music - not much difference.
Video - advantageous, pending the above comments.
Webpages - diddly squat.