that broadband connections are subject to the same limitations as any network connection - bandwidth contention.
Your broadband service will be subject to a contention ratio of 50:1, which means that - in theory - there may be up to 50 other people contending with you for the available bandwidth on your ISP's gateway. It's unlikely that this scenario would apply too often - if at all - but it is certainly the case that quite a few people will contend with you at any given moment, and the number will increase at peak periods. What happens then is that you are all allocated a share of the bandwidth available, and both your downstream and upstream speeds will suffer.
Another (increasingly common) situation in home wireless networking is that a neighbour, or several neighbours, who are able to detect your network can surreptitiously access and share your bandwidth. This can only happen if your network is unsecured, but if several people start sharing in this way you'll see it reflected in your connection speed.