You don't say how big your disc is or how fast your CPU is, but no, my old AMD Athlon used to take several hours to sort my 60Gig drive if I didn't do it too often. I soon got used to doing it weekly.
The first few percent always take the longest. It's like a jigsaw, the more you do, the less there is to sort out, and the faster you do it. This being your first defrag you'll probably find that after it's done it, it starts doing it again, don't worry, it's just sorting out a few oddments that it missed the first time around.