In lieu of any Excel experts, here's what works on Star and I believe the functions are compatible.
Format the input data cell range to time format [HH]:MM:SS then in the cell you want to use as the average put in the function =AVERAGE(A3:A6)
A3 being the top cell and A6 the bottom, substitute whatever yours is. I've assumed the data was entered in a single column.
VoG probably knows a much more elegant way of doing it, so it may be worth waiting for him.