In last 2.5yrs of performance testing, I have come across some reports that only report an average response time (also during discussion have noticed people only mentioning average response time). For example, consider following two response time data set(graph and dataset). Also assume that the SLA is 6 seconds.
The average response time for both the dataset is 5 and therefore satisfies the SLA. However, looking at the data points you will find that the variation from the mean in DataSet1 (SD=3.19) is more than that in DataSet2 (SD=0.71). You have a mininum value of 1 and maximum value of 9 for dataset1 and you have minimum of 4.5 and maximum of 6 for dataset2. Also 4 out of 10(40%) response time values for dataset1 are above the expected 6 second target. Therefore there is somekind of performance anomoly. This kind of information is missed out if you only report average and therefore may lead into making an incorrect decision by stakeholder(s).
If possible, avoid using only average. However, if you don’t have an option then along with average, I suggest you also report minimum, maximum, standard deviation and sample size. It provides bit more information about the response time.
Update: Only thing that is reported is average response time and there is no graph to show the average response time over time
