The Nature of Measurement
By its nature, we need to make our measurements both regularly and randomly. While we record the nature of the system we observe with regular measurements, we can get rid of the effects that may arise from seasonality with random samples. For example, heavy traffic after work may create the perception in people that traffic is always heavy, while traffic is actually quite calm during the day.
GTmetrix Results
When we run several tests of the same website in the same business owner database location from GTmetrix, we see that the measurements are significantly different from each other. I would like to show you a few examples below:
Here are the measurements we took: 7.7, 4.4 and 17.2 . These measurements raise a few questions in our minds:
Which of these metrics is the actual page load time according to GTmetrix?
If I average these three measurements can I find my real loading time? (their average is around 10) You can compare this result at the end of the article.
If I take three more measurements, how will they differ from the previous three?
Is the value 17.2 a repeating value? Or did we encounter this value only once by chance?
We can expand on these questions and try to estimate our actual loading time, or we can take more measurements and calculate the most reliable average value using the confidence interval.
Mean and Confidence Interval
The more measurements we make, the better we can understand the nature of the system we are measuring. We can start by seeing if the data range is finite or infinite, and we can check if the measurements change over time. Most importantly, we can reduce our confidence interval around the mean and get closer to our true mean value. In this example, we chose Vancouver, Canada, as the default location for our measurements in GTmetrix. Below you can see how I calculate the actual load time: First, we measure the same site from the same location 16 times using GTmetrix.
Then we calculate the mean, standard deviation and confidence interval values. You can use an Excel type software for all of these calculations. The mean() and standard deviation() functions on Excel will allow you to do these operations. You can also use this ready Excel file for this operation. Mean value = 6.35625 Confidence interval = 1.960 * Stddev(loading_time) / root(n); = 1.62777 This shows that our average value is between 6.35625 ± 1.62777 . The lower and upper limits are as follows: 4.72848 and 7.98402 This calculation shows that the average of our site's loading time is between 4.72848 and 7.98402. However, the average is not yet as precise as we would like. We continue to measure. When we reach the 100th value, we can calculate the mean and confidence interval again. 100. Measurement Mean value = 7.52 Confidence interval = 1.06173 We see that we have a smaller range here. 7.52 ± 1.06173 Our real average value is between 6.45 and 8.59, we can continue the measurements and narrow down our range, but this is enough for now. Another thing that caught our attention here is that the range has shifted a little compared to our first calculation.
