New study paper on popular “speed test” crowdsourcing apps available now
There are many so-called “speed test” apps available in app stores. Such apps offer basic packet-data network performance tests to their users, typically by measuring download and upload data rates and round trip times. Some apps also offer an extended spectrum of tests aimed at assessing network neutrality or fitness for particular purposes.
A typical underlying business model of app providers is to collect results of such tests with geographical information, for sale to interested partners. In some cases, apps are also run by regulators or media houses for use in their publications.
As generally in crowdsourcing, test results are obtained under unsupervised conditions and therefore have limited applicability. Beyond that, it is a well-known fact that apps operated under practically the same conditions with different apps report different results.
In order to bring some quantitative light into this topic, Focus Infocom has recently completed a study paper which compares and characterizes popular “speed test apps” for Android. While the focus is on traffic patterns generated by those apps, the study also describes the underlying methodology – including reference measurements on data rates - and offers comparative examples of numerical results produced by such apps.
With this study and the methodology offered in it, better insight can be gained on how different apps conduct their testing with respect to traffic patterns and which amount of transferred data they produce. By comparison of data taken in public mobile networks and typical home WiFi networks, in can also be assessed how sensitive such apps are with respect to changes in real-world fluctuations of network performance.
This study was written for entities considering the use of crowdsourcing or looking for putting respective results into context of other measurement methodologies.