“iPhone beats Android” Chip Online headlined its report about mobile internet benchmarks performed by Focus Infocom.
Focus Infocom provides a comprehensive collection of tools for smartphone-based measurements that allow advanced and differentiated measurements with a high degree of end user perspective. The tools are available for Android and iOS devices.
Focus Infocom’s test tools allow a wide spectrum of test setups including benchmark drive tests to directly compare devices or mobile networks, and stationary tests to study the behavior of single devices competing for resources in a network.
The CHIP Online article focuses on a demo setup intended to show only a small section of the versatile possibilities provided by Focus Infocom’s measurement tools. Nevertheless, it offered some interesting insights into the performance of iOS vs. Android devices and their scramble for resources.
For the benchmark tests, Focus Infocom deployed the iPhone 4S, the HTC One S and, for reference purposes, the HTC Desire HD smartphones. They were run in the networks of the four German mobile operators in parallel with a mix of services. The measurements were performed on several approx. 50 km long drive test routes near Darmstadt and Wiesbaden that led through heavily populated as well as rural areas on inner city routes, country roads and highways.
The tests revealed that the overall success rates of the iPhone 4S – calculated over all networks and services – were generally higher than those of the HTC Desire HD (i.e. 91percent for the iPhone and 83 percent for the HTC Desire HD. However, the HTC Desire prevailed in the upload domain with a success rate of 95 percent as compared to 92 percent for the iPhone 4S.
For the test iPhone vs HTC Desire HD, the median and average values for upload and download rates do not show considerable overall differences, a fact that shows that there were no high upper or lower spikes in the measurement data.
The download rates for iPhone were generally about 20 percent higher than for the HTC Desire HD with the exception of the ePlus network. Taking into account that such figures contain both device and network dependent factors, they are to be understood as indicative values only.
With regard to the upload data rates, the HTC Desire HD has a slight advantage in the Telekom network while the iPhone prevails in the three other networks, resulting in an overall pole position for the iPhone (approx. 10 percent higher rates). With both devices, however, the general order of the four networks remains the same.
When compared to the HTC One S, the advantage of the HSPA+ technology of that device compared to the iPhone, shows a clear effect in average and median download and upload data rates. It is interesting to observe, however, that the iPhone tops the HTC One S in two different networks, once in upload data rates (D2) and once in download data rates (o2). A possible explanation lies in the fact that the higher-order coding schemes in HSPA+ are more sensitive to poor radio conditions during some parts of the routes.
Session Times for Webbrowsing were also measured, and compared iPhone vs HTC Desire HD on the one hand and
iPhone vs. HTC One S on the other hand:
Both comparisons show clearly faster session times for the iPhone. Having said that, it is important to keep in mind that these values are not really comparable as regards network performance, because the values also depend on the devices’ different browsers.
Additionally, Focus Infocom looked at how devices scramble for resources if they are booked into the same cell. Several combinations were tested, with a focus on the question if a network prefers certain phone types over others in a systematical way. While the test showed that there were strong mutual influences when using the iPhone and the HTC Desire HD within the same network cell, no effects on this scale could be observed when operating the two HTC devices in the same setup (which included using the same SIMs). It has to be emphasized that - on the current level - these measurements are primarily intended to show the potential of the tools used. It would be premature to draw quantitative conclusions.
However, the measurements revealed two important facts:
The tests confirmed the results of the CHIP Online Benchmark test published in September, showing that using other devices for the same sort of test does not change the general outcome as regards the performance of the four German network providers.
More important than the concrete results are the underlying questions that can be answered using Focus Infocom’s smartphone based measurement systems for different service tests and on multiple platforms.
Focus Infocom’s smartphone-based tools are available as standard drive- and walk test as well as multi-device load generator configurations. Additionally, SLA validation tools that can be adapted to specific tasks are also available for direct use on end user devices.
The general range of supported service tests in smartphone-based products includes these and more packet data based test cases as well as a wide range of voice services tests (MOC, MTC, mobile to mobile) including speech quality evaluation (PESQ®, POLQA®) in both narrowband and wideband/SWB configurations.
Supported radio access technologies range from GSM/GPRS/EDGE over HSPA/HSPA+ (including Dual Carrier) to LTE.