This article is more than 1 year old

The Register Monopoly Pubcrawl Mobile Map: Vodafone wins Voice

Ideal if you use your phone for talking

Special Report Check out the Register's interactive mobile coverage Monopoly map (voice only for now) right here.

While all the sexy stuff these days in mobile phones is web, apps and tweeting, voice calls are still the major thing people use mobile phones for. The Register and mobile network survey experts GWS took to the London streets to find out which network is best.

Drissa-Coulibaly,-GWS-Opera

Drissa Coulibaly, GWS Operations Director explains DC-HSPA to the barmaid

The Register and Global Wireless Solutions Inc, aka GWS spent the last week of May and the first week of June testing the mobile phone networks around London. You can read more about how, why and where here.

Voice and Packet Data testing was conducted in tandem. Our backpacks contained both voice test devices as well as two data test devices per operator (3G and 4G). We’ll get on to the data in a later article so for now it’s the voice.

We’ll cut to the chase with the “which one is best”. The Winner was Vodafone by a hair over O2 with Three in third place just nudging ahead of EE in fourth place. Our magnificent Monopoly coverage map - for voice only, data will come at the weekend - is here.

Voice testing focused on 3G network predominantly, with fallback to 2G at the network decision level. While for our data testing we separated out technologies, for voice we used what the network chose to give us and tested that.

For the testing we looked at Accessibility (being able to make a call), Retainability (being able to keep the call active to completion) and Quality (the clarity of the call). Accessibility and Retainability were weighted 50/50 into a Reliability score and the Quality was used separately as another ranking factor.

If we had only looked at the reliability scores we would have had a dead heat between O2 and Vodafone for first place and between Three and EE for third place. Only by using voice quality - where Three came top, there was a dead heat between EE and Vodafone for second place and where O2 came last - were we able to produce a meaningful separation of the networks’ performance.

To measure quality we looked at the signal to noise ratio of Ec/Io, and although we didn’t factor it into the rankings we also looked at how often the call dropped back to 2G. What we did do when a call dropped to GSM was use a metric called RxQual to derive the voice quality so our measurement of what sounded best is a blend of the data.

Jasper and Simon plan property purchases

Jasper and Simon look for something to do while Drissa and Fabrice are out testing Oxford Street

The 28 test locations represent places on the Monopoly board, so we had hoped to get a second tier of winners for the colour groups of Purple, Brown, Pink, Green, Blue, Orange, Red and Yellow. We also looked at the London stations. Eagle-eyed readers will spot that we didn’t include Fenchurch Street. This is a reflection of our rigour: we tested Fenchurch, but when we looked at the data we found a number of anomalies in the set and as we want all our testing to be something we can stand behind it was dropped from the mix.

The Water Works we used was at Chelsea, The Electricity company was represented by Battersea power station and we took readings around Pentonville jail. We did try rolling three doubles in the hope of getting sent inside but that didn’t work.

The results when analysed meant that with reliability mostly in the high 90 per cent range it’s too close to call a winner. The worst reliability was O2 at Pentonville Jail at 91 per cent but that was with a low number of calls – 23 on O2 and 89 across the four networks.

Coverage was remarkably poor around the Jail

O2 coverage in this location was remarkably poor. RSSI (Received Signal Strength Indication) was -106dBm which is as bad as it gets and RxQual was at the bottom of the scale with a 7. This occurred when the device was on GSM network mode at network decision. There was also one dropped call on the O2 network, which again occurred while on GSM mode. Drop was attributed also to poor signal level and quality (-103dBm signal strength and 7 for RxQual). Overall at this location, testing revealed the O2 network was dominantly GSM at around 77 per cent.

Colin Myers, GWS's most important person, he pick up the bar tab

Jasper explains to GWS' Colin Myer that wherever they start from they will be late getting back to the office

The worst colour group was EE on Blue (Euston, Angel, Pentonville Road) with 93.7 per cent based on 63 calls. The best location was Pink where three of the four networks got 100 percent reliability and only O2 let the side down with 98.8 per cent.

The groupings are admittedly arbitrary: it rather looks as though the person who designed the Monopoly board didn’t correlate anything with a map. Still it’s their arbitrary choice not ours. Mayfair is a district rather than a location so we made over 300 calls in the area, while Vine street is the kind of place you could have carpeted with a large rug and we made just over fifty calls.

One of the huge advantages of the SwissQual test equipment that GWS uses is that it gives Layer 3 network level messaging so we could really see what was going on. We made 3,513 voice calls but with full access to the information we could see reasons to discard 186 calls.

As an example, some failures were removed because the mobile test device never actually initiated a request to the wireless network operator to set up a call (despite being instructed by the test equipment to do so), since it was busy performing the required network updates, which in that case took priority - and this could be handled differently on different devices so we don't include these types of failures for completeness.

As a result, this data comes from 3,327 calls. Bear in mind that techniques such as using shop-bought phones or crowd-sourced data don’t have this level of sanity checking.

What needs to be borne in mind is that this is outdoor coverage testing (we’ll look at indoor in a separate article) in the centre of the country’s biggest city. As a contrast, landlines are rated at “five nines” 99.999 per cent which is why assistance buttons for those people who are likely to suffer falls are much better connected to fixed rather than mobile connections.

Looking at sound quality – a metric which we’ve never seen publicly stated before – Three came top with a 99.7 percent Voice Quality. EE and Vodafone came second with 98.4 percent, and O2 was on the naughty step with 97.4 percent, however with only 2.3 percent from first to last place it’s a pretty close run thing.

The drop back to 2G is interesting, partly as precursor to figures we’ll look at in the next part of the testing where we see if the networks are sacrificing 3G to promote 4G.

Naturally Three uses 3G 100 per cent of the time, having no 2G to fall back to. Vodafone comes next only dropping back a mere 7.78 percent of the time, EE falls back 19.13 per cent and O2 ran home to its GSM roots a whopping 35.92 per cent of the time. We are planning further analysis of this particular result.

Our Monopoly map uses an application called MapHook where we could create “hooks” or custom places for each of the Monopoly locations. These are superimposed on Google maps, it’s kind of Google maps meets Foursquare but a lot less regimented than Foursquare and much easier than hard coding using the Google APIs. It’s really designed for people, organisations and companies to create their own maps for finding, telling, organising and sharing interesting activities and stories by location. There are Android and iPhone apps and these can pull in data from Yelp, Wikipedia, Foursquare, Twitter, Groupon, and Yahoo Trends, spotting trending information on Yahoo Twitter and FourSquare.

Now that we know that Vodafone is the best network for voice and we need to know what wins when it comes to mobile data. That will be revealed tomorrow when we show the results of the 22,915 file transfers GWS did in the name of research. In doing so they ran up mobile phone bills of £4,982.

It’s interesting to note that even without using voice quality to separate the top two, they were Vodafone and O2, and this is a very different result to that given by RootMetrics which in its London voice testing saw a three way tie between O2, Three and EE with only Vodafone trailing. As we’ve noted before RootMetrics does not have Layer 3 data to help sanity-check for problems and its definition of London seems to be based on whatever map airlines use when they call Gatwick, Stansted and Luton “London”.

Testing is always controversial, it’s one of the topics where we see a lot of reader input: indeed in the comments on the RootMetrics story mentioned above, one reader suggested that The Register ran its own independent tests. With seven parameters measured on each of the 3,327 calls on just the voice testing there is a huge amount of data and we expect Reg readers to have some strong views. So on 29 August we’ll be holding a live chat with Dr Paul Carter, the boss of GWS, to explain the decisions that were made in testing and to look at some of the interesting lessons learned, such as when mobile networks have better upload than download capabilities and who is sacrificing 3G to promote 4G.

You can email questions ahead of the session to livechatquestions(at)theregister.com or log on next Friday (29 August) to post them live. ®

More about

TIP US OFF

Send us news


Other stories you might like