I've been a big fan of the work done by OECD to compile data on broadband delivery in various countries. The data compiled at the OECD Broadband Portal is an excellent resource that I've used many times in recent months. The very nature of the OECD however and the fact that its work is done in a very official and sometimes constrained manner imposes limitations on what they can analyse and publish.
One area of frustration, typically, is the actual amount of download and upload bandwidth delivered by broadband offerings the OECD countries. What OECD measures is the average download rates of all the offerings on the market, but since there's no way of knowing how these offers are spread in the market, this in unweighted average, and therefore not terribly meaningful in understanding what customers are actually getting. Furthermore, since we all know that nominal access link capacity and actual bandwidth vary considerably, it makes using these figures interesting in terms of trend analysis, but very disputable by whomever would dislike the conclusions drawn from them.
In early September, Cisco released a study that they financed but was performed by independant scholars from the Oxford Said Business School and the Universidad de Oviedo that offer actual measurements of end to end bandwidth and latency from statistically significant numbers of users in 42 countries, including all of the OECD countries. The research team used actual measurements in May 2008 performed by www.speedtest.com.
From this, they aggregated the data and build trimmed averages for three key indicators, namely download throughput, upload throughput and latency. This is, to my knowledge, the first ever source of such data made publicly available. From there, Cisco makes its own pitch about why this matters (you didn't think they didn't have an idea in mind!) by building what they call the broadband quality score (BQS) which is an aggregate of these three measures.
An important point to note is that the measurements (and therefore the associated score) are end-to-end. So what we're looking at is not just access line capacity (real or nominal), it's customer experience. In a sense it's more elusive because the factors that explain the results are harder to pinpoint. In another sense, it's more real because it's actually what customers get.
The study ranks the BQ scores which gives interesting and significantly different results from the OECD rankings based purely on average advertised download rates. Furthermore, the study establishes thresholds for what they consider to be today's and tomorrow's customer needs:
-
Today's customer needs are considered to be 3.75Mbps download, 1Mbps upload and 90ms latency. This compounds to a BQS of 32.
-
Tomorrow's customer needs are considered to be 11.25Mbps download, 5Mbps upload and a latency of 60ms. This compounds to a BQS of 75.
Here is the ranking graph:
There are a few surprises in there. The one that drew the most comments and ink is the UK which is on the wrong side of the current quality threshold (barely). You can read UK lags in broadband quality on ITPro which is a fairly measured commentary on this issue. Looking back, it's not all that surprising that the UK wouldn't quite cut it since there's been a lot of dissatisfaction with UK download rates in recent months. More interestingly, 3 out of 5 of the "big" western European countries don't make it. Not only the UK but Spain and Italy as well.
A comparison with the OECD download ranks (which were, until now, the best measure of broadband quality we had) is very interesting. Only considering OECD countries, Japan stays on top, Germany gains 6 ranks (to #7) and the US gains 2 (to #12), other G7 countries get hammered: France drops 6 ranks (to #8), UK drops 5 ranks (to #17), Italy drops a shopping 17 ranks (to #23), and Canada drops 4 ranks (to #19). Some other non G7 OECD countries get boosted. The Netherlands gains a stunning 20 ranks (to #3), Switzerland gains 17 (to #5) and Denmark gains 15 (to #6).
Of course, Cisco's big message is that only one out of 42 countries (Japan) makes tomorrow's customer requirement. And on paper, they're not all that stringent. Nominally, every country in the western world where ADSL 2+ has been deployed should make the download threshold if theoretical rates were actually delivered. Obviously, that's not the case and they don't. There's a clear connection to fiber deployment here: the top 5 countries are all amongst the most advanced in fiber deployment.
Looking at uploads is even more stunning. Only 13 countries in the whole sample actually break the 1Mb/s upload rate, and Japan is the only G7 country amonst that subsample. Only 7 OECD countries offer more than 1Mbps on average. Now I don't think anyone would consider a 1Mbps upload to be anything to write home about and yet very few of us actually get that. Here's a graph I did using the data in the report. It's telling to see some of the countries in the lower end of the graph...
There's a lot more in there that would be worth commenting, in particular a two axis mapping of the 42 countries looking at both BQS and penetration, but I want to keep this digestible, so you'll have to find that for yourselves!
So what does this all tell us?
First of all, that even if you don't agree with Cisco's take on tomorrow - or even today's - services, the quality of the broadband that's currently delivered is not all that good even in countries that tend to boast their OECD ranks. Rankings, it has been said and should be repeated, are inaccurate tools. Even these are. But rather than pick them apart to dismiss them (as low ranked countries tend to do with OECD ranks) or boast about them (as high ranking countries often do) countries should aim to fill these inaccuracies themselves and identify where and why they should improve.
This report tells us a number of really interesting things, I think:
-
fiber matters: it matters to downloads, it matters to uploads, it matters to latency. No surprise there, but it's always good to see hard figures that prove it.
-
next generation broadband ecosystem is needed to deliver today's services: to me this study is a pretty convincing argument that the real question shouldn't be about NGAs delivering next-generation services. I'm willing to bet, for example, that if Cisco runs this study again, we'll discover that Spain and Portugal will radically improve as fiber to the home is rolled-out (and adopted) en masse. The flip side of that is that next-generation services will be needed to pay for the infrastructure, and the business models are still uncertain. We're back to this "who is building the next gen services?" thing. But the need for the infrastructure is clear.
-
access links is not everything: may sound contradictory, but it's pretty clear that overall small countries do better in the ranking than large countries, no matter the broadband penetration or presence of fiber. I think that's due to two often overlooked topics, namely the quality of backhaul and the quality of IXes. These former is usually a lesser issue in small countries. The latter plays an important part in actual customer experience as is evidenced in the amazing performance of the Netherlands in the rankings. Sure, there's emerging fiber in the Netherlands, but I would argue not enough at this stage to significantly impact the results. On the other hand, Amsterdam has the largest IX in Europe, and that matters a lot in end to end connectivity.
At the end of the day, at a country level, the question that remains is "does this actually matter". It's the same old macro-economic argument, where everyone intuitively agrees that it does but since you can't measure hard figures to prove it, it's down to faith. And even if a nation decides (as Greece and Finland seem to have) that it matters enough to put hard cash on the table, infrastructure alone isn't sufficient to make a difference.
As James wrote on his blog last week (I don't want to say ranted, but it is in red...) Japan may have this great infrastructure, but they're not exactly a poster child for all the cool stuff you can do with it that will generate value to the economy. I would argue as he does that it's a lot down to cultural aspects, I would also suggest that we're seeing signs of changes in that respect, but I would suggest that so far the poster child for the wider economic adoption (and associated milking) of next generation networks is Sweden.
Speed matters, but it's not everything.
