I don't think Herman and myself expected this to become so big, so fast! My post Is the Bandwidth Hog a Myth? and his Congestion Neutrality were picked up by Ars Technica, Slashdot, Gizmodo, Techdirt, DSL Reports and is being discussed on various fora. Thanks to all who picked it up, whether they agreed with our assertions or not.
I've read the comments here and tried to read most of those on Ars Technica and Slashdot although I must admit to being overwhelmed. Still, I've identified a number of trends from what I've read and I'll try to briefly address some of the questions raised or misinterpretations made.
First of all, I guess it's worth stressing a number of things as a bit of context: neither Herman nor myself are affiliated with anyone or getting money from anyone here. Since most of the comments or emails inferring that seem to be coming from the US, it might be worth noting that we're both European and that, although we're intellectually interested in what's happening in the US around the FCC's broadband plan and Net Neutrality rulings, we have no stake in them whatsoever.
I should also add that as far as I'm concerned this post was written of my own initiative and in no way related to my employer. Whether you think my thinking or counting skills are deficient (as one commentator here seems to) or that I'm spot on, the blame or praise is my own.
Overall, I've found the debate of high level so far, so don't let these first few paragraphs suggest that it's all mudslinging. Thanks to all who have kept things civil so that we can hopefully progress in either debunking the myth or being shamefully proven wrong.
Anyway, as far as the various comments and questions go, here are a few clarifications. Most importantly, allow me to focus the debate on the core point of our original posts and maybe relegate some of the other issues raised (some of which are crucially important) to the side for now.
What are posts were not about:
- Our point was not to suggest that congestion didn't happen (this was clearly stated in Herman's post in particular) but that congestion could not be consistently attributed to the "excessive" usage of a small number of users.
- Our point was also not to suggest that caps were bad (which seems to have been how Ars Technica read it) but we certainly suggest that they won't solve congestion issues. There is an interesting question and debate around capped/tiered/variable pricing models, which I've adressed here in the past (Ruminations About Broadband Pricing).
- Finally, our point was not to suggests that some ISPs, especially the small and rural guys were not in a terrible squeeze due to the costs of backhaul. This is another crucially important issue, but it's not what we're talking about. As I've recently noted, there have been important and efficient policy efforts (Intelligent Ways to Solve the Middle Mile Problem) to introduce competition in backhaul, and this is perhaps an area even more important than access regulation when it comes to driving broadband penetration. But that does not imply that a small number of users are responsible for most of the growth in bandwidth usage, and hence for the increase in cost.
At the core, our posts were about answering the question: "Are a small number of well identified users responsible for bandwidth congestion?". Existing literature (see Kenjiro Cho's The Impact and Implications of the Growth in User to User Residential Traffic and the more recently updated Observing Slow Crustal Movement in Residential User Traffic) suggest that while at any given time a small number of users account for most of the aggregated bandwidth usage these are never the same users at different times.
Herman's analysis of the chokepoints in the network (oversubscribed links) and the congestion management mechanisms of TCP/IP suggests to us that the responsibility for congestion and its effects are a design factor of the network. This, together with the existing literature suggests that looking at the aggregate data usage of a given user over a period of time as an indication of network disruption is misguided. Herman's analysis also suggests that there is room for more intelligent implementations of congestion management which will mitigate the congestion problem. And given the growth of video on the Net, the chances of getting congestion on oversubscribed links are increasing.
And as I pointed out the first time, our interest is not in being right on this one, it's on understanding the facts and analysing whether pricing and network management policies are based on sound data points or on suppositions. This is an opportunity to hopefully get down to the bottom of this...
So far I've had two companies get in touch. I hope to get more. I will publish the data set requirements early next week and hope to spark a discussion about how feasible it is for ISPs to gather such data and my ability to do a compelling data dive on that basis.
Again, thanks for the interest and discussions!
