If you were watching telco news out of America this week, you would have the right to be mystified about the stoush between Sprint and Comcast. How is it that thousands of customers could go dark merely because two companies are in a dispute over their peering connection?
It's an issue that's worth understanding for the business broadband user, because it's something that could influence the way you buy your Internet access.
“Peer” is one of those words that gets applied in a host of different circumstances. At the service provider level, “peer” means something different to what it means in the world of “peer-to-peer” clients in the consumer industry. When two ISPs “peer” with each other, they're agreeing to exchange traffic on a commercial basis.
The way ISPs choose to exchange traffic is at the foundation of how the Internet works – and why you're able to send e-mails, access Web sites, use VoIP phones, and set up VPN access to the office from a home location.
The Structure in a Nutshell
Once your ISP has established your connection, it needs some way to get your traffic to the interconnected networks that make up “the Internet”.
At the “inside” of the Internet (if such a diffuse creature can be imagined as having an “inside” and “outside”) there is a host of interconnected routers owned by so-called “Tier One” providers. If you request a US-hosted Web page from an office in Sydney, your ISP needs some way to get traffic from Sydney to the Tier One networks, and from there the traffic needs to find the Web server you're seeking, to set up a communications link.
Let's stick, however, with “your” end of the connection.
To get traffic from its own network to the Internet, a non-Tier-One ISP needs to buy services from someone with access to those Tier One links – in Australia, this will often mean buying services from Telstra, Optus, AAPT or Verizon, the “gang of four”, or from an international transit provider that has a presence in Australia.
At the other end – presuming that the Web site you're accessing is similarly at the end of a non-Tier-One link, there will be another ISP buying access to a Tier One network, and a customer (the owner of the Web site) buying service from the ISP. This is outlined (in simplified form) in Figure 1.
The problem is that upstream links are expensive: ISPs will pay hundreds of dollars per month, per Mbps of capacity they buy.
But ISPs can also see something about the traffic they're sending over that link – they can see where the traffic is going, since every packet contains an IP address. Your ISP can't help but buy upstream traffic if its customers are accessing Time Magazine in the US – but what if your ISP notices that some of its traffic doesn't need to travel across the Pacific?
It might, for example, analyse customer traffic and discover that customers are, for some reason, communicating with another local ISP. When ISPs find themselves exchanging a lot of traffic, they can circumvent the need to use their expensive upstream Internet links by creating a direct connection from one network to another. If, for example, two ISPs happened to have a connection to the same data centre, they can put a cable directly between their switches and create a route that passes traffic from one network to another – and agree not to charge each other.
If lots of ISPs come to a similar arrangement, they might set up a peering network in which they all agree to exchange their traffic directly, saving the upstream links for content that isn't available on their own networks (Figure 2).
They will also look at the content, and see if they can't get an even greater benefit by inviting major content hosts to “peer” with them as well: take a look at the peering list on Pipe Networks' Website (URL) and you'll find not only ISPs but content providers as well.
Peering in Australia
The advantages to peering aren't only financial. They can also significantly reduce the latency across the network, since traffic doesn't need as many router hops between source and destination – something beloved of gamers, but also important to VoIP or video traffic.
If, for example, Telstra and Optus did not “peer” each others' networks, traffic could take a very long trip indeed. A Telstra user's e-mail to an Optus e-mail address might have to travel across the Pacific on Telstra's international links, to a network termination in America, to be routed to a network to which Optus has a connection, to make a return trip over the pond so as to finally reach the Optus network.
Such things have happened – for example, when (check NZ story)
Peering in Australia, then, exists in various models: ISPs outside the so-called “gang of four” might set up their own peering connections (including in some cases setting up a purpose-built data centre) for traffic that doesn't need to traverse the Telstra/Optus/AAPT/Verizon networks nor travel to the US; and the “gang of four” peer with each other to pass customer traffic that doesn't need to make the American trip. There are also commercial peering models – companies providing peering facilities.
But there's another aspect to peering, one which while more relevant in America is still worth understanding, and that's peering as a commercial tactic.
The “Tier One” market has evolved differently in America to Australia, and peering traffic is often exchanged on a “differential” basis. If Provider A and Provider B are sending the same amount of traffic to each other, they might agree to pass that traffic without fee; but if there's a difference in one direction or another, the “freeloader” (the network that's taking more traffic than it gives) will have to pay, based on the gap.
In that case, it makes sense for a peer to try and build up a fund of popular content, in the hope that it will be able to some day send a bill to the provider at the other end of the peering link. And here is where the Comcast / Sprint dispute has come to a head: the commercial dispute centres on whether the two parties are meeting the terms of their peering contract, as they relate to payment for traffic.
The impact of Sprint's decision to “de-peer” Comcast has been to stop Comcast users getting their traffic to Sprint destinations. That suggests an over-reliance on a single route from one place to another (which goes somewhat against Internet best practise of making sure you have as many routes as possible, so that traffic can 'route around' a failure).
Why Peering Matters
Australian users won't see the same kind of disputes, because the commercial model at the heart of the Comcast-Sprint dispute isn't relevant here. Small ISPs never built the content base that turns peering into a commercial weapon, differential charging is not the default model for buying Internet transit within Australia, and even when the ACCC has looked at the commercial implications of the Australian peering market, it's been met with an indifferent response from the industry.
However, business broadband customers should still try and understand how peering works, and what peering relationships exist, because it is relevant to the service they buy. There are several considerations here.
Most importantly, an ISP with many peers in the local market should offer better performance between networks. Just ask any enthusiastic online game-player about the value of good “ping times”.
While the business user isn't going to lose his character to someone whose ISP passes through the “shoot!” command quicker, performance is important to sensitive applications such as VoIP, as well as to employees accessing the corporate network from a remote location.
For most companies, it's not feasible to mandate which ISP their staff will use at home. But if the business's ISP is peered to the consumer ISP used by a staff member, then remote access will work better (especially if the staff are using VoIP to connect to the office phone network) than if the packets have to traverse lots of upstream networks (or, in an unlikely worst case, make the trans-Pacific journey).
Many ISPs publish their own tables of peer networks, or at least list the peer exchanges in which they maintain a presence, so that's a good starting point for research. You can also look at the member lists from organisations that provide peering such as Pipe Networks' peer list, or that provided by WAIX.
And finally, if you're taking the time to talk to sales staff at an ISP, it's worth asking with whom they have peer connections.