Punt!
Written by Dane Jasper
August 27, 2008 | 2 min read
As mentioned in the status blog (formerly system MOTD), BroadLink Communications had a system failure in a critical component of their network today. A hard drive in their core router failed, and while they do have spare parts, they were unable to rebuild it.
BroadLink’s wireless towers in Santa Rosa continue to serve a small number of customers, many of whom still cannot obtain DSL broadband service due to their locations. While BroadLink is a business in decline, it does provide a very valuable service to some under served locations.
Designed over ten years ago, the network is based upon 802.11 wireless access points which serve intelligent Linux CPE at customer premises. The network’s core converts bridged Ethernet to ATM PVCs, one for each customer. (A PVC is a permanent virtual circuit – think of it as a unique customer’s connection inside a larger pipe.) The ATM is fed via a T3 to the Sonic.net network, where we provide IP routing to the Internet. Provisioning and automated management of the per customer configurations is all via the ATM PVCs. This system was designed prior to the viability of more modern solutions like dot1q vlan tagging, PPPoE or MAC RADIUS auth, which could provide similar functionality. The ATM PVC configuration was a very innovative and tidy solution, and if anything, it was before it’s time.
So, in summary we’ve got Ethernet on the WAN (the wireless), which feeds into a magic one of a kind box (The “Red-C”) that converts it into ATM and PVCs. It’s handed off to Sonic.net, where we terminate the ATM into a Redback SMS, basically in the same way that a DSL customer is provisioned. In this way, the hundreds of wireless customers are managed in our systems in the same way that nearly 50,000 DSL loops are managed. Out the other side: Ethernet. Hmm. Basically exactly what went in, but with per-customer provisioning, locking, diagnostics and management in the middle.
The Red-C died – hard drive failure. So, we pulled it and the Redback out, and are simply routing BroadLink’s entire bridged network to the Internet. See the pic for the combined Sonic.net and BroadLink response team working on these changes to the network this afternoon.
This change is transparent to the end-user, but not really sustainable for the long term due to the inability to provision, diagnose, lock, disconnect and manage individual end-users. But, everyone is online, and we will address the bigger picture another day.
The photo includes, top to bottom, Jason Kane, Sonic.net wireless product manager, Tim McAllister, BroadLink board member, Nathan Patrick, Sonic.net network architect, Scott Woods, former BroadLink engineer (Thanks Scott!) and Josh, BroadLink’s technician. Unfortunately, CEO Warren Linney is at Burning Man and is unreachable. Warren, I hope you remembered your googles, water, sunscreen, and a spare cloned backup hard disk drive!