Month: May 2011

Legacy DSL Maintenance

Tonight, Saturday May 28 at 12:01AM, we will be performing several minor DSL maintenance tasks. This work will disrupt some DSL service in the LATA 1 (Bay Area) and LATA 8 (Salinas area) for 5-10 minutes.

-Jared, Nathan and Matt

forums.sonic.net launched!

Over the years, our local sonic.* Usenet newsgroups supported a vibrant community of customers and staff.  Unfortunately, participation has dwindled and hence, their value as a resource for our customers.  Despite our nostalgia for the local groups and Usenet as a whole, we know that we need to provide a place for a new community to grow that will be easier for all of our customers to access and a better resource moving forward.

We hope you’ll join us at http://forums.sonic.net/

With the cooperation of customers from all different backgrounds and our employees, we’re sure the forums will become an excellent resource.  Also, those participating in the forums will be the first to learn about and influence new Labs features and services.

ATM Switch Reboot

Tonight at about 6:15 PM the ATM switch serving our newest ATM OC12 suffered a software failure and had to be rebooted to restore service. The reboot caused 5-10 minutes of downtime for affected customers. The switch is back up and operating normally at this time, and we are continuing investigation into the software error.

-Jared

Authoritative Name Server Relocated

One of our authoritative name servers, b.auth-ns.sonic.net, had a hardware failure last Friday and services were restored on temporary hardware shortly thereafter.  (We don’t think these kinds of non-service impacting events warrant an MOTD.)  Rather than fix the original hardware, we decided it was a good opportunity to further increase the geographic and network diversity of our authoritative name servers and have moved it to a facility in Texas.  Our authoritative name servers are now located in three different networks, all with IPv6, in California, Texas, and New York.  -Kelsey

News Service Interuption

The article numbering server in our news cluster suffered a catastrophic failure a couple of hours ago.  All services have been restored on to the backup server but there may be some delay in new articles as the servers catch up.  It is possible that some articles were lost and cannot be re-fed into the system.

Update:  Due to an error promoting a reader slave (this is something we’ve only had to do once before, many years ago) to the article numbering master server, the article numbers being assigned to new articles were grossly incorrect.  As such, all articles received between when services were brought back online and the article numbering server was fixed, are lost to our readers.  I’m sorry we didn’t catch this sooner.

Update:  The news cluster continues to have some issues that we are working to iron out.  The replacement article numbering server is having trouble keeping up with the feed and we’ve had to force it to catch up by flushing out its back log (loosing the articles) several times.  Please note that this only affects our overviews (article lists) and not our ability to retrieve articles by message-id from our spools.  We’re hope to have the situation stabilized soon.

-Kelsey

ATM Switch Reboot

Tonight at about 6:20 PM the ATM switch serving our newest ATM OC12 suffered a software failure and had to be rebooted to restore service. The reboot caused 5-10 minutes of downtime for affected customers. The switch is back up and operating normally at this time, and we are continuing investigation into the software error.

-Jared and Nathan