Month: October 2012

Core Router Maintenance

Tonight, October 30, starting at 11:59pm, we will be performing maintenance on some of our core networking equipment. We do not anticipate any customer downtime during this window.

 

-Tomoc and Robbie

Update: Maintenance completed as planned, no service interruption.

NOC Hotline Down

Due to Hurricane Sandy on the East Coast, calls may not ring though to the Network Operations Hotline. This does not affect calls to Customer Support.

 

Update: We believe that the Network Operations Hotline toll-free service will suffer an extended outage due to Hurricane Sandy. We’ve activated 707-206-4202 as a replacement until the original number is back in service.

Legacy DSL Maintenance

This evening, beginning at 11:59PM, we will be performing maintenance on equipment serving a subset of legacy DSL customers in the Bay Area. Expected downtime is less than 10 minutes for affected customers.

-Tim, Robbie, and Tomoc

Update: All work has been completed as planned.

Flexlink Outage

Tonight, October 22, equipment serving a small subset of Flexlink Ethernet customers became unreachable. We are dispatching immediately to replace the failed hardware and will provide an update when there is one available.

-Tomoc and Robbie

We have replaced the failed hardware and all affected customers are now back online.

Legacy DSL Outage

At around 7PM this evening an aggregation device that serves a subset of legacy DSL subscribers in the  Bay Area became unresponsive. The device was reloaded and service was restored by 7:15PM. We are investigating the cause of the issue further to ensure it does not happen again.

-Tim

Sacramento Backhaul Upgrades

Tonight, October 11, starting at 11:59pm, we will be cutting over to a new backhaul for our ATM and telecom services in the Sacramento area. This migration should not cause any down time for telecom services, and expected down time for legacy ATM services is less than 10 minutes.

-Tomoc, Robbie, and Clay

Update: All upgrades have been completed, down time was less than 10 minutes

Network Maintenance

Tonight, October 9th, starting at 11:59pm, we will be performing maintenance on equipment serving our Santa Rosa data center.  No customer impact is expected for this maintenance window.

-Tomoc

Update: While performing this maintenance, a software bug on a redundant set of routers caused  intermittent reachability to several Sonic.net services between 12:15am and 1:00am.  All services have been restored at this point, and we are working with our vendors to ensure this does not happen again.

Router Maintenance

Tonight, October 3rd, starting at 11:59PM, we will be performing maintenance on several routers that serve legacy dsl customers customers in Northern California. Expected downtime is less than 15 minutes.

– Robbie and Tomoc

Update: Maintenance completed successfully

Switch Failure

Today, at 11:37AM, one of our atm switches serving FRATM and Business-T customers experienced a software failure and rebooted. As of 11:48AM the switch is back online and working normally. We are currently investigating the cause of the failure.

– Noc

Server Infrastructure Upgrades

Tonight, October 1, 2012, starting at 11:59pm, we will be performing upgrades on some of the networking equipment in our Santa Rosa data center. These upgrades should not cause down time for any Sonic.net connections or services.

-Tomoc and Kelsey

Update: Upgrades are complete