Web cluster instability

UPDATE (07/22/16 2:30PM): We have been dealing with a sporadic attack towards value hosting customers during the last two days. Value hosting services were restored at approximately 10PM last night. These attacks also affected colocation services, and NOC has taken steps to reduce the impact if another denial of service attack occurs.

UPDATE: NOC is currently blocking parked.sonic.net on our network edge, since this is the destination of the attack. This is resulting in reachability issue for customers with the value hosting service, and we are reviewing other options.

Our web cluster is currently experiencing a denial of service attack. Stability may fluctuate while Operation mitigates the attack.

 

SOC

Incoming mail configuration error

Recently senders using a google.com address began receiving bounce messages when sending to sonic.net users. This was due to an erroneous configuration on our end, and the problem has been resolved.

 

SOC

Webcluster Outage.

Starting at 1:30 pm we experienced a service outage for a large segment of our shared hosting cluster. Services have been restored, and operations is investigating the cause of the issue.

Grant and SOC

Fusion Fiber Maintenance in Brentwood and San Francisco

Tonight starting at 11:59pm we will be conducting maintenance to equipment serving Fusion Fiber customers in the Brentwood and San Francisco areas.  Estimated time of completion is ~2hr.

-Brandon

Update: Customers will experience a brief interruption in voice and data during this time as equipment has to be upgraded.  Expected downtime is about 15 minutes.

Update: (1:45am) Maintenance is now complete

-Brandon

Sebastopol Fusion Fiber Maintenance

Tonight at 11:59pm we will be conducting maintenance on equipment serving Fusion Fiber customers in Sebastopol.  Expected time for maintenance completion is 1 hour.

-Brandon

Update (12:33am) – Maintenance is now complete.

-Brandon

Northern California Telecom Maintenance

Tonight, July 7, starting at 11:59pm, we will be performing maintenance on core equipment in the Brentwood and San Rafael areas. Although end to end connectivity should not be affected, customers may experience a brief period of routing instability.

Update(3:10am): Maintenance complete

-Michael

System Maintenance

UPDATE: Maintenance complete.

Starting at 11:59pm SOC will be running system updates. The following services may experience brief interruptions:

  • Webmail
  • Membertools
  • Sonic.com website
  • FTP service

Interruptions to individual services will be brief, and the Maintenance window is expected to last until 1am.

-Grant and SOC

Northern California Telecom Maintenance

Update (2:01AM): Maintenance complete

Tonight, June 30, starting at 11:59pm, we will be performing maintenance on core equipment in the Richmond, Hayward, and San Jose areas. Although end to end connectivity should not be affected, customers may experience a brief period of routing instability.

-Michael and Tomoc