System Maintenance

Update: This maintenance is complete.

Tonight at 11:59pm we will be performing updates to several systems; Impacted customer facing services will include mail, member tools, ipv6 tunnels, sql databases, custom hosting, and vpn services.  The maintenance should be completed within 2 hours. Thank you.

  • Dexter K – Sonic System Operations

Petaluma – Fusion/FlexLink Intrusive Maintenance

Update (1:56AM): Maintenance complete

Beginning tonight at midnight, we will be performing upgrades on equipment serving Fusion and Flexlink customers in Petaluma. Expected downtime for this operation can be up to 2 hours, but is typically less than 30 minutes for most customers.

-Network Engineering

Disk Usage Notifications

Our systems experienced a glitch in how we calculated disk usage for customers between November 29th and 30th. We have corrected the error, and no action is needed to ensure that you are billed correctly for the month. Some customers received an email (in error) notification that disk usage was over quota. We apologize for any confusion this has caused and have taken steps to ensure the same glitch does not happen again.

To view your account’s current disk usage, please visit our Member Tools page and go to Account -> Resource Usage -> Current Disk Usage.

— Joe @ System Operations

Northern California Telecom Maintenance

Tonight, starting at midnight, we will be performing maintenance on equipment serving our Northern California Telecom network. No downtime is expected, however some customers may experience brief periods of routing instability.

Update(11:11pm): Maintenance postponed until tomorrow night at midnight

Update(12/1 4:51am): Maintenance is now complete. An equipment failure during this maintenance caused a disruption of service for customers with Fiber SSE services.

-Tomoc

Sacramento Area – Fusion/FlexLink and Legacy DSL Maintenance

Update(1:50AM): Maintenance complete.

Beginning tonight, at midnight, we will be performing maintenance on equipment that provides transport for all Fusion/FlexLink and legacy DSL services in the Sacramento area. We do not expect any downtime from this operation though users may notice some routing changes.

  • Network Engineering

Ukiah – Fusion/FlexLink Intrusive Maintenance

Update (1:13AM): Maintenance complete

Beginning tonight at midnight, we will be performing upgrades on equipment serving Fusion and Flexlink customers in Ukiah. Expected downtime for this operation can be up to 2 hours, but is typically less than 30 minutes for most customers.

-Network Engineering

Sonic Network Outage

Sonic is currently experiencing a network wide issue.  Some locations and services have been restored.  We are working fast to get everyone up and will update with more information

Network Engineering

Update(12:56pm) – We believe all services should be restored.

Followup(2:32pm) – This outage was caused by a member of our team applying route filtering changes to core networking equipment. The new filtering policy was tested in a lab environment, but testing failed to take into account certain real-world conditions. The problem became apparent almost immediately but our engineering team required time to restore all equipment to it’s previous state. We will be going over and reinforcing proper testing and procedure to ensure any preventable outages do not occur in the future.

Our sincere apologies, and thank you for your patience

Legacy DSL Outage – Chico

We are currently experiencing an outage for Legacy DSL services in the Chico area.  We are investigating the issue and will update when more is known.

-Brandon

Update(2:22am) – We are currently working with our upstream providers to look into the matter and are awaiting further information from them at this time.

Update(6:38am) – Service is now restored

SSL Certificate Issues

UPDATE: Our CA has reissued our certificates, and they have been installed.

The Extended Validation (EV) certificates we installed last week have blocked a small, but significant, subset of users from accessing our websites due to the use of Elliptic Curve Cryptography (ECC).  In most cases, using Firefox instead of IE, Safari or Chrome resolves the problem.  We recognize this isn’t always and acceptable solution and have asked the Certificate Authority (CA) to reissue the certs using RSA and will install them as they become available.  However, we strongly encourage customers using Windows XP or old version of OSX to upgrade their operating systems to a modern supported version.  Using an old unsupported browser from an old unsupported operating system is a “Bad Idea.”  It is also exceedingly likely that future steps taken to ensure the privacy and security of our customers at large will cause similar issues for these old systems.