Archive for December, 2003

Oakland POP outage: The router at Oakland…

Sunday, December 14th, 2003

Sun Dec 14 14:32:01 PST 2003 -- Oakland POP outage: The router at Oakland lost power a few minutes ago. We're currently investigating the failure and should have an update shortly. -Nathan, Russ and Kelsey Update at Sun Dec 14 14:44:04 PST 2003: Power has been restored, and ...

Focal router issues: The router at Focal in…

Saturday, December 13th, 2003

Sat Dec 13 20:46:51 PST 2003 -- Focal router issues: The router at Focal in San Francisco is currently down. All dialup numbers ending in 9606 and 9811 are returning authentication failures. An employee is currently en-route to investigate the issue, and should arrive in approximately 30 minutes. As ...

AOL Mail Delivery Problems: Due to an…

Thursday, December 11th, 2003

Thu Dec 11 00:23:18 PST 2003 -- AOL Mail Delivery Problems: Due to an exploited customer cgi that was being used to send SPAM to AOL customers our outbound mail servers got blacklisted by AOL. Since we became aware of this problem at 9:30PM, we were working hard to ...

DSL outage in Santa Rosa: issue now resolved.

Sunday, December 7th, 2003

Sun Dec 7 20:48:54 PST 2003 -- DSL outage in Santa Rosa: issue now resolved. Tonight we experienced a DSL outage affecting some Santa Rosa customers. The problem cleared itself up as we were investigating and we are currently not clear as to the cause. We will continue to ...

Router Maintenance Update and More Kernel…

Friday, December 5th, 2003

Fri Dec 5 02:58:17 PST 2003 -- Router Maintenance Update and More Kernel Upgrades: The router maintenance at Focal went more or less as planned. We encountered one issue with the new IOS release but we able to work around it. So far, so good. We hope this will ...

Focal Router Maintenance: Tonight at Midnight

Thursday, December 4th, 2003

Thu Dec 4 17:40:30 PST 2003 -- Focal Router Maintenance: Tonight at Midnight we are going to working on the router at Focal. Users that are not directly connected to the focal router may notice degraded performance for the duration of the maintenance, while those that dial into the ...

Upgrades and Changes to custmx: We upgrade…

Wednesday, December 3rd, 2003

Wed Dec 3 16:25:28 PST 2003 -- Upgrades and Changes to custmx: We upgrade the RAM and disks in our customer secondary MX server today to better handle the load on the box. We also enabled use of mail-abuse.org's RBL+, a combination of the RBL, DUL, RSS and OPS, ...

Problems With the new Kernels: Our web…

Tuesday, December 2nd, 2003

Tue Dec 2 07:54:35 PST 2003 -- Problems With the new Kernels: Our web cluster has been having stability problems with the new kernel. www.sonic.net and all of our hosted domains web sites have been having reachability issues since early this morning. We're working to resolve the problems to ...

Kernel Upgrades: Due to a new exploit that’s…

Monday, December 1st, 2003

Mon Dec 1 21:50:04 PST 2003 -- Kernel Upgrades: Due to a new exploit that's been seen in the wild, we are upgrading all of the kernels on our servers. All machines which customers have access to have already been patched and rebooted. We're continuing to work down the ...