The Server North
N

'blog

Home Main Site NOC/Info Site Twitter Stream Archives Login
You are currently viewing archive for February 2008
Feb
23
All Clear. All systems should be back up and running. Yes we suffered some outage due to one of our upstream's colocate suite losing power, and then our equipment was all depowered for a few minutes. We kept the SAN / jails (VPS) / hosting offline until all power work was completed as hard-shutdowns are a 4-6 hour recovery process. I'll be posting some details on my personal 'blog in the next few days.

Posted by: Myke |

Feb
22
Surprise maintenance! Tomorrow (Saturday 2008-02-23) Northam will be shutting down the power to all of 151 Front Street for annual maintenance.
This is the building where we have colocated our (and your) equipment in 3 separate suites, and most of our upstreams have as well.

It is possible that the UPSes and Gensets may fail. It has happened before, in 2006. I (Myke) will be onsite for the majority of the duration to tend to things if there are problems. Unfortunately, there's not much warning if there's going to be a problem, my role is soley reactionary. A friendly ISP told us "if anything's going to happen, it'll be in the first hour or two, or the last hour. It's pretty boring in-between.", thus, I'm hoping for a very boring day.

It's entirely possible we will remain unscathed, but our various upstreams may be. I remember in 2006, both my Rogers and Telus cellphones stopped working for a few hours too...!

Posted by: Myke |

Feb
13
SAN Failure All jails and some hosting are affected. We are working on it, unfortunately ETA looks to be a couple hours at this point. It takes a very long time to fsck a terabyte, while rebuilding a mirror of it. Read More »

Posted by: Myke |

Feb
12
ADSL Maintenance Our ADSL supplier (on very short notice) has to do some work on their infrastructure right now (early AM.)

Apparently it was a surprise for them too. Apologizes all around.

Posted by: Myke |

Feb
05
Switch replacement tonight After being completely frustrated by VLAN 1 leaks on various switches, I've decided I'm going to ditch the Ciscos and use one of my Foundrys instead. That is, until we get our paws on one of those new Juniper switches in a few months. Do I need PoE? No. Would I like to have 48 ports of non-lame GigE? Yes very. I'll even gladly accept the 10Gbe scalability! I'd be happier if we could have two immediately.

Tonight (Toronto TZ), expect a few brief interruptions that shouldn't last much longer than disconnect/de-rack/rack/boot/reconnect/MAC population...

At the same time I'm upgrading our GigE switch to the same thing, just more ports in the blade, same deal - shouldn't take more than a few minutes if all goes well. Clearly I intend to have them preconfig'd before we do it, and all of the cables are already labeled.

As result, we should be able to start peering directly on TorIX later in the week!

Posted by: Myke |

Powered by NucleusCMS | Ported by VinhBoy | Designed by DemusDesign