The Server North
N

'blog

Home Main Site NOC/Info Site Twitter Stream Archives Login
Feb
03
Maintenance Window Warning - Diesel Day 20140301 Diesel Day, aka "Annual Power Maintenance" will be occurring throughout the weekend of March 1st, 2014. During this period, the entire 151 Front building will be removed from the Ontario power grid and will be running on diesel generator power.

Things don't always go right, there is a very high likelihood of outages. If you have mission-critical tasks occurring that weekend, please contact us to investigate deploying in our Montréal data-center services.

Posted by: Myke |

Dec
29
Major Outage - Montréal Datacenter Currently the entire facility is unreachable, we have a partial diagnosis relating to power, however this is unconfirmed.
Repair time is being hampered by the holiday-season as well.

Expect all services to be running by midnight, if not sooner.

UPDATE: Outage caused by failed PDU.

Posted by: Myke |

Aug
15
Surprise Cable Modem Firmware Updates Unbeknownst to us, Rogers has sending firmware updates to Motorola SB6120 modems since August 12th. No user action is required, however if you notice your modem blinking in an atypical fashion, leave it be for some time and don't reset or power-cycle it.

Posted by: Myke |

Jul
17
Major cable Internet outage Fiber cut somewhere in the upstream system.



Cable Repair Crew on Bronson


UPDATE 7PM - Most connections restored.

Posted by: Myke |

Jun
07
151 Front Running on Generator Power


Posted by: Myke |

May
02
Extended Montréal Outage For approximately 57 minutes now we have been experiencing a total outage in our Montréal facility due to an upstream vendor failure. This is presently impacting E-Mail users the most.
Unfortunately, due to the nature of this failure and our network design, we are unable to provide any mean-time-to-repair estimates, however this is the longest disruption we've faced with that site in over 6 years. All parties are working as fast as possible to resolve this issue.

Posted by: Myke |

Nov
09
Major Rogers-Cable Outage We are getting widespread reports of our cable Internet customers being offline. Unfortunately, as result of needing Rogers for the "Last Mile" element of your connection - while we're up, our pipe to you is not.

We will update as more info is available...

Update 15:27: “There is currently a Rogers-Wide Issue. At this time we are not opening tickets. There is currently no ETA but Rogers is working hard at resolving the issues”

Update 16:02: Reports of connections coming back online

Update 16:17: Reports of LOTS connections coming back online :)

Posted by: Myke |

Jul
07
Keeping Tabs & DNS Changer Privacy is important to us at Server North, we've actually got a pretty decent policy (posted on our Alpha website) that protects our customers rather well. In fact, we brandished it a few weeks ago in dealing with a court order.

But from time to time, we need to look at who's doing what on the network. Usually we're working with a customer to help debug a problem, or occasionally we need to diagnose or explain strange traffic on the network. We're an ISP, we really can't know what to expect on our network at all times so we do have to peek. A rough guesstimate would say that 98% of the time we're looking, it's at volumes or sources & destinations of traffic - it is exceedingly rare that we look at the contents of the network traffic.

Again, in just about every instance we do this, is while working with a customer. The only indiscriminate monitoring we do is volume or flow, which makes pretty graphs that allow us to keep an eye on usage across the network - and that looks like this:
Traffic Graph

Nothing terribly sensitive to the customer-base there. (On the other hand, we don't like sharing those graphs, that's an internal file server's network interface.)

Why are we going on about this?

It starts with the FBI shutting down some temporary name servers (DNS) that were formally used for nefarious purposes, as result, there's a lot of headline-grabbing occurring with media outlets screaming "Government turning off Internet for millions of innocent Internet users!" and whatnot. Without getting into a disseration about hyperbole and crying FIRE! in a crowded theatre, this isn't entirely true.

The short version is: hackers used viruses and malware to change settings on people's computers to use their evil DNS servers. The bad guys got shutdown but the FBI kept the servers running - but in a non-evil mode. Now they've decided they're going to shut them down and for the infected computers, their ability to convert names like "ServerNorth.net" to IP addresses (98.124.60.65) will go away. An affected machine will still be on the Internet, but the online "phonebook" will be gone, so it'll be rather difficult to use the Internet. Unless you enjoy surfing by IP: https://199.59.148.10 ;)

For more information about what's going on, check out the DCWG website, there's also a lot of resources there including links to testing tools.

Monday those servers are going down.

Now as you likely know, Server North is a small company, we have a few hundred customers and know almost all of you by name. We figured we could monitor our outgoing network traffic for DNS queries headed for the affected servers and pre-emptively contact the customer and help them get cleaned up. We fired up our packet watcher and waited to see who might need a hand...

Packet Dump


One DNS request. One.

... And that was me (Myke) testing that the filter rule was working. (If you look carefully, you'll see I looked up "foo")

Hmm, guess you guys don't need our help after all...

Have a nice weekend!
Read More »

Posted by: Myke |

Jun
03
Wiring a house for Voice & DSL Here's what those boxes on the outside of your house are for:
Thanks to WTL from DarnerMedia for the recording, and Myke for the dissertation.


Phone line comes in from a splice-case on the pole via an aerial cable, is attached to the side of the house, runs down the pipe, into the Bell Canada NID, (aka "Protector", "Fuse", "Demarc") which is the demarcation between my wiring and the telco's wiring. From there it's run into an ADSL/VDSL/xDSL POTS splitter (just a big Z-Filter) and then the twisted-pair cables are run the the various phone jacks and ADSL modem. The little blobs you see are Scotch Locks, which are handle little wire connectors - you don't even need to strip the insulation off of 'em!

Also - you can now get integrated protector+POTS-Splitter modules in your NIDs:
NID POTS Splitter demarc


If you live in a apartment building ("MDU"), this video shows what the other end of a DSL line looks like. (It's a trick though! That's just the middle, since your dial tone still comes from somewhere else!)

Posted by: Myke |

May
15
Paypal Subscription Cancellations If you are a High-Speed DSL or Broadband Cable Internet subscriber, you may have received notice that your Paypal subscription has been cancelled. This is due to the recent price increases we E-Mailed you about a few weeks ago. We need to terminate the Paypal Subscription Payment as there's no mechanism for us to update the price. (Which makes sense, otherwise we'd start charging a million dollars for service and retire really early ;)

Don't worry, we are not turning off your service!
(Nor are we charging a million dollars for it.)

Your next invoice will include a link to re-set up the subscription payment, please follow through immediately once you receive the URL, as those links are time-limited.

Sorry for the lack of clarity on the automatic Paypal message.

Posted by: Myke |

Powered by NucleusCMS | Ported by VinhBoy | Designed by DemusDesign