Jan
19
19
No Pain, No Gain
Unfortunately it's been a rough couple of weeks for our main mail server, Vice.
First there was the unexpected release of Mozilla Thunderbird 3, which we know a lot of our customers use. TB3 had a couple minor bugs/interactions that would make it complain the first time you opened a folder, it also would complain the first time you tried to send a message, and then again when it copied your message to the Sent folder. Adding to the pain was that TB3 caches all of your mail on the server to your computer, which put a very heavy load on the server - for up to a few hours, depending on how many messages/attachments you had.
Secondly we've been pushing to get our proper TLS/SSL certificates in place, (it was mentioned a while back) but there were some issues with the software and the developers have been working very hard on this - they actually redeveloped the entire encryption part! In the past week there've been a few failed attempts to make this work, you probably noticed these.
Third, one of the above 'tests' outright crashed the mail server. We had to reset the entire machine. That hurt.
Fourth, a bug was found in PostgreSQL, the excellent database server that stores all the email and our business data. This caused mail to get stuck or occasionally get delayed.
Fifth, in the past 24-36 hours, Rogers has been having serious issues with 3G/EDGE and even regular GSM service, this has hit most Canadian iPhone users, and also compounded load issues. They appear to be ongoing today, but at this moment of writing are okay. At this point, if you're having issues with your E-mail on iPhone (or even iPod Touch) please reboot it.
An additional ongoing issue in some Canadian Federal Government departments (namely Parliament and HRDC - both managed by PWGSC?) have had their own issues receiving mail. This isn't anything we can do anything about, but given we have a sizable Ottawa customer base - a lot of people may have noticed this.
I (Myke) and the developers have been intensely hacking away at the server for the past 16 hours or so, and while there were some ugly bumps this morning, I think we've got all the problems licked! TB3 reports no errors. Properly CA-signed certificates are in place for SSL/TLS connections. The stuck-mail issue has 3 bugfixes in place and is being monitored. PostgreSQL extra-row bug has been mitigated.
If you're getting errors or warnings, please ensure the name of the mail server you're connecting to is Mail.ServerNorth.net and restart whatever device you're using. If you're still getting errors, do contact us and describe how/when it happens... if possible or relevant, attach a screenshot.
Also watch this space for details on our 5th Birthday Party!
First there was the unexpected release of Mozilla Thunderbird 3, which we know a lot of our customers use. TB3 had a couple minor bugs/interactions that would make it complain the first time you opened a folder, it also would complain the first time you tried to send a message, and then again when it copied your message to the Sent folder. Adding to the pain was that TB3 caches all of your mail on the server to your computer, which put a very heavy load on the server - for up to a few hours, depending on how many messages/attachments you had.
Secondly we've been pushing to get our proper TLS/SSL certificates in place, (it was mentioned a while back) but there were some issues with the software and the developers have been working very hard on this - they actually redeveloped the entire encryption part! In the past week there've been a few failed attempts to make this work, you probably noticed these.
Third, one of the above 'tests' outright crashed the mail server. We had to reset the entire machine. That hurt.
Fourth, a bug was found in PostgreSQL, the excellent database server that stores all the email and our business data. This caused mail to get stuck or occasionally get delayed.
Fifth, in the past 24-36 hours, Rogers has been having serious issues with 3G/EDGE and even regular GSM service, this has hit most Canadian iPhone users, and also compounded load issues. They appear to be ongoing today, but at this moment of writing are okay. At this point, if you're having issues with your E-mail on iPhone (or even iPod Touch) please reboot it.
An additional ongoing issue in some Canadian Federal Government departments (namely Parliament and HRDC - both managed by PWGSC?) have had their own issues receiving mail. This isn't anything we can do anything about, but given we have a sizable Ottawa customer base - a lot of people may have noticed this.
I (Myke) and the developers have been intensely hacking away at the server for the past 16 hours or so, and while there were some ugly bumps this morning, I think we've got all the problems licked! TB3 reports no errors. Properly CA-signed certificates are in place for SSL/TLS connections. The stuck-mail issue has 3 bugfixes in place and is being monitored. PostgreSQL extra-row bug has been mitigated.
If you're getting errors or warnings, please ensure the name of the mail server you're connecting to is Mail.ServerNorth.net and restart whatever device you're using. If you're still getting errors, do contact us and describe how/when it happens... if possible or relevant, attach a screenshot.
Also watch this space for details on our 5th Birthday Party!
Posted by: Myke
Comments
No comments yet
Add Comment
Powered by NucleusCMS | Ported by VinhBoy | Designed by DemusDesign
This item is closed, it's not possible to add new comments to it or to vote on it