Sometimes it just doesn’t work. Even with all of the planning and documenting that you do, sometimes snow happens, or someone does the wrong thing or for some crazy unknown reason it just doesn’t work. With technology (ok, most things too), it always seem to be that little bit of crazy that comes in at the last second and takes you for a ride. I’ve been dealing with some of that at CNS. Just this morning our email server decided to give us a little hiccup. Earlier in the month, the snows cancelled not one, not even two – but THREE events that we have been planning for quite some time. Last week, we had snow melting into our offices through the roof. Taking these all together is certainly a bit of chaos, but taken on at a time (as these things ‘normally’ occur), what is your strategy for persevering?

I am a big fan of the fall back plan. This probably comes from my background as an IT Project Manager. When performing a server migration, the number one thing to have in mind is how I will recover from a failure. Back in the old days, it was quite common to have failed migrations. While Microsoft has improved their software and made failure much less common, it does still happen – as evidenced this morning. To make a long story short, my email server was unhappy about being converted to a virtual machine. What has the fallback? Turn the physical machine back on and re-group. I’ve virtualized plenty of servers and don’t expect what happened today as the norm, but before I even start, I make sure I am in the position to fallback – whenever I feel the need. We had a little inconvenience this morning, but in the end it was just that – inconvenient. Without a fallback plan, it could have been a lot more.

Taking this out of IT, what scenarios could you use a fallback plan? Do you leave breadcrumbs on the trail so you can find your way back?

The weather outside is frightful, and if I had a fireplace it would be delightful. But I don’t have a fireplace, and I also don’t have any more patience for this snow. I am writing this on Day 5 of Snowmaggedon and the Federal Government is closed for the third day in a row. CNS, like many small businesses in the area, follows the Federal Government for closing delays and that has meant a lot of lost productivity and missed work days. However, over the past few days I’ve discovered that not all of our customers closed for the snow. In fact, I’ve been fielding calls and emails from employees and customers alike with problems and outages. It seems that even though CNS was closed, our customers weren’t. To address that, we made the difficult decision to break away from the government and open for business for the remainder of the snow emergency – albeit a little modified.

Being a technology company, we know a thing or two about enabling our employees to work remotely. We are able to leverage the VoIP system which we procured to support our employees in Baltimore, Florida, and Ohio. But the real key here is that we can do this because we have setup our infrastructure to allow it. Our key systems are located in a power and Internet redundant data center. While the Poolesville office only lost power for a few hours over the weekend, this certainly is a key consideration to ensuring our operations. We have also invested in VPN technologies that allow our employees to securely access our network environment. Finally, the most important of all, is that we actively ensure that our employees know how to access these systems by encouraging telework. I’d rather not even think about how difficult it would be to install and configure the necessary software across our remote user base in the middle of this type of event – add to that the user training and it is almost not worth it.

So far this week, I have recovered 6 failed servers, brought an Exchange server back online and dealt with a sick voicemail server. I’ve also helped customers implement mass text messaging service, help people past failed flash drives and try to help someone setup remote access – all from the comforts of my home office with my four year old and Scooby Doo in the background.

How well did you fare?