There was nothing remarkable about the weather in Cedar Rapids, Iowa, on Wednesday, September 21, but Dave Wallrichs, director of business continuity at GreatAmerica Financial Services, was starting to get concerned. Looking out the office window of the GreatAmerica headquarters in downtown Cedar Rapids, the Cedar River provides a picturesque backdrop for the work day for most of the company’s more than 500 employees. But from a disaster planning perspective, the body of water is also a potential threat. Wallrichs learned that an area to the north, home to tributaries that feed the Cedar River, was getting severe rain. This meant the river could rise. That evening, he made a move.
“I reached out to our executive business continuity team and reported, ‘Heavy rains have created an interesting development on the Cedar River,’” Wallrichs said, “basically putting them on notice even though there wasn’t an event yet.”
Two days later, GreatAmerica officially “declared,” – there was a flood on the horizon that could threaten access to the company’s building. By that Saturday at 5:00 p.m., the entire downtown of Cedar Rapids was restricted to essential personnel only.
But GreatAmerica didn’t go dark. Thanks to thoughtful strategy, thorough employee training and smartly-deployed tech solutions, the company kept doing business. While no two companies are the same – nor are any two disasters – GreatAmerica’s business continuity success offers insight to SMBs fine-tuning their own strategies and solution providers selling the hardware and consulting services to help deal with disaster.
GreatAmerica’s winning strategy came together based on lessons learned during a similar event eight years earlier.
Learning from Past Disasters
In 2008, Cedar Rapids experienced the biggest flood in its history. According to Jim Burns, vice president of information technology, that time they had only about a day of notice that the office had to be evacuated.
GreatAmerica had relationships in place with other local institutions, including a local community college, to use space during a disaster. So GreatAmerica’s employees kept working, albeit squeezed into unfamiliar settings. But that still left the data center in the GreatAmerica building. Were there water damage to the building, the servers could be damaged too. The IT team moved fast.
“There was no damage, but that was only because of a heroic effort of the IT staff,” Burns said. “We actually disassembled 90 percent of the data center and reassembled it overnight” in pre-arranged data center space.
When things returned to normal, GreatAmerica made some changes – ones that would make things run a great deal more smoothly, and with less guesswork, in 2016.
New Hardware and New Locations
By the time the 2016 flood occurred, GreatAmerica had relocated its data center to a building eight miles away from the office – outside of the flood zone. It had also entered into an agreement for an auxiliary office to be more self-reliant in the event of a disaster. The remote office remains in place, ready to go live, year-round.
“That’s a huge commitment from our executives all the way down because they support it and feel it is a very important element of our continuity plan,” Wallrichs said.
Like the main office, the remote office connects to the data center by fiber-optic cable. GreatAmerica maintains a minimal connection year-round so IT can ramp up the connectivity speed in the event of an emergency. From that point, employees can work just as they would from the main office.
But it wasn’t just tech infrastructure that kept GreatAmerica afloat through the flood. Part of business recovery is technology, but the greater part is mobilizing people.
The GreatAmerica Strategy: From Creation to Deployment
GreatAmerica’s business continuity plan began with a business impact analysis. Wallrichs had gathered together leaders of each business unit and put together a picture of what each unit needed in order to maintain their baseline of doing business. It also picked an immediate response team – its first staff set up to work in the satellite office to keep things up and running.
When the disaster struck, Wallrichs leveraged his crisis communication system. Communications went out to all staff by Web, email and phone to make sure that nobody missed the message. The immediate response team were contacted first and they knew exactly what to do.
“I had recently conducted a training exercise prior to this event; so we had just done this very thing,” Wallrichs said.
On Monday, the week of displacement began. Members of the immediate response team occupied the satellite office with other teams working remotely.
“With the data center in place our IT guys weren’t busy with servers at all,” Wallrichs said. “They were able to help our immediate response team, making our facility better for the deployment.”
Then the GreatAmerica business unit leadership took over operations, ensuring the comfort of employees and day-to-day logistics.
Helping Businesses Plan – With the Worst in Mind
Disasters are unpredictable and can occur with less warning – and more force – than the flood in Cedar Rapids. CompTIA Security Executive Council member Mike Semel of Semel Consulting knows this well. Having spent 14 years working with the Red Cross Disaster Services and as a rescue captain for a fire department, he has seen chaos – and how people react to it – up close. Watching the news on Hurricane Matthew battering the southeast, Semel speculated on what businesses were enduring and noted one factor that often frustrates disaster planning.
“It all boils down to the fact that we’re all people,” Semel said. “Even in our own homes – are the batteries in our disaster kits still good? Over time we all become complacent and I think that really transfers to the workplace.”
So what is the worst? Protracted power outages. Failures in the communication infrastructure. And even the loss of an employee or employer. Planning to keep a business going amid such dire circumstances is difficult, but critical.
And for those selling services, it’s important to understand just how comprehensive planning needs to be, and how far it extends beyond just technological solutions.
“It becomes a bigger picture than just having the IT environment built for resiliency,” Semel said. “It’s about following it through and looking at what the rest of the processes are in order to help the client not just recover the data that’s in their system but recover their business.”
Building systems that will work with or without given players and creating communication strategies that allow for multiple avenues of contact are likewise key.
“The more you plan, the less you will have to react to the disaster,” Semel said. “Nobody has the password list in their head. There’s never only one set of keys.”
And Semel noted another easy-to-overlook part of disaster planning.
“Most plans fail because people don’t test them,” Semel said.
GreatAmerica: Back in the Office
But GreatAmerica had tested their plan. And while the flood may not have been as chaotic as some disasters, without a plan in place it could have meant downtime or infrastructural damage. Instead, the team delivered service seamlessly to their customers.
The downtown of Cedar Rapids opened again and GreatAmerica resumed business next to the river as usual. And as Wallrichs continues to keep an eye on the weather, he notes one critical part of a business continuity strategy.
“Communication with your employees is key,” Wallrichs said. “It is critical we let them know what’s happening and what actions we’re taking in response to the event.”
Matthew Stern is a freelance writer based in Chicago who covers information technology, retail and various other topics and industries.
