As the coastal states deal with the final storms of the 2016 Hurricane Season – the most active and costliest since 2012 – we’re reminded of how quickly things can change in the path of a storm. Hurricane Matthew was a particularly fickle hurricane, changing its course and defying the tracking models of Americans and Europeans alike. No matter how advanced our meteorological tools may be, it all comes down to timing: the timing of the related weather systems, the timing of the storm’s ongoing development, and the timing of our responses.
Timing may also be the key to how well businesses react to and recover from this year’s storms. Seventeen years ago when Hurricane Floyd hit Florida, there were no reverse lane traffic plans in place and two million evacuees were stranded in traffic snarls for hours. In fact, Matthew was the first major hurricane to hit the U.S. since the advent of today’s pervasive social media tools, enabling businesses to communicate with employees and customers better than ever before.
But as we all know – an unexpected IT impact can damage a company’s business transactions – and reputation – for years to come. No matter how well prepared we may be, there’s never a bad time for businesses to re-examine their IT resiliency plans. Matthew serves as a reminder for businesses to:
- Make sure your data center, call center and end-user facilities are ready if the storm hits.
- Check on things like fuel, access, food, special needs (first aid, etc.) and water in the office in case emergency workers are unable to leave the premises for a period of time.
- Dust off old business continuity plans (either electronic versions or hard copies). Those responsible for business continuity should have a working copy with them should they experience a loss of facility, technology, and/or staff.
- And most importantly, ensure that employees and their families are also ready, prepared and safe during the event.
With more than 35 years of experience under our belts offering solutions in business continuity, information security and IT disaster recovery, Sungard Availability Services (Sungard AS) prepared a list of lessons learned following our experience supporting customers after Hurricane Sandy. We look at the IT recovery challenge as having three layers, all of which must be addressed in order to recover successfully:
- Data Protection: If you don’t have your data at an off-site, secure location, then you really have nothing at all. How you choose to get your data off-site – via tape, disk backup, storage replication, or server replication – will depend upon the mission-criticality of your particular applications and the Recovery Time Objective (RTO) and Recovery Point Objective (RPO) for each. However, simply having your data protected at a second location does not in and of itself make for a disaster recovery plan.
- System Recovery: This constitutes the platforms, servers, operating systems, backup software, backup hardware, hypervisors, networks, and storage that you will use to actually recover your applications. Your recovery environment should align with your production environment, and be compatible with all the above perspectives. If your recovery environment has changed over time, then ideally you have performed adequate change management between your production and recovery environments so that when you attempt to recover your data, the two are in sync.
- People, Processes, and Programs:
- People: It should be obvious that it will be people – your staff – who perform the recoveries. Therefore, it is imperative that they have an operational place to work, with the right equipment, space, and communications to enable them to do their jobs. It is also important that they have the right expertise and focus to successfully recover your data and applications.
- Processes: These are the procedures and “runbooks” that document the steps of the recovery. Your recovery will only be as successful as your “last-known good” procedure, so if these are not updated or correctly maintained, then you run a significant risk of failing at recovering your applications and data.
- Programs: This refers to the ongoing lifecycle and management of the DR program, and governs crucial activities like test planning and execution, post-test analyses, execution of change management, and active integration of best practices and lessons learned on an ongoing basis. Many companies do not have the wherewithal, time, resources, or budget to maintain a robust program conducive to an “always-ready” recovery posture.
Hurricane Matthew’s unpredictability gave U.S. residents a little more time to prepare and brace for the storm’s impact – putting emergency and evacuation plans in place. But that’s no reason to delay in checking and testing your disaster resilience plans. Because winter storms may be just around the corner, and believe it or not, last year the U.S. experienced the highest snowfall from a single snowstorm on record.
While residents in several regions deal with the aftermath and devastation of Hurricane Matthew, Sungard AS remains concerned for all those who have been impacted by the storm, and grateful to first responders and aid workers for their tireless efforts to bring assistance to those in need.