Archive for the ‘Business Continuity’ Category

Planning for the Unthinkable: Don’t Forget About Your Workforce in Your Business Continuity/Disaster Recovery Plan

By

Importance of workforce in business continuity and disaster recovery planningBusiness Continuity and Disaster Recovery professionals tend to be perceived as the “Chicken Littles” of the world. We’re always running around exhorting people to be prepared for the “unthinkable.” Sadly, the unthinkable happened in Boston last week. Tragedies like these can take a psychological toll that lasts far beyond the time it takes for you to get back into your building once the “Do Not Cross” tape is taken down. People are companies’ most important assets, as they are the ones who help get the systems, databases, and applications back up and running — and they are frequently the ones most overlooked when companies are building their business continuity or disaster recovery plans.

I am encouraged to see that more and more often, companies ARE including their workforce in their BC/DR plans. What happened in Boston got me thinking more and more about cases where your IT systems are up and running, your building is undamaged, there is no physical reason why your employees cannot enter the building, and yet your employees STILL require somewhere else to work. You might be thinking, “How can this be?”

Well, think about what happened in Boston – let’s imagine that your company had offices near the site of the explosions, or within the perimeter of the lockdown. First of all, your employees COULD NOT have returned to work due to the initial investigation, followed by the order to stay home on Friday, April 19th (which affected nearly 5 million Bostonians and cost the city some $333 million, according to some conjecture by Bloomberg Businessweek). But there is also a psychological cost, in addition to the financial: wouldn’t it be understandable if some of your employees were reluctant to return to “the scene of the crime,” due to the fears and memories they may still retain?

I have another example from several years ago. I was working for a Silicon Valley company whose accounting department was moved to a multi-story building away from the main campus due to lack of space. When the 1989 Loma Prieta earthquake struck, most of the employees were so shaken (no pun intended) that they wouldn’t return to the building. A skeleton staff returned to recover essential functions like payroll and accounts receivable until alternate work space was located. And because this event could have seriously impacted our financial future, I got the funding that I had requested the previous summer to put together my first ever disaster recovery plan – that included people.

Dr. Robert Chandler, director of the Nicholson School of Communication, has stated that employees might be emotionally blocked from entering a building if it triggers unpleasant memories. In addition to the examples above, an act of workplace violence such as an active shooter incident could also keep employees away from their workplace. Dr. Chandler talks about how the cameras focus on the SWAT team and the shooter, but completely ignore the survivors who, after entering the building, go into their office or cubicle, curl up, and cry.

While we’re on the topic of workplace violence, I will just mention that there are many actions that you can take to minimize the risk, even if you cannot 100% guarantee it will not happen. For example, new-hire and ongoing background checks, physical security, and employee harassment training are obvious requirements that are sometimes mandated. I also recommend that you consider active shooter response training and exercises, pre-arranged employee counseling services (perhaps as an extension of your EAP program), and adding work area recovery to your BC plan.

But no matter the actual cause, there are good reasons to have alternate workspace in the event of a disaster. Depending on the industry in which your company is engaged, work-from-home or work-from-Starbucks® might not be appropriate strategies. Your company might be in an industry where your employees cannot work in a non-private place like home or their local coffee shop due to regulations like HIPAA or other mandates for information security.

We can never guarantee against the unthinkable. But we CAN prepare and get our employees the help and resources they need to come out the other end of the dark tunnel. We owe it not only to our employer, but also to our co-workers.

Finally, I want to say that my heart, thoughts, and prayers are with the victims, our courageous law enforcement officers, and everyone else affected by the Boston tragedies. May all of you all get whatever assistance you need to come back into the light.

Ushering in the New Era of Business Continuity and Disaster Recovery Software with SunGard AssuranceCM

By Derek Bluestone, ABCP, Senior Director of Product Management

business continuity software, disaster recovery software, business continuity plan, disaster recovery planWhen I joined SunGard a year and a half ago, I had a very clear charter: to extend our business continuity/disaster recovery software offering and use it to usher in a new era of business continuity assurance. We decided to do this by putting the customer at the center of our product development process. We literally met with hundreds of our customers to understand their pain points, challenges, and future needs. Consequently, we learned a VERY important lesson – one that has served as our “North Star” as we built this product: it’s not just about sitting down to create a knee-jerk business continuity plan to fulfill a compliance requirement. It IS about business outcomes, being operationally resilient, and instilling a sense of confidence that those outcomes can be achieved for business leaders and business continuity professionals alike.

Over the course of the last 15 months, our customers have invested their valuable time with us prototyping, designing, and every 3 weeks, joining us to observe our latest and greatest software build. For that, I’d like to thank them – without their insight and help, we could never have created something that so well addresses their specific needs. The good news is that my team and I will continue to work with them every day – the true business continuity and disaster recovery heroes – to take their guidance and feedback directly into the product. Hear from SunGard’s disaster recovery and business continuity customers on what their experience was like to help us create this new product.

Speaking of which, what is this next-generation business continuity/disaster recovery software called? It’s SunGard AssuranceCM, and we are unveiling it at DRJ Spring World 2013 in Orlando today – so if you happen to be there, stop by our booth to see what we’ve been up to. We will be making SunGard AssuranceCM generally available as a Software-as-a-Service offering on May 31, 2013.

Business Continuity and Disaster Recovery Has Changed

Since the 1980s, business continuity management has seen numerous shifts in regulatory pressures, from data center recovery and Y2K, to terrorism and now state-sponsored cyber threats.  Each issue has forced our customers to react swiftly to address these shifts. In addition to ever-increasing pressures from regulators, business leaders today are demanding broader participation in the planning process and increased confidence that today’s plans will lead to better outcomes, These changes are driving a need for a new business continuity approach.  Successful solutions must therefore avoid the hazards of all past approaches.

The new approach must be enable business continuity and disaster recovery planners to capture valuable input from less “technical” novice planners (we call them the “Innocent Bystanders”) AND operate under a shrinking service budget. This means less of a focus on static plans and templates – and by the way, if you’d like a free disaster recovery plan template, we’re happy to provide one to you – and more of a focus on a solution that is visual, logical, and smart enough to anticipate hazards that could cause companies to detour from their destination, which is positive business outcomes.

As I mentioned earlier, we started from the perspective that it wasn’t just about compliance. Business Continuity Assurance has to go beyond that to deliver what ultimately matters the most for our customers:  better outcomes and increased confidence.  Business continuity and disaster recovery professionals uniformly told us that for them, it’s about engaging the whole company to find the vulnerabilities that matter so that they can guide the next best action, expect change and accommodate it often, and then take what they’ve learned back into the planning cycle and share it across the company. This is how confidence in plans is created and better outcomes are delivered.

Global Warming No Excuse for Lack of Winter Storm #DR Planning

By Bob DiLossi

The 2012 hurricane season has thankfully come to an end and now is the time for businesses to prepare for winter storms. As some parts of North America have been experiencing a milder winter, winter storms can still and will occur – take winter storm Nemo that plagued the Northeast in early February for example.

On average, the United States has roughly four catastrophic winter storms annually with storms occurring most commonly in the northeastern United States. Being prepared is key, in some ways, winter storms can be the most challenging weather systems because they spawn so many types of emergencies.

Blizzards, electrical storms, hail, high winds, ice, sleet, and snow can contribute to communications failures, power outages, and risks to your buildings. Storms also lead to many driving accidents and you can lose critical personnel to injuries from slips and falls.

You need to prepare for all events that may occur, from damage to buildings to your business to your people. All three need to be part of the business continuity plan and part of the testing of your plan. As companies strive to meet the demand for continuous service, they expect 24/7/365 availability. However, the average organization’s requirement for recovery time objective (RTO) from an outage now ranges between two and 24 hours.

To help better protect your organization from the impact of winter storms, below you will find a checklist to gauge where you stand on preparing for winter storms. As you read the list, consider the impact each of the items would have, if they occurred, on your operations.

Building:

  • Building managers unable to get to the building to assess and mitigate damage
  • Communications infrastructure failures
  • Explosions
  • Freezing and flooding of interior building areas that may result in ceilings collapse
  • Gutter clogging with ice dams, leading to leaks
  • Hazardous material accidents
  • Power outages, causing building environmental controls to shut down
  • Roof damage or collapse due to ice, snow, or fallen trees
  • Structural damage or collapse
  • Transportation accidents or closed roads that trap people in or out of your building

People:

  • Communications issues
  • Employee safety
  • Lack of corporate presence during recovery
  • Lack of lodging/logistics
  • Not focused on recoveries
  • Team players not available to travel

When it comes to the business itself, you need to consider a winter storm’s influence on several areas of operation. Run through this checklist and determine how you would satisfy these conditions if problems arose:

Business: 

  • Customers expect supplies and services to continue—or resume rapidly
  • Employees expect both their lives and livelihoods to be protected
  • Insurance companies expect due care to be exercised
  • Regulatory agencies expect their requirements to be met, regardless of circumstances
  • Shareholders expect management control to remain operational
  • Suppliers expect their revenue streams to continue

After going through the checklists and developing ways to address all of these items, you then need a plan of action to use once a disaster strikes. To that end, there are three major steps to begin the process of managing the incident:

  1. Mobilize a central command center, activate a business recovery plan and identify exactly how long the organization will operate in a recovery state, and plan accordingly.
  2. Following-up closely is the need for your organization to carefully document your processes, both in terms of how to recover and how to operate.
  3. You also need to practice and refine processes using a variety of scenarios.

To help with these preparations, a free business continuity toolkit is available from SunGard.

Recognizing the potential disruptive dangers from winter storms, in our next blog we discuss the importance of developing and practicing a suitable DR plan.

SunGard Carlstadt Business Continuity Center serves as Command Post, Shelter During Hurricane Sandy

By George Gobla, Technical Service Delivery Manager

Police, firefighters and EMTs from Moonachie, N.J. used the SunGard Availability Services business continuity center in Carlstadt as an emergency command post during Hurricane Sandy.

Like most other residents of the East Coast, I had been following the news about the approach of Hurricane Sandy vigilantly. As a New Jersey resident, my interest was even greater, as the storm the media dubbed “Frankenstorm” was tracking to make landfall on the evening of October 29 over the New Jersey shoreline and proceed inland.

When it became more likely that Hurricane Sandy would be as destructive as many experts were predicting, the storm was also becoming a concern from a professional standpoint.  As the technical service delivery manager for the Northeast region for SunGard Availability Services, it’s my job to make sure that our facilities in Carlstadt, N.J. – a town about 15 miles west of Manhattan—are operational for our customers during any crisis.

A week prior to the hurricane’s arrival, SunGard activated its three-stage hurricane preparedness process. As part of the process, we carefully followed tested procedures to help keep our employees safe and our customer data secure, our facilities secure and our communications consistent. Along with personnel at other data centers that could be affected by the natural disaster, our on-site facilities team verified that all environmental and electrical gear was in full working order before the storm.

We felt well prepared in Carlstadt despite the fact that New Jersey would face the full power of Hurricane Sandy.

On Monday, October 29, the weather worsened throughout the day. At about 9:30 p.m., I started the one-hour drive from my home to Carlstadt. As I found out later, I was the last person to drive through the local area just before the hurricane hit.

After navigating several detours, I arrived at SunGard’s mega center in Carlstadt—home to two data centers and a business continuity site, which provides customers with a fully functional alternate work space for employees to use while in disaster recovery.

At our Carlstadt data centers, we provide advanced recovery, testing, advanced replication and hosting for customers. That night, my colleagues and I were working furiously to assist customers. Some customers initiated an orderly process of shutting down their equipment, and we were able to control the situation so there was no customer impact due to data center issues.

We also had a number of customers at our facilities and we communicated with them personally and kept them updated throughout the evening. Additionally, there were multiple notifications from our Service Desk and direct phone calls to customers.

As this was happening, at around 11:45 p.m., we had some unexpected visitors. The fire chief of a small nearby town, Moonachie, arrived in his SUV with three ladder trucks, two ambulance squad trucks, and a police cruiser in tow. Moonachie was being overrun with floodwaters from a storm surge caused by Hurricane Sandy, and the officials said they needed refuge and shelter for their own operations, and also for citizens that would be rescued throughout the night.

They asked if SunGard would open its business continuity site for this purpose, and I immediately said yes.

Within minutes, the fire chief had pulled his SUV to the front of the building, opened the back hatch and began using the area to respond to 911 calls and direct emergency operations in the field. Soon after, more emergency responders and the mayor of Moonachie, Dennis Vaccaro, arrived at the business continuity site, and the area became the command post for the duration of the night.

Those that were rescued from their flooded homes, and in some cases from the roofs of their cars, were taken to the SunGard business continuity center. Sheltered and comforted with sheets and blankets, they remained in safety while the hurricane and flooding lashed Moonachie.

In total, our facility provided shelter for approximately 60 residents rescued from danger, and 40 fire, rescue and police.

The Carlstadt facilities remained dry and operational throughout the storm, and I was extremely proud that we were able to assist the community in a small but useful way during Hurricane Sandy.

Q&A with @SunGardAS User Group Forum Keynote Speaker, Michael Leiter


Michael LeiterMichael Leiter
serves as an expert on counterterrorism, cybersecurity, and national security for NBC News and worked as the director of the National Counterterrorism Center (NCTC) from 2008 to July 2011.

On October 15, Mr. Leiter will deliver the keynote address at the annual SunGard Availability Services Business Continuity International User Group Forum at the Chicago Marriott Downtown from Oct. 14 – 16. In his address, “Leading in a Crisis: Before, During, and After,” Mr. Leiter will share lessons on instilling leadership while managing a crisis and describe his experiences with helping manage scenarios that impacted the nation’s security.

The SunGard International User Group Forum is a symposium that offers peer-to-peer sessions on business continuity (BC), real-life case studies of disaster events and success stories of business continuity plans resolving operational disruptions. Attendees will also learn about implementing BC software in an organization and view the next generation of BC Software enhancements that will shape the future of business continuity management. Follow the conversation on Twitter at #SunGardUGF

In advance of the User Group Forum, SunGard asked Mr. Leiter for his opinions on leadership, crisis management, overlooked factors, and how enterprises can learn from national security threats.

What is the most important factor in leading an organization through a crisis?

First and foremost, it’s the idea that planning is not just for a predicted future. Planning is critical for responding in a time of crisis. It allows you to understand your organization, its surroundings, and what you are faced with. And when an unpredictable or predictable event occurs that throws a wrench in works, it is that planning which allows you to respond in a crisis and change the organization’s priorities because you understand it so well.

When is leadership most important when a crisis or business disruption arises?

I’m a very strong believer in leadership from the very top at all times, but especially before and during a crisis. It affects every part of the organization. Part of the responsibility of the highest level of leadership is to create champions in every part of the organization for your business continuity and crisis plan.

Your professional background includes roles in the highest levels of government, including at the U.S. National Counterterrorism Center and the Office of the Director of National Intelligence.
Even at highly organized organizations with respected leaders, do you find that crisis planning still has a role?

When you have a crisis, the best laid plans go out window, except for those pieces that help you understand how your organization can shift and change to respond to new situations. Also, in my experience, in terms of planning during a crisis, it’s critical for a leader to understand all components of an organization and what its capabilities are. Because unfortunately, no matter the organization, many people in the organization may very well lose their cool. The more you have thought about what the organization can do and cannot do, the better position you will be in to react to that time of crisis, to adjust to changed circumstances, and then reshape the organization beyond the period of crisis to be more effective when you have new requirements upon you.

How do you apply lessons you learned from managing major national security threats as director of the U.S. National Counterterrorism Center to business continuity planning for enterprises?

Let’s take the raid on Osama bin Laden [in May 2011] as an example. This was an undertaking that required an enormous amount of planning in the run up to the raid. At various intelligence agencies, people had been working on this mission for over a decade. They had been planning and thinking and identifying every possible eventuality. In this particular case, we knew when the crisis might arise, which was when the operation would be enacted. So in last two weeks before it became active, the plans were shown to an entirely new group of people who had not been involved at all. Everything was presented to them, and we said, “Come up with all the eventualities you can and tell us all things we might be getting wrong.”

Another example is the attacks in Mumbai, India [in 2008]. In evaluating security threats, it was typical at the time to always talk to local authorities like the police. In the Mumbai attacks, it turned out the attackers used fire as a weapon while inside a large building. This was an eventuality no one had thought about.

I think any organization can learn from these examples. It’s great to have intelligent, well-informed people involved in business continuity planning, but also it’s also critical before the finalizing that plan to step back. You want to give that same information and the scenarios to a group of people outside the organization who understand the problem, but who aren’t emotionally involved to the outcomes or the plan. The goal is to try to come up with alternatives to find where the planning may have gone off the mark and to identify the problems.

What are your recommendations for initial steps for building a BC/DR plan for an organization of any size?

You have to start small. You can’t plan for all eventualities and you shouldn’t start with everything falling apart. Start with a smaller crisis, such as what happens if you lose your company email. That can be a crisis, for sure, but it’s much different that losing all your electronic storage. In national security planning, we don’t start with a nuclear attack on Washington. We ask what would happen if there were a suicide bomber in Washington, D.C. and how we would react and handle that. It’s much better to start from a smaller crisis and build out.

As a former national security leader, you had stakeholders across a wide range, such as the White House, agencies like the CIA and FBI, Congressional leaders, and, more broadly, the American public. What advice can you share with organizations about communicating effectively during a crisis to all its important stakeholders?

One thing that immediately comes to mind is that it’s very easy to assume that you understand what a customer wants when a crisis hits. In the case of my own crisis planning at the National Counterterrorism Center, I tried to understand what the President, the White House and members of Congress wanted for information. But I found it’s much better to sit down and ask them, “How do you want this information? When do you want it? What information do you want first?  Who else do you think should be informed about this?” It may be difficult for some businesses to plan this way, but I think it’s important to engage customers and explain that during all these preparations to become well positioned for eventualities, you want to understand their requirements and what information they will want to know.

And internal communications is just as important. In my experience, the vast majority of people find this kind of strategic planning to be an annoyance. When you have a crisis plan developed only by the crisis planning team, it’s helpful but not nearly as useful as one developed by a broader cross section of users.

It’s incredibly important to engage stakeholders inside the organization and, sometimes, leaders have to do that with internal stakeholders by twisting their arm a little. You want to do that in a way that reduces the workload on them, but so that they understand it’s important and you need them. In the end, you will have a much better return on investment. If you leave any part of the organization out, it’s almost guaranteed that’s the part of organization that will open the plan for the first time at the moment of crisis.

During the crisis, it’s obviously about communication, communication, communication. If you can’t effectively communicate messages to employees and leaders across the organization, you could soon be faced with a workforce that thinks there is no plan. Your business crisis will quickly become an existential crisis.