Disaster Recovery Pressures Rise Due to Cost of Downtime and More Stringent RTOs
Symantec Corp. announced the global results of its fifth Disaster Recovery survey, which demonstrates rising DR pressures on organizations caused by soaring downtime costs and more stringent IT service level requirements to mitigate risk to the business. The study also shows that while DR budgets are higher in 2009, they are expected to remain flat over the next few years – requiring IT professionals to do more with the same or less.
The survey highlights that while recovery time objectives were reduced to less than four hours in 2009, disaster recovery testing and virtualization are still major challenges for organizations. Respondents report that DR testing increasingly impacts customers and revenue, and one in four tests fail. Nearly a third of organizations don’t test virtual environments as part of their disaster recovery plans, and a slightly larger percentage of virtual environments aren’t regularly backed up – pointing to the need for more automation and cross-environment tools.
Downtime costs are significant
The average cost of executing/implementing disaster recovery plans for each downtime incident worldwide according to respondents is US $287,600. In Australia and New Zealand, the median cost was $570,000 compared to $900,000 in North America. Globally, this number is highest for healthcare and financial services organizations.
This is alarming when one considers that one in four tests failed and 93 percent of organizations have had to execute on their disaster recovery plans. Respondents reported that it takes on average three hours to achieve skeleton operations after an outage, and four hours to be up and running. This is dramatically improved over the 2008 findings, where only three per cent of respondents reported that they could achieve skeleton operations within 12 hours, and 31 percent believed they would have baseline operations within one day.
2009 DR spending bucks trend
The research shows that the annual median budget for disaster recovery initiatives, including backup, recovery, clustering, archiving, spare servers, replication, tape, services, disaster recovery plan development and off-site costs at data centers surveyed is $50 million. According to respondents, this number will continue to grow throughout 2009, but more than half (52 per cent globally and 55 per cent in Australia and New Zealand) of respondents believe that budgets will be flat in 2010, making it more challenging for IT management to better leverage their assets including hardware, software and personnel.
Executive involvement doubled in past year
According to the 2009 disaster recovery survey, 70 per cent (66 per cent in Australia and New Zealand) of respondents reported that their disaster recovery committees involved the CIO, CTO or IT director – a significant increase from last year’s research where 33 per cent of respondents indicated executive involvement. As budgets increased over the past year, disaster recovery initiatives have become more of a competitive differentiater, and impact of downtown on customers is greater than ever. Another reason for executive involvement is the increase of applications that are seen as mission critical. Sixty percent of applications globally and 55 percent in Australia and New Zealand were deemed mission critical by respondents, and nearly the same amount (61 percent in Australia and New Zealand) is covered in disaster recovery plans. Any sort of outage to these systems will have an enormous impact to the business.
Disaster recovery testing improves but still a major challenge
This year, 35 percent of respondents globally and 39 percent in Australia and New Zealand reported that they test their DR plans once per year or less frequently – a 12 percent improvement globally (21 percent improvement in Australia and New Zealand) from last year. In addition, one in four tests still fail, showing a dramatic need for improvement in this area. Reasons most respondents cited for why organizations aren’t testing include:
- Lack of resources in terms of people’s time (48 percent globally and 40 percent in Australia and New Zealand)
- Disruption to employees (44 percent globally and 41 percent in Australia and New Zealand)
- Budget (44 percent globally and in Australia and New Zealand)
- Disruption to customers (40 percent globally and 39 percent in Australia and New Zealand)
Also a concern is that more organizations reported that disaster recovery testing increasingly impacts customers and revenue over previous years. Forty per cent of respondents globally and 39 per cent in Australia and New Zealand reported that disaster recovery testing will impact their organizations’ customers and nearly one third (27 percent globally and in Australia and New Zealand) reported that such testing could impact their organisation’s sales and revenue (up from one-fifth or 21 percent globally and 12 percent in ANZ in 2008). Symantec recommends that organizations implement disaster recovery testing methods that can be run frequently and without disruption to business operations. Symantec believes that people and processes are the main reason tests fail, pointing to the need for more automation.
Virtualization still a major challenge
Sixty-four percent of worldwide respondents and 59 percent in Australia and New Zealand reported that virtualization is causing them to re-evaluate their disaster recovery plans. This is up from 55 percent globally and 44 percent in Australia and New Zealand in 2008. Still, nearly a third (27 percent globally and 35 percent in Australia and New Zealand) of organizations do not test virtual environments as part of their disaster recovery initiatives. This number has improved in the past year, lowering from more than one-third (35 percent) of organizations who did not test in 2008. Additionally, more than one-third (36 percent) of data on virtualized systems is not regularly backed up, showing no improvement in the past year (37 percent in 2008). Over half of the respondents cited lack of backup storage capacity and automated recovery tools as top challenges to protecting data in virtual environments.
In addition, the study found that globally, more than half of respondents cited:
- Lack of storage management tools as the top challenge in protecting mission critical data and applications in virtual environments (53 percent globally and 51 percent in Australia and New Zealand).
- Lack of backup storage capacity (52 percent globally and 39 percent in ANZ) and lack of automated recovery tools (50 percent globally and 49 percent in ANZ) both came in a close second globally. Within ANZ, lack of automated recovery (49 percent); insufficient backup tools (42 percent in 2008; 14 percent in 2008); lack of available backup and storage capacity (39 per cent); and different tools for physical and virtual environments (39 per cent in 2009; 28 per cent in 2008) also were cited as key challenges to protecting mission critical data and applications in virtual environments.
- Resource constraints such as people, budget and space as the top challenges to backing up virtual machines suggesting a need for greater automation and the ability to leverage existing IT investments in order to lower costs.
Recommendations
As demonstrated over multiple years of this study, lack of resources continues to be an issue, yet the costs of downtime are staggering. Organizations also can do a better job at curbing the costs of downtime by implementing more automation tools that minimize human involvement and address other weaknesses in their disaster recovery plans.
Because disaster recovery testing is invaluable, but can significantly impact business – including customers and revenue – organizations should seek to improve the success of testing by evaluating and implementing testing methods which are non-disruptive.
Finally, organizations should include those responsible for virtualization into disaster recovery plans, especially testing and backup initiatives. Virtual environments should be treated the same as a physical server, showing the need for organizations to adopt more cross-platform and cross-environment tools, or standardizing on fewer platforms.
“This year’s Symantec-sponsored research clearly identifies key issues, hidden risks and best practices in implementing disaster recovery plans. While some aspects are trending well, the impact of downtime is greater than ever before,” said Rob Soderbery, senior vice president of Symantec’s Storage and Availability Management Group. “The surging cost of downtime places greater emphasis on business – which means more pressure on IT. If organisations are not protecting virtual environments, not testing their disaster recovery plans and seeing one out every four tests fail then something needs to change to better manage risk to the business. Organizations should implement solutions that address these needs while allowing them to leverage existing assets.”