A few months ago I was asked to provide some detailed statistical information about the “precise rates of occurrence” and the “precise costs of computer crimes and other security incidents” so that a financial department could calculate the Return on Investment (ROI) for some planned security projects.  As anyone who has been in this position will tell you – there is trouble on the horizon when this question is asked. 

Unfortunately in today’s economy, the question will be asked again and again in and around the boardrooms and hallways of many companies around the country.  Now more than ever there will be pressure to trim the budget and unfortunately Risk Management and Information Security are often the first places people look to trim. 

In cases like this I think that it is just as important how we respond to these inquiries as it is with what we are responding with.  At my client’s suggestion, what follows is an excerpt from a follow up email that I sent to the client CFO regarding the question of Security ROI.  We initially covered this material in a face to face meeting that went very well.  To date the meeting was successful as my client has told me that the finalized budget for next fiscal year has come out and his information risk management projects have been funded at the levels he requested.  (Both my client and their CFO have approved the following as it is generic and does not identify them or their industry in any way.)

FROM:           Graydon McKee

TO:                  CFO

SUBJECT:     RE: Hard figures for ROI on security

Mr./Ms CFO,

Thank you for your time earlier today.  I appreciate the opportunity to place some context surrounding the calculation of ROI when it comes to risk management and information security activities. 

To summarize our discussion:  Detailed statistical information within the field of information security is unreliable and woefully inadequate to achieve appropriate levels of confidence for use in return on investment (ROI).  ROI can be calculated however it must be understood that, in the case of information security, any resulting figure will be the result of both a qualitative and a quantative assessment as opposed to a clear statistical exercise.  Allow me to discuss the issues involved . 

Ideally, in order to calculate ROI we would want to evaluate our level of risk by comparing ourselves to other organizations with similar systems and business characteristics.  Unfortunately, this information is not currently reliable.  Two of the fundamental difficulties that exist are in the nature of computer crime detection and reporting. 

The most appropriate illustration of the problem is that of the Defense Information Systems Agency and the studies they conducted between 1994 and 1996.  They performed 68,000 system penetrations during that period and found that two-thirds of their attacks succeeded.  Of the two-thirds or approximately 45,000 attacks, only four percent were even detected.  Of those detected, only a fraction of one percent were even reported. 

Now by all accounts our detection capabilities have improved since 1996.  What is more telling is the number that were reported.  Let me break down those number.  Four percent of 45,000 is 1,800.  Of the 1,800 detected attacks that were detected less than 18 were reported (18 being one percent of 1,800). 

These numbers are still consistent with the commonly held view within the Information Security field that only one–tenth of all crimes committed against or with a computer are detected.  Of those detected only about ten percent are actually reported.  This would render the reliability of existing statistical data virtually useless with regard to the determination of rates of occurrence.  All this said there is a positive note that I’ll get to shortly. 

Another issue is with the studies themselves.  In order for the studies to provide real data upon which we can run statistical analysis, we need to determine that we have both an appropriate sample size as well as assurance that the sample is truly random.  The very nature of the studies lends itself to bias in that the individuals and companies which are more likely to respond to such surveys, can be realistically assumed to be more security conscious and therefore be paying more attention to security.  This creates a bias within the survey data itself.  In addition, the existing surveys do not track the same respondents from a year to year basis thus making correlation from year to year tenuous at best.   

The final issue that I’ll point out is the issue of Association versus Causality.  Some studies can be misinterpreted by equating the presence of technology with good security.  Technology deployed by highly skilled and knowledgeable staff can result in greater protection and therefore a greater ROI over time but the same technology installed by less skilled and knowledgeable staff can actually increase the risk of a security event occurring.   Too many variables exist to rely on such association judgments.

Determining the Return on Investment for Security is something that has plagued the industry from its very inception.   Many different methodologies have come and gone with various degrees of success but at the core of each of them lies a significant amount of subjectivity and assumption. 

The positive note that I mentioned earlier is this.  While we may not be able to rely upon information security studies to provide us statistically reliable data upon which to base ROI calculations, we can rely upon it to show us trends.  We can use this trend data to help focus our attention on areas of increased risk.  This is qualitative analysis. 

If we combine this qualitative analysis with a quantative measure of ROI (acknowledging the fact that we are using insufficient data at best) then I believe that sound risk-based and cost-based decisions can be made. 

** Let me conclude this post by making one final statement.  While I acknowledge the issues surrounding information security surveys I don’t want to give the impression that I think they are useless.  I don’t – I actually think they are quite valuable.  I am cautious about using them to present precise figures though. 

These figures can be easily misinterpreted so when pressed to provide figures I always try to do so within a context.  In this particular case, the business cases for the security initiatives in question were accompanied with an attempt at calculating ROI.  This was done because it was a requirement of all funding requests within this organization.   The ROI calculations were built using a rough model that I put together based upon the ROSI work done by the New South Wales Government, some work out of Carnegie Mellon University, and a few other places.  The data used came from a few different surveys. Each data set is run separately through the model and then compared and averaged.  It isn’t perfect but it does provide a basis upon which qualitative analysis can begin.   It is my belief that in the end it became less about the numbers themselves than about the process.  By establishing the process and attempting to take a 360 degree view of the problem, solution, and the benefit the company would gain, senior management was convinced that we were making reasonable prudent funding requests. 

In part one we talked about some of the mistakes that companies make when it comes to risk management.  In this post we will focus on some of the items that a company can do to improve their risk management programs.   What I will lay out are three points/characteristics/aspects that a solid risk management program must have in order to be effective. 

Point One:   A common risk framework must exist throughout the organization, not just within one department.  This framework must:

·         Use a common definition for “risk;”

·         Support appropriate standards, regulations, guidelines;

·         Clearly define the key roles, responsibilities, and authority relating to risk management;

·         Support all of the business units and functions both in the way that these units accomplish their jobs as well as in the performance of their risk responsibilities.    

Many organizations recognize that risk means “the chance of something going wrong, hazard, statistical odds of danger” to quote the Encarta Dictionary.  What they forget is that there are positive aspects to risk.  Risk can be seen as the opportunity to create and preserve value.   

When I think of risk in this way an old saying comes to mind:

“When Life gives you Lemons, make Lemonade.”

In other words you need to create opportunity out of adversity.  Business is about risk.  There is no way to avoid it so why not simply seek to nullify its effects when you can leverage it to gain an advantage.   In my experience, the companies that embrace this concept of managing risk succeed not only in risk management but in the marketplace itself. 

Point Two:  Senior management must have the primary responsibility for the risk management program.  This means its design (it must be appropriate for the whole organization), its implementation (it must not favor one unit or function over another), and its ongoing operation.  Most importantly senior management must have complete visibility into how the organization (and each of its constituent components/units) manages risk. 

This means that risk must be coordinated across the entire organization.  Risk must be everyone’s responsibility; even those people who do not think they have any responsibilities with regard to risk.  True implementing technical security controls may be the primary responsibility of the IT department but in order for that implementation to be successful all departments and functions must share the responsibility.  IT needs to know if a particular control causes too much interference with the way the business is run so that they can make adjustments or implement alternative controls to reduce interference to a minimum.  The other departments and functions must realize that there are valid business reasons that these controls must be implemented. 

Senior Management needs to send the message that risk is a collective concern.  In order to do Senior Management needs to ensure that they communicate clearly and effectively.  They need to nurture a culture focused on risk (how to manage it and overcome it for the organizations benefit).  They need to institute a rewards program to provide positive reinforcement and they need to institute an effective learning program to educate everyone on what parts they play in the grand scheme of things. 

Point Three: Risk is an everyday concern and on every agenda not just on certain scheduled meetings. Each business units/function is responsible for the performance of not only their business and the management of risks they take.  This is important because it speaks to ownership and accountability. 

Not everyone is going to like this.  Honestly they don’t have to but they do have to climb on board and support the effort.  It is analogous to having to abide by the covenants in your homeowners association.  If you move into a neighborhood with a home owners association, then you agree to abide by the rules that the association agrees upon.  If you don’t want to do that then there are other homes that are not part of associations just as there are other companies to work in.  (Of course there are always rules set forth by the local, state, and a national government that we must abide by – that is part of living in an ordered society. )

Now not all business units or functions have the same scope when it comes to risk.  Some departments “own” risk management because they are the profit generating arms of the organization and other departments (such as HR, IT, finance, legal, etc) support these profit generating arms.  These supporting functions own the risk that arises out of their own area of responsibility in addition to sharing in the overall responsibility of supporting the overall organization.  It is very important (to harken back to Point One) that these functions have well defined articulated roles within the overall risk management program.  They must participate in risk discussions even when it is not clear that these discussions are directly related to them. 

I could go on but this post is getting a bit long already.   To sum everything up – risk is everyone’s responsibility.  Companies trade risk for reward daily so it shouldn’t be too large a leap to remind ourselves that the risks we face on a daily basis need not only be seen as a drag on the balance sheet.  They can be seen as opportunities to be leveraged.  Instituting a risk management program that pays attention to the three points that I have made above will do just that. 

What happens when a company experiences a data breach or a security incident?  Well aside from the obvious, someone often tries to figure out what went wrong.  Hopefully this means that they want to fix things so that it doesn’t happen again and not for finger pointing and assigning blame. 

Regardless of the reason, what can we learn from these events?  In this, the first part of a two part post, I’ll review the common mistakes that companies make prior to a security incident.  In part two we’ll discuss how companies should approach risk management in order to minimize their exposure to risk.  (Imagine that – a risk management program that actually manages risk. What a novel idea)

This list is in no way exhaustive or ordered by priority. This list comes from my experience and that of the other information security professionals that I have talked with over the years in my attempts to learn from other’s mistakes and experiences.  If you have other items that you’d like to offer for consideration please post a reply.

1.  Risk Management is diffused across the entire organization. 

Managing risk should be everyone’s responsibility but it should have a focal point and a champion.  One problem that occurs is that risk management activities are carried out my many different people from many different departments with little or no coordination between them.  This causes a repetition of effort and can actually create more risk than it actually addresses. 

2. Overlapping and interacting risk factors are often underestimated or ignored all together.

Much like the diet drug Fen-Phen, the interaction of two risk factors can have an exponential increase in the level of risk exposure.  In companies that experience security incidents the interaction between these factors are often ignored if they are even recognized in the first place.  When the interactions were highlighted by information security professionals, senior management often downplayed the interaction.  We can only speculate as to why.

3.  Warnings about security vulnerabilities and risk agents were ignored and those who gave them were criticized as malcontents or for not being team players.

When examining the events that lead up to a security incident, it is not uncommon to find that the warning signs were there.  In certain situations it wasn’t uncommon to learn that those who did voice the warning were criticized for what management considered “disruptive behavior.” 

4.  When risk modeling is used too much emphasis was placed on probabilistic modeling. 

Most security studies are highly inaccurate from the standpoint of the quantifiable measurement of security incidents.  (I can go on about this but it is really a separate topic.  If it is one of interest to you let me know and I’ll devote a post to that topic.) Most information security professionals believe that these studies only capture something like ten percent of the actual events that are occurring.  When you use these studies upon which to base probabilistic risk models you are doing so using inaccurate data.  This is fine as long as this fact is acknowledged and the numbers generated from the model are tempered with qualitative analysis. 

5. Senior management was so focused on making their numbers that other programs and initiatives (such as risk management and information security) were cut.

This problem is all about thinking tactically or strategically.  A long term strategic approach includes addressing the needs and requirements that can have the greatest impact over time.  If managements view is too tactical and short term then they run the risk of neglecting the long term concerns such as those having to do with appropriate risk management activities. 

6.  Companies lacked a comprehensive approach to risk management. 

A comprehensive approach takes into account quite a few different aspects and points of view rather than one or two narrow views.   Companies that lacked a comprehensive approach typically viewed risk management as a compliance exercise rather than as a business enabler.  I’ll go into more detail about this in Part Two so stay tuned. 

Again, these are just a few items that came to mind.  If you have your own that you have noticed then please share them.

Bill Brenner, Senior Editor over at CSO Magazine has a great Podcast where he covers a recent security gathering in Boston, MA.  The one that captured my attention is a summary of some Forrester Research study on the increase in security spending.  According to their research, and I’m just going on the information from the Podcast,  I haven’t read the paper yet, FUD has fueled an approximate 10% increase in information security budgets – a 2% increase over last year. 

If you have the time – and you only need seven and a half minutes – it is worth your time to listen to Bill Brenner’s Podcast. 

NetworkWorld just released an excellent article on the impact of Cybercrime on businesses.  Now this article isn’t technically oriented so it is perfect to print out and send to senior management to highlight the need to properly secure company information. 

The article showcases the efforts that computer and electronics retailer TigerDirect goes through to combat credit card fraud.  TigerDirect’s system is homegrown and looks to flag online transactions that originate from countries known to be hotbeds of credit card fraud and online anonymizer sites.  These transactions can then be investigated further (by either calling the customer or the bank) to determine if they are legitimate. 

The one item in the article that I disagree with is the following quote:

In spite of caution and preemptive actions, TigerDirect will still get hit by costly card-related fraud each year through a small percentage of bad sales — which the retailer absorbs, not the victim of the stolen card. “It costs us millions and it costs the industry billions,” Fiorentino says.

(Gilbert Fiorentino is the CEO of TigerDirect)

The retailer does not absorb the cost.  The customer does.  Every product that is sold carries a mark-up that includes a percentage of the retailer’s overhead costs as well as the profit margin.  Now you may not be able to find a fraud markup on the balance sheet but rest assured it is there typically hidden in some sort of overhead figure. 

Consumers carry the burden of fraudulent activity as well as other increases.   Just look at the cost of Milk.  The New York Daily News ran an article earlier this year on the cost of Milk jumping 36%.  This price increase was attributed to the increase in demand on the feed corn.  The corn is being used to meet the increased in demand for ethanol therefore it is becoming more expensive to purchase.  Just as these increases in costs are impacting the price of the final product (the milk) so do increased overhead costs impact the retail prices of all other products. 

Now if we can work to reduce the cost of fraud by implementing appropriate security measures then we can contribute to lowering costs for consumers, or by increasing the profit margin for the company.  Either way it is important to tie information security to these final end results as it could very well help you to make the business case for information security.