There is a great post over on the Guerilla CISO.  The post is a guest post by a friend of mine and wonderfully illustrates how seeking to be merely compliant with current regulations can sometimes lead to problems when faced with a crisis. 

I won’t take any wind out of the author’s sails by commenting further right now.  Check out “Could the Titanic have changed course?” over on the Guerilla CISO.

I received this question the other day:

My CFO and I were discussing compliance with SOX and the GLB the other day and we got into an argument about whether we should be doing security assessments or security audits as the best way of meeting our obligations under these laws.  Can you help me with what I should say about the differences and provide some guidelines on how my company can use these appropriately to meet our compliance requirements?

The lines between auditing and assessing are often blurred.  In the strictest sense, an audit is a measurement of compliance with a standard or a set of criterion and an assessment is a measurement of effectiveness.  Using these strict interpretations, an audit can be looked upon as objective and an assessment can be viewed as subjective. 

Let me place each of these activities into context as perhaps that will shed some light on the differences between the two activities and how they can be blended thus causing the confusion that you experienced with your CFO. 

Audits typically can be imagined as a series of Yes/No or multiple choice questions: “Does the organization….?”  The answers to these questions are designed to highlight areas of non-compliance or gauge how far from compliance an organization is. 

Let us take a look at an example:

“Does the organization have a policy concerning the separation of duties?  Yes/No”

This is a typical audit question.  This question simply asks for the presence of a policy that addresses the separation of duties.  In order to verify this, an auditor would need to see a copy of that policy. 

An assessment would go a bit further and ask to see procedure on how the policy will be implemented and then interview multiple people in order to ascertain if the policy and procedure is being followed.  A judgment is then made concerning how effective the control is (the control being a policy on the separation of duties). 

Audits tend to weight answers to questions in an attempt to also reach some sort of judgment concerning the level of compliance.  Personally, I have an issue with this practice. 

Audits strive to be objective measures.  When the answers to an audit questionnaire are weighted, the auditor is actually making judgments concerning how important certain questions are in the attainment of compliance.  This provides a very subjective foundation to what is suppose to be an objective exercise.  (In my humble opinion of course)

Assessments on the other hand are intended to be subjective and opinionated.  Assessments attempt to place their subject into context.  Referring back to the question above an assessment statement might look something like this:

“Organization XYZ does not have a specific policy on the separation of duties within their network environment.  The organization indicates that funding concerns have limited the size of the network support staff making this control impractical.  In order to compensate, a detailed auditing and review program has been instituted.  Audit logs are reviewed daily and serve as an adequate compensating control.”

The difference between auditing and assessing computer systems is rather clear when using strict definitions of audit and assessment however in practice the line between the two has become blurred.  The benefits of each are often confused thus leading to disagreements such as you experienced with the CFO. 

I prefer to utilize both activities in their strictest sense.  I audit to determine where I am compliance and where I am not compliant.  I also then expand upon these findings to ascertain how well this level of compliance is actually securing the computer system.  I can then make two statements concerning the system: its level of compliance and how effective that level of compliance really is in terms of achieving the goals of the regulation being applied. 

I hope this helps to illustrate the concept.  Please keep those questions coming. 

The following is a piece that was originally written for Network World’s Security Newsletter and published in March of 2006.  It was a collaborative effort between Joe Faraone, a close friend, and I.  The issue we touched on really hasn’t gone away so I thought that I’d dust it off and update it a bit. 

Sometimes we hear senior managers and executives expressing frustration with government regulation by saying that the issue has come down to a choice of either being secure or being compliant.  In fact Compliance remains the number one driver of information security as reported in the 10^th Annual Global Information Security Survey conducted by Ernst & Young. 

This is consistent with the article that sparked the original version of this post.  That was an Information Security Magazine article from October 2005 in which security managers were asked to respond to the question, “What is your biggest obstacle to implementing and managing security related government regulations?” Responses reveal that the top two obstacles faced were unclear compliance related responsibilities and interpreting regulatory language. This point of view appears to be more prevalent in the private sector than the public sector.

One of the reasons for this discrepancy is that the public sector must follow a specific framework for measuring the maturity and effectiveness of its information security programs: certification and accreditation (C&A). The Federal Information Security Management Act of 2002 (FISMA) was intended to provide for the development of and maintenance of minimum controls required to protect information systems and to provide for a framework for ensuring the effectiveness of these controls. While many of the overriding principles followed in C&A are contained within the law, nowhere are the words “certification” or “accreditation” found.

FISMA points to the Office of Management and Budget (OMB) as well as the National Institute of Standards and Technology (NIST) to obtain guidance. OMB has issued its Circular A130, which requires that all federal information systems to be certified and accredited following guidelines developed by NIST.

Now the C&A process has gotten a lot of bad press, some of it well deserved but coming from someone who has worked with the process in one form or another for the past ten years I’d say that it comes down to a matter of implementation rather than issues with the process itself.  If the process is viewed as just another paper exercise intended to satisfy auditors then it is a waste of time but it is also missing the forest for the trees. 

NIST has done a laudable job of developing and revising this guidance. It provides a methodology to fully document, measure, assess, track and report on the health of information systems from the aspect of security. These guidelines show how to integrate an information security program with the systems development lifecycle as well as how to test the implemented controls.

Those in the private sector are probably wondering why this should be important to them.  Many argue that the control sets mandated by the process are too much for a private sector environment.  Implementing the full set of baseline controls would be too costly and provide little ROI others say.  Again, my response is that you’re missing the forest for the trees.  The process itself is what is valuable here and is flexible enough to allow any set of requirements to be utilized not just the set of baseline controls provided by NIST. 

NIST publications and the methodology for conducting certification and accreditation are freely available and constitute an untapped publicly available security resource. Inputting the government regulations (Sarbanes Oxley, Health Insurance Portability and Accountability Act, etc.) into this framework allows the private sector to document, measure, assess, track and report upon the security posture of their information systems and how well government regulations are adhered to. The private sector can assess the maturity of their information security programs and determine how well these programs integrate into their overall business processes.

What is key for the private sector is that the process must be tailored to your environment and needs.  Herein lies the problem that has plagued C&A from its beginning – it is often applied improperly. 

When the emphasis is placed on being compliant, people go through the motions and focus on technology and checking boxes rather than leveraging the power of the framework to assess the effectiveness of their programs.

The two most basic elements of any system are often overlooked or underemphasized: the information being protected and the people who use the information. You can put in all the high security devices you want in a system, but if you do not account for the people who need to use the information system and the criticality of the information within the system, you still will not be secure.

If the C&A process is improperly applied then it does result in a lot of wasted time and paperwork. If it is properly applied then it becomes a wonderful tool to assess the effectiveness of your security controls – everything from policy and procedure down to control functionality and configuration.  It provides a holistic view of the network and security with the emphasis on being secure. Compliance is simply a milestone on that journey.

The beauty of this is that the information that you need to implement this framework is _free_ and fully available at the NIST Web site. Do you need to hire high priced consultants to come and set this up for you? No, you don’t. Although consultants can save you some time on the learning curve, the guidance available through NIST will allow you to begin the process on your own. You can then use consultants to give you an independent review of your program or to bolster areas where you might feel less comfortable. But remember that you must tailor this framework to fit your environment – use what works and make sense and discard that which does not.  (Sorry government readers – this doesn’t apply to you.  You don’t have the same latitude to do that as does the private sector.)

The CIO implementing this approach can concentrate on the details of how information is protected and used rather than scurrying about wondering how to bring order to the new herd of cats that legislation has unleashed.

Take the framework that NIST has so diligently given us, plug in the requirements that you are subject to, and then sit down with your network architects, your user representatives and your key project managers and find a way to work efficiently but securely. With the NIST framework, you will be able to assess, measure, track, and deliver a more secure and user friendly network and in the process, achieve compliance.  I have done this with amazing results so I know for a fact that it works. 

Alternatively, keep enjoying your view of the forest.