Examining the Common Criteria Toolkit
23 October, 2004
category: Library
by John Morris, president and co-founder of Corsec Security, Inc.
In our last two articles, we covered FIPS 140-2 and Common Criteria at a high level, and then looked into what’s actually involved in getting a FIPS 140-2 validation. In this article, we’ll delve further into Common Criteria evaluations, what’s involved, and how they are used by vendors and purchasers.
What exactly is Common Criteria?
Common Criteria is an internationally accepted language for writing security standards, by which a vendor describes their products’ security functionality and then offers proof that it actually delivers those features specified. Common Criteria can be applied to hardware, software, or firmware products, alone or combined. Again, the evaluation does not focus on the entire product, but only on its security components as outlined by the vendor.
Unlike FIPS 140-2, which is primarily a U.S. and Canadian standard, Common Criteria (or CC) is an international standard (ISO/IEC 15408) used all over the world. It provides an accepted framework for defining security features and evaluating against them – a toolkit if you will, used to build security standards for all types of products. An international Mutual Recognition Agreement (or MRA) allows many countries to perform CC evaluations of security products using this toolkit, as well as to recognize CC evaluations performed by other countries participating in the MRA.
What do all those acronyms mean?
If you’ve experienced Common Criteria before, you’ve not doubt noticed it makes liberal use of acronyms. For instance “CC” refers to Common Criteria, and “MRA” refers to the Mutual Recognition Agreement which makes CC internationally accepted. Many evaluations are done following a specific “PP” or Protection Profile, which is a set of requirements which may be met by a vendor’s “TOE” or Target of Evaluation and described in an “ST” or Security Target. Think of the PP as being like a request for proposal (RFP), and the ST and TOE as the vendor’s answer to that RFP.
In practical use, a purchaser participating with the MRA (such as a national government) would use CC to specify their security requirements for a particular product type (such as a firewall). The specification for that firewall would be written according to the CC language and published as a Protection Profile (or PP). The vendor then answers the PP by documenting their TOE and having it tested accordingly (See? Now you know CC speak, but if you are still confused, more CC terms are defined here http://www.corsec.com/ccc_terms.php).
How is a Protection Profile used?
A Protection Profile is a set of requirements for a product type, which has been evaluated and published according to the CC. Anyone can author a PP, but it is expensive and time consuming to do so. Therefore, once a PP is published for a particular product category, it tends to be the one that everyone follows for that product type when being evaluated for CC.
Depending on their customers’ requirements, vendors can choose to evaluate their products against a published PP, but following one is not mandatory. In fact, the majority of CC certificates issued do not comply to a specific PP. However, preference is often given to those products that are validated against one, because it specifies an easily recognized set of requirements accepted according to the CC standard.
For example, the U.S. government could publish a firewall PP which might be used by a dozen different nations in selecting which firewalls to buy. A vendor from France could then be evaluated against that PP, and sell their product into thirteen different nations (that’s a lucky number in France, I hear).
Remember that any group can set the requirements for a particular PP for a type of product. PP publishers can include U.S. agencies, foreign governments, commercial vendors or targeted user groups (such as smart card users or financial institutions). If there are enough purchasers who buy only products evaluated against a particular PP, then vendors will be more motivated to have their products evaluated against those requirements. Thus, the purchasers ultimately set the rules on complying to PPs through their enforcement (or lack of it) of purchasing products evaluated to them.
What is the purpose of the CC program?
CC evaluations are designed to let purchasers know if commercial products that provide security services actually meet their requirements (such as those defined in a PP) with a certain level of assurance. It’s difficult or impossible for end consumers to dependably evaluate whether a product correctly implements dozens of security details from design, development, and delivery, through dealing with bugs after install. Therefore, a CC evaluation will allow a vendor to know that a product does these things within a specified level of assurance. How much assurance depends upon how much proof the vendor provides according to a specific Evaluation Assurance Level (or EAL).
What exactly is an EAL?
All CC evaluations are tested against a specific assurance level, or EAL. Typically, a PP will specify which EAL is needed to guarantee the security functions specified for that product (although there are seven total levels of assurance possible in CC, only EALs 1-4 are tested by the labs and mutually recognized under the MRA, while EALs 5-7 must be tested directly by the scheme issuing the certificate). It is important to note that although the higher EAL levels are much more stringent, they do not necessarily mean that a product is more secure. To the purchaser, the higher EALs indicate that the user can have a higher level of assurance that that product does everything it claims to do from a security standpoint. To the vendor, the higher EALs indicate a greater commitment to achieving CC, since the documentation, time, and expense required for testing increase in tandem with the assurance level.
What documentation is required?
A typical CC evaluation consists of 70% documentation and only 30% actual product testing, so the burden is on the vendor to produce an enormous amount of paperwork. In addition, the documents needed must follow an exact format specific to CC, and things like grammar and spelling count! Even a seemingly insignificant mistake can cause the document to be rejected by the lab. Since much of the information is proprietary, there are not a lot of published examples to follow so the whole process can be quite overwhelming. To complicate matters further, laboratories might alert the vendor that there is a problem with their document, but not notify them as to exactly what the error is. The subsequent iterations between the lab and the vendor cause the greatest amount of delay in the testing process, so the better prepared the documents are, the more likely the evaluation will stay on schedule and budget.
Who measures the results of a CC evaluation?
After a lab has been satisfied that the vendor has indeed provided assurance about the security features of their products according to CC requirements, they send a report to their country’s scheme to issue the certificate. Each MRA member nation can decide to become an evaluation certificate producer with their own scheme (although not all of them do). Each scheme accredits independent testing laboratories, and oversees the testing of submitted products before issuing CC certificates to participating vendors.
In the United States, the national scheme is called NIAP (National Information Assurance Partnership) and consists of cooperative efforts between NSA (the National Security Agency) and NIST (the National Institute of Standards and Technology). It’s NIAP who sets the U.S. rules for U.S. testing laboratories (run by commercial companies) and it’s NIAP who oversees the evaluation of products in the U.S. All other MRA signatories have agreed to accept the results of NIAP’s certifications, just as they have agreed to recognize certificates issued by other schemes (the MRA only recognizes evaluations up to EAL 4). Within the participating MRA nations, there are eight other countries that have schemes and produce certificates (CC schemes are located in the following countries: U.S., U.K., Canada, Germany, France, Japan, Australia and New Zealand). The list of schemes, and links to their websites can be found at http://www.corsec.com/ccc_links.php.
Who participates in the MRA?
When the CC was adopted in 1996, the MRA consisted of the United States, the United Kingdom, Germany, France, Canada, and the Netherlands. Today, as many as 20 nations have signed the MRA including Australia, New Zealand, Austria, Finland, Greece, Hungary, Israel, Italy, Norway, Spain, Sweden, Turkey and Japan. It is important to note that many other countries follow the lead of the MRA without actually signing it. For instance, the European Union has published EU directives specifying the use of CC validated products by their 25 current member countries, thus making the appeal for CC that much more far reaching.
What does CC mean to the purchaser?
CC evaluations are useful tools for purchasers. They help to assure that the security of the products they are considering for purchase meet defined requirements. However, due to the complexity of the program and the plethora of options it’s sometimes difficult for purchasers to make effective use of the program. Currently, over 65% of CC evaluations are performed without conforming to a PP. Without conformance to a common PP that is useful to purchasers, it’s difficult to compare apples to apples, or even to quickly understand to what criteria each product has been evaluated.
So is CC worth all the effort?
CC entails significant effort for a vendor at any EAL. The vendor must define which security features they want to claim, what protection profile will be used (if any), what level of assurance makes the most sense for them to pursue, who will assist them with their documentation requirements, and which lab will perform the testing. Only after these decisions have been made can the actual CC evaluation begin.
With security issues at the forefront of everyone’s minds, a lot of vendors have recently been told that they must have their products CC evaluated to win a government contract, even if they have sold the product to the same agency in the past without it. That certainly encourages participation, and the proof of this lies in the increasing number of certificates issued each year. At the recent CC Users Forum in Washington DC, NIAP released figures stating that there are currently 126 products in-evaluation for CC, as compared to 76 total evaluations completed in the three year time period between 2000-2003.
Still, many vendors are surprised at the amount of documentation required at higher EAL levels, and are not adequately prepared for the amount of work, time and costs incurred by their laboratories. Often purchasers may be waiting for a specific product to complete a CC evaluation before they can purchase it, only to find that it takes up to a year to complete (to combat this, NIAP’s website publishes a list of products that are in-evaluation, and many purchasers can acquire products appearing on this list before the certificate is issued).
A common complaint of vendors is that by the time their certificate is issued, they may very well be on the next version of their product release. For some, the total amount of effort spent on a CC evaluation may not warrant the perceived assurance benefit provided to consumers. However, vendors really only measure satisfaction through return on investment, and thus will participate or refuse to participate solely on whether they will earn more in extra government purchases than they will spend on validation activities. Corsec (my company) works with customers to help them to make a well educated decision in pursuing validations, understanding the full costs and efforts, measuring the ROI, and planning a validation effort that minimizes time and costs.
Next month, we will cover the testing process for Common Criteria and how it relates to FIPS 140-2 validation. In the meanwhile, you can read more about both CC and FIPS 140-2 requirements at http://www.corsec.com, or contact me at jmorris at Corsec dot com.
Compare FIPS 201 ProductsResearch and evaluate FIPS 201 Approved Products and get the latest info on compliant credentialing systems at FIPS201.com. Click to visit FIPS201.com.
