Scoping FIPS 140-2 and Common Criteria Validation Efforts

26 November, 2004

by John Morris, president and co-founder of Corsec Security, Inc.

In our last articles, we covered the mechanics of FIPS 140-2 and Common Criteria (CC) validations, and this month we will discuss the testing processes for both, as well as how to effectively scope the efforts required to achieve these validations.

The scope of effort required to perform government validations is one of the most commonly misunderstood aspects of the process, and often the costliest. Government consumers may not understand the vendors’ reluctance for pursuing FIPS 140-2 and Common Criteria, especially for entire product lines. Vendors in turn may underestimate resources, budget, and time when promising future validations to their government customers. Even laboratories and government evaluators can be overly rosy about project scopes, times, and costs. So why is accurately scoping validation efforts so elusive? To understand that, we need to examine the validation process and several often-overlooked areas of cost and effort.

1.1 What Type of Validation is Needed?
Vendors have many choices to make when beginning FIPS 140-2 and/or Common Criteria, but those choices can often mean the difference between difficult or easy validation efforts. The first and most difficult choice is usually whether FIPS 140-2, Common Criteria, or both are required for a product.

Regardless of whether or not a vendor pursues Common Criteria, a FIPS 140-2 validation is required by law anytime a U.S. federal agency wishes to purchase a product that does any kind of encryption. In a Common Criteria validation, the cryptographic part of the product must be FIPS 140-2 validated in order to pass the CC evaluation as well. Therefore, if a government purchaser is demanding that a vendor provide Common Criteria for their security product, this could very well mean that they will have to pursue a FIPS 140-2 validation in order to meet the requirement for CC as well. On the other hand, a FIPS 140-2 validation alone may satisfy some customers’ requirements, making careful consideration of which validation to pursue critical.

1.2 Consider All the Options
Once a vendor decides to go after either FIPS 140-2 and/or CC, they must next choose which level of validation to target. The purchaser will often specify what is most appropriate on a client by client basis, and the vendor must then determine if one client’s request can translate over to meet other customers’ needs as well. Other factors, such as what the competition is offering, should also be considered. How fast the validation is needed is also a big factor is deciding which roadmap to follow. Some vendors may choose to pursue a lower level validation to begin with, and then build on that by following with a higher assurance level the next time the product is evaluated.

With Common Criteria, the vendor must also decide if they will adhere to a specific Protection Profile (PP) or not. Adherence to some Protection Profiles can mean a more difficult validation, with extra government oversight and review. Customers will often request vendors adhere to rigorous PPs or pursue high Evaluation Assurance Levels (EAL), but may not appreciate that those requests may double both the time and cost of the validation. Similarly, vendors may pursue FIPS 140-2 validation of very complex product lines, when they might more efficiently validate a single common cryptographic module instead. All of these things must be factored in when scoping the project in order to ensure the most efficient path possible.

1.3 Time Keeps on Ticking
The timeframe for validations depends on many factors, some of which are often overlooked. FIPS 140-2 validations typically take from six to eighteen months to complete, and Common Criteria (CC) evaluations can take even longer typically lasting from eight to twenty-four months. This ugly reality of slow validations is often masked by the success stories from products that are taken through complex validations in less than six months. Often, the only difference between a speedy evaluation and a never-ending project can be the validation path chosen and the experience level of the parties involved.

1.4 Changing Products, Moving Targets
One of the most difficult aspects to deal with in any validation is synchronizing product development timelines with validation schedules. Major hardware vendors often release new versions of products quarterly, and some software vendors release new versions even more frequently. Obviously this makes timing a six to twenty-four month validation effort problematic, and can cause frustration and delays. If a schedule was underestimated or suffers unexpected setbacks, a vendor may end up changing the version of the product under evaluation, which could introduce further delays. The single greatest factor in validation times is often product changes. Fortunately, experience with the validation programs can allow validation activities to be scheduled with product development, shortening the overall efforts, and resulting in validation soon after product release.

1.5 Negotiating the Right Testing Lab
Once the vendor has chosen which type of validation is needed, what level to pursue, and which version of their product to test, they must next contract with a testing laboratory in order to validate their security claims. Due to the wide variation among the labs (including experience evaluating a particular product type, their pricing strategies and timeline schedules), this can be a major accomplishment in and of itself. It is crucial to have key milestones, timelines and clear costing schedules negotiated up front and in writing. Otherwise, the vendor may experience scope creep on an hourly T&M basis, which delays the effort while also bleeding the budget dry.

This is one of the areas that an experienced consultant can help the vendor with, as they will know what is reasonable to expect throughout the process. In addition, a consultant will be able to keep all parties on track and advocate the vendors’ position as compared to blindly having to deliver every change suggested by the lab. As with any beaurocratic process, the standards may leave room for interpretation and this is where inexperience can become costly for everyone involved.

1.6 Specialized Validations, Specialized Documentation
Once a laboratory has been chosen, negotiated and contracted with, it is time to produce all of the required documentation, which can be the most overlooked aspect of the process. Both FIPS 140-2 and Common Criteria require specialized documentation written to satisfy their respective testing requirements. Although the authors of the standards may have intended normal product documentation like functional specifications, architecture documents, finite state machine (FSM) models, and product manuals to supply the required evaluation evidence, the testing programs do not work that way in practice. Instead, all commercial products require specialized documents written using specialized terms. Documents like a Non-Proprietary FIPS 140-2 Security Policy, Security Target, FSM, High Level Design, and Crypto Officer Guidance are often written or re-written specifically for a validation effort.

1.7 Setting Reasonable Expectations
The first evaluation that a vendor goes through can be the most difficult, because they may be engaging in the cumbersome validation documentation production for the first time. From creating documents, to commenting source code, the scope of effort may be more than they had envisioned. Government purchasers need to keep in mind the difficulties and costs that vendors face in achieving validations. However, this cannot be an excuse for vendors to fail to pursue the evaluations required.

Both vendors and purchasers must work together to set reasonable expectations throughout the process. Expect vendors who have begun a first validation to struggle, and look for positive signs in their story, such as engaging experienced consultants and signing contracts with laboratories. Expect initial schedules to be tentative, as vendors work through their scoping efforts, but look for steady measurable progress like appearing on in-evaluation lists, and completing portions of validation such as algorithm testing (FIPS 140-2), or entering testing (CC). If a clear plan is established ahead of time and adhered to during the process, then the effort will be a success for everyone. Do not ask for more than can be delivered and do not be afraid to ask for help. If you have any specific questions, feel free to send them to me at jmorris at corsec dot com, or visit the resource centers on our website, http://www.corsec.com.

Compare FIPS 201 Products
Research and evaluate FIPS 201 Approved Products and get the latest info on compliant credentialing systems at FIPS201.com. Click to visit FIPS201.com.

Related articles: