Over the last 20 years of doing IPPAs, a common process and practice has emerged. This produces relatively repeatable results, subject to minor variations in opinions, given that those performing the process are suitably knowledgeable and put forth the time and effort to do the job properly. The basis of the certification program is providing a way to verify that those performing IPPAs are properly qualified and use the process effectively to generate these sorts of consistent results.

Why are IPPAs so confidential?

One of the reasons IPPAs are so highly confidential is that they are internal assessments that are very specific to internal issues, threats, vulnerabilities, consequences, and time frames with sequencing for spending monies in mitigation, acceptance, transfer, or acceptance. The revelation of such information can be potentially devastating to an organization, if it is revealable insiders will be highly hesitant to tell it like it is, and the fact that such a document exists subjects it to subpoena and other similar actions that could be dangerous to the enterprise. Unlike an audit that is designed for external consumption, and IPPA presents a wide array of facts and opinions associated with individuals and adds expert opinion and interpretation. IPPAs are not done against a standard, but identify likely outcomes if standards-based audits were undertaken. Companies that do IPPAs may not reveal customers of these IPPAs to others under any circumstance.

How to do a Protection Posture Assessment

People who ask for assistance in doing IPPAs often want to do the same job with lower cost people. They ask for checklists and all sorts, special tools, of other short cuts, but in the end, this is just not how you do a really good protection posture assessment. That is not to say that you cannot do a lower quality assessment with things like checklists and, in fact, many companies offer such a service. But this is not how you do an IPPA properly. The proper approach is described here.

An overview

An IPPA is a complex process involving a group of intelligent and experienced people who get at the issues and explain them clearly. While there are some stylistic components, at the heart of the matter is a set of people who know what they are doing in their fields and are not afraid to say what they think.

How many people and days?

An IPPA is typically a 30-day process involving a collection of 3-7 individuals with proper mixes of backgrounds and involving from 15 to 30 individuals from the client company. Team leads tend to be very experienced senior people while other team members may have less general experience but specialized backgrounds in areas closely aligned to the efforts undertaken. The total amount of person days in a typical IPPA runs from 40 to 150 depending on the depth and breadth of the overall project.

Report length and content

Resulting report lengths typically run on the order of 40 to 150 pages, corresponding roughly to one page per person day. Reports contain limited amounts of listings and similar technical detail, those being used only to demonstrate specific issues with greater clarity than would be attainable otherwise. Where possible such listings are limited to example cases and explained in detail.

Reports generally consist of (1) an executive summary of 1 page in length - in rare cases it may be 2 pages, (2) a table of contents, (3) an overview of the report and its findings, (4) a details section containing descriptions of what was done and observed where and when and a description of all interviews and what was indicated in them, (5) comparison to standards including the IPPA assessment standard, ISO17799, GAISP, CMM-SEC and other standards as may apply, (6) findings including urgent, tactical, and strategic time frames, what should be done in those time frames, and expected costs and complexities, and (7) summary and conclusions.

All reports are custom written with only pre-defined overall format and tables included for the standards-based assessments. Relatively standard language is used for certain sections like the introduction to each standard which includes a one sentence description of the standard and one sentence description of how it is used in the IPPA, but other than these few sentences, the whole report is written from scratch every time and no standard descriptions may be used. This implies that skilled authorship must be involved, copy editing and review is necessary, and any attempt to pull out results from previous reports should be shunned and are almost certain to result in bad advice, poor clarity, and take longer than doing the job right in the first place.

Areas covered

Areas covered in an IPPA include but are not limited to; protection management, protection policy, standards, procedures, documentation, protection audit, technical safeguards, incident response, testing, physical protection, personnel issues, legal considerations, protection knowledge levels, protection awareness, and organizational suitability. These are described in more detail in the material on all.net including but not limited to the New Security Database, the Feature Articles area, and the "Protection and Security on the Information Superhighway" book.

Personnel

The information protection posture assessment requires a team of expert personnel with general knowledge and experience in a wide variety of areas, and special expertise in all of the aspects of information protection. Specifically:

• Consultants with 10 or more years of professional experience in the areas covered by the assessment are required in order to assure that their breadth and depth of knowledge is adequate. Since no single individual has this much experience in a broad range of areas, a team of well-seasoned professionals is required. The use of a team also implies the ability to work together and the use of a team leader who is in charge of writing the assessment document. Because of the nature of this sort of study and the requirement for the team leader to make technical and management decisions, this person must have substantial management and consulting experience and a great deal of expertise in all areas of information protection.
• Because this sort of assessment is done over a very short time frame and covers a wide range of topics, it is best done by a team of 3-7 people who have worked together before. Because this assessment involves the top-level people in the organization under study, the team members must be more knowledgeable about protection issues than the people working for the client in order to justify the use of outside expertise. This means that the people must be very good at what they do. They need the necessary experience in the proper set of interrelated areas, they have to be able to relate to the client, and they have to be able to get the necessary information quickly, efficiently, and without offense.
• Because the study must involve issues of protection, understanding of the computing environment, detailed understanding of the hardware, software, operating systems, networks, telecommunications equipment, standards within the industry, physical environment, personnel issues, business operations, and other similar matters, it is important that each team member have a thorough background in computer hardware and software, a good working knowledge of how systems and networks work, understanding of the historical context of many current technologies, and deep understanding of information protection issues across the full spectrum.
• At least one team member should be a specialist with more than 10 years of experience in the specific industry being studied or experience in similar studies for others i9n the same industry. This background should include consulting work for a variety of companies in the industry, a good working knowledge of the standards and practices used by others in the industry, intimate knowledge of the hardware and software platforms used in the industry, their shortcomings, their features, their vulnerabilities, and methods used to protect them.
• By its very nature, the protection posture assessment is a qualitative analysis, not a quantitative one. For this reason, it depends heavily on the quality of the people doing the assessment. The people must not limit themselves to things like checklists or the result will almost certainly be poor. The people doing these assessments must understand many different viewpoints that may exist in the organization and reconcile the meaning of what different people say. They have to be able to judge the people in the organization by the way they act and make qualitative decisions about how to weigh their words. It is often necessary to ask a lot of questions in order to get a vital piece of information and, because each of the people interviewed has different biases, it helps to have experience interacting with similar people in similar situations.
• The people who do these assessments must be hard to push around or lead astray. This is hard to find in a large corporation because there is a tendency to promote those who don't make waves. By its very nature, such an assessment must make waves in the organization being assessed. If it does not make waves, either the organization was already very well protected, or the study didn't find the things it was supposed to find. That means that the people doing the assessment must expect and be able to deal with some hostility, lies, bizarre behavior, and other such things. It also means that they are there to make waves of a particular sort for a particular group within the client's organization. People who are afraid of losing their jobs by making a misstatement with a client are not going to do a good job of getting the necessary information.

Coverage

The areas covered include organizational perspectives as identified above. However, coverage also goes to issues of (1) business, system, people, and data lifecycles, (2) objectives including integrity, availability, confidentiality, use control, and accountability, (3) Defense processes including deterrence, prevention, detection, reaction, and adaptation, (4) business functions including but not limited to people, sales and marketing, public image and brand, processes and workflows, the transformation of resources into value, financial systems and mechanisms, (5) technologies in use, and most importantly (6) risk management processes and practices. Coverage also takes runs at high-valued vertical application environments from the physical components through the users, looks at interdependencies associated with everything from people to continuity of government, addressed business continuity and disaster recovery issues, and looks at risk aggregation issues. In other words, it is a full spectrum examination at a level of granularity defined by the context of the effort.

Another important issue to be understood is the extent to which depth is pursued in different sorts of protection arenas. At what level of depth and against how many locations is dumpster diving, operations security issues, perception management, external intelligence, and vulnerability testing going to be done? Typically, the IPPAs involve some of each with the objective of demonstrating weaknesses where they exist, but with deeper inspection only when weaknesses are not immediately apparent.

In order to communicate effectively and be viewed as a professional, you must be able to switch languages while speaking to different people and understand the nuances of these different languages. For example, the term wire room means something very different to a facilities manager than to someone in charge of electronic funds transfers. Similarly, the abbreviations used are very different for different people with different backgrounds, so it is vital to have enough knowledge of these areas to be understood and understand properly.

Process

The assessment process consists of the following steps:

• Site visit: This should take one or two days and should be scheduled so that client representatives that know how each part of the environment under consideration operates at the most detailed level are available for interview. Typically, the top-level manager and technical experts in each unique environment should be present. If there are tours to be taken, they should be taken on that day. Although experiencing everyone's job may be useful, it is far more important to have the details available in a concise form. The client should also provide copies of all relevant documentation at the site visit.

  The members of the assessment team should be well prepared to ask any question related to their areas and a very thorough record of what was said by whom should be kept. The goal of this effort is to reconnoiter the client operation. That means that, like a grand jury, it is appropriate to go anywhere and ask anything.

  For lack of a better analogy, this is very much like a tiger team where you get the best experts to tell you how they could attack. The difference is that you don't touch anything, you just ask questions and make observations that allow you to assess the current situation. If the client doesn't know the answers to the questions, this is an area they should do more work on. If they do know the answers, these answers will reveal the weaknesses and strengths.

• Visit write-up: In the write-up, each team member provides a copy of their notes to the team leader who combines them to provide as accurate a depiction of the site visit as is possible. This is then returned to the team members for verification of factual accuracy and to resolve any disputes between different viewpoints. The result is a detailed description of the visit that the client team members should agree is an accurate depiction of what they said. The visit write-up should be completed and agreed to by the end of the first week of the study. It is vital to do the write-up as soon as possible after the site visit because this will assure, to as high a degree as possible, that the write-up is accurate. Contemporaneous notes are the best resource.
• Analysis and Findings: Each team member should concentrate on their areas of specialty as assigned by the team leader and perform a detailed analysis describing all of the weaknesses detected in the assessment process. In this phase, we rely on the experience and expertise of the team members to accurately and quickly detail the known weaknesses in the technologies in use. If the team members do not know this ahead of time, they cannot normally look it up at this point in the time frame of such a study, however, a good research team can help assure that these results are up-to-date. An important aspect of this activity is that the presentation not include all possible attacks, but rather systemic issues with examples of how systems can be exploited.

  Another critical factor to the success of the findings is that they relate the results to comparable organizations so that the client gets a feel for what is normal and prudent, what is critical, etc. In the findings, it is the responsibility of the assessment team to make value judgments about the relative import of different issues. For example, in a glass factory, if a particular router is used to connect the Internet to a file server which is used for advertising, it is vital to understand that regardless of the potential for abuse, this component is inherently less critical to this particular company than the temperature control in the ovens. This combination of technical and business understanding is what makes the findings valuable to the organization being assessed.

  In order to meet the typical 30-day time frame, the analysis and findings must be completed in draft form by the end of the second week of the study. Thus, the team members get only one week to generate, write up, and initially integrate their findings.

• Plan of Action and Findings Review: Once a draft of the site visit and findings is available, the team members have to read the total document with an eye toward catching any errors, omissions, or poorly stated items. This should include each team member cross-checking other team member results, the team leader resolving disputes, and a copy edit to review the document for writing, spelling, sentence structure, and presentation. In the meanwhile, the team leader has the dual job of generating a proposed plan of action to mitigate the problems revealed in the findings. This is done at two levels. Since this is only a qualitative assessment, an overall plan separating the most time-critical changes from the long term issues is appropriate. Then, a sample scenario for implementing this plan and estimates of costs are generated. This process should be completed by the end of the third week of the assessment.
• Closure: The draft document is now ready for review by members outside the assessment team. These reviewers should read the entire document with a critical eye. Final document preparation should be done. The executive summary should be written to summarize the report. Overhead viewgraphs should be made to summarize the document for a 15-minute presentation to top-level management. All of these should be completed and reviewed by the assessment team before the end of the fourth week so that the final presentation can be made on the last day of that week.