May 11, 2018

Last Updated on May 11, 2018

Editor’s Note: This post was originally published in June 2013 and has been updated for accuracy and comprehensiveness.
NIST/FISMA guidance is generally more prescriptive than most other forms of information security guidance. However, there are limits to any standard’s ability to be prescriptive.
These limits are well illustrated by a recent call that I had with a potential client. Our conversation began with him telling me: “This should be a pretty quick conversation—I just need a price for a NIST Penetration Test.” 
As you probably already guessed, the call was not nearly as quick as the client had hoped. The challenge is that there is no definitive definition of a NIST Penetration Test. NIST Special Publication 800-53 (Rev. 4), “Security Controls and Assessment Procedures for Federal Information Systems and Organizations,” is covered under CA-8 PENETRATION TESTING. The definition of the penetration test (see below) is more “real-world” and moves physical penetration testing to a “can include” item, which is consistent with how most organizations test.   
A couple of interesting caveats pertain to CA-8:

  • It applies to systems with a High security categorization. 
  • It includes a control enhancement that calls for the test team to be sufficiently independent. 
  • It includes a control enhancement that allows for the pen test to be extended to a red team exercise to better simulate a real-world attack and the organization’s ability to defend itself from that attack. 

The “centerpiece” of the NIST information security framework is NIST 800-53—hence, it was the first document I referred to during our conversation. Since Revision 3 of the framework there has been some definition added regarding penetration testing: 

“Penetration testing is a specialized type of assessment conducted on information systems or individual system components to identify vulnerabilities that could be exploited by adversaries. Such testing can be used to either validate vulnerabilities or determine the degree of resistance organizational information systems have to adversaries within a set of specified constraints (e.g., time, resources, and/or skills). Penetration testing attempts to duplicate the actions of adversaries in carrying out hostile cyber-attacks against organizations and provides a more in-depth analysis of security-related weaknesses/deficiencies. Organizations can also use the results of vulnerability analyses to support penetration testing activities. Penetration testing can be conducted on the hardware, software, or firmware components of an information system and can exercise both physical and technical security controls. A standard method for penetration testing includes, for example: (i) pretest analysis based on full knowledge of the target system; (ii) pretest identification of potential vulnerabilities based on pretest analysis; and (iii) testing designed to determine exploitability of identified vulnerabilities. All parties agree to the rules of engagement before the commencement of penetration testing scenarios.”  

While I have read NIST-800-53 dozens of times, I did not recall the physical security element mentioned above. Thus one could argue that a NIST Penetration Test should include physical penetration testing, although fewer than 5% of all penetration tests that Pivot Point Security conducts include physical security testing. Further, most physical penetration testing naturally encompasses some form of social engineering (e.g., impersonation of an employee or authorized visitor, fake employee badges, tailgating).
The important takeaway is that the extent and rigor of the testing should be proportional to the risk the system presents. A system protecting information on troop movement in Afghanistan should be tested more thoroughly than one housing public land deed records.
During my initial call with the client, we agreed that a NIST penetration test is a test aligned with good practice where the coverage (e.g., application, network, exploit types, social engineering, and physical security) and extent/rigor is aligned with the risk profile of the solution being assessed. The challenge with this approach is the test was being requested by a third party and we wanted to make sure the testing was going to meet their requirements before actually conducting the test.
There are two approaches to this chicken-and-egg scenario: 1) Request clarification from the client on their expectation; or 2) Get the client’s blessing on the scope you intend to execute. While there is no right/wrong approach, over the years we have found that in most instances the former ends up with a smaller scope of work.
At the end of the day, this particular client was happy with a relatively straightforward network and application penetration test that did not include physical security testing or social engineering. This was the best fit with the risk profile of the system being tested—and also a good-practice application of NIST guidance to penetration testing.