Press Release
Last Updated on January 15, 2024
Editor’s Note: This post was originally published in February 2014 and has been updated for accuracy and comprehensiveness.
I’m still surprised how often our clients want us to perform black box penetration testing on their internet facing systems, instead of white box testing. What’s the difference? In black box testing, the client gives the tester no information about the systems, so he or she has to start by using techniques like “DNS mining” to try to find out about them. It’s like a game of hide-and-seek. In white box testing, conversely, the client shares in-depth knowledge of the internals of the systems being tested. That understanding is used to simulate attacks that directly assess how secure the systems actually are.
There seems to be a mystique around black box testing, possibly because it supposedly simulates the approach that a real-world attacker would take. Therefore, it’s perceived to be more “realistic” and thus more accurate.
In my experience, the opposite is true: black box testing can provide a “false sense of security” because it does not actually test how secure your systems are. It really only tests how well hidden they are… on this occasion, for this simulated attacker. It can’t guarantee that all the data that can be found will be found.
In other words, a black box test tells you what an attacker can straightforwardly find out about your assets. Say a black-box tester finds 90% of your IP addresses. What if your greatest vulnerabilities are on the machines he couldn’t find, or couldn’t figure out were yours?
A real-world bad guy on a different day, with different skills and resources, might get different results. If it’s on the Internet, somebody will eventually find it and touch it. And once an attacker is locked on, your system had better have its shields up.
For example, during a recent engagement we were asked to perform a black box penetration test: discover as many of the client’s hosts as possible, using only their e-mail address. Say the e-mail domain was “example.com.” We started by finding the mail and DNS servers for that domain.
From there we leveraged the IP addresses of the servers, using a purpose-built, open source tool to see if the IPs for that domain were part of larger IP blocks assigned to the company. We also utilized another open source tool to attempt to copy all their DNS records to fill out the list of assigned IP addresses. At that point we attacked the servers, fast and hard.
What did that prove? Only that for a reasonably skilled attacker, sufficient information to launch an attack was accessible in the public domain. In my humble opinion it’s best to assume that’s the case 100% of the time, and proceed with white box testing from there.
What specifically are the advantages of white box penetration testing over black box testing?
- Because the white hat hacker has insider information, s/he can perform rigorous and comprehensive testing.
- Because no time is spent playing hide-and-seek, white box testing is more time- and cost-efficient than black box testing.
- White box testing provides attestation on the security posture for all your exposed hosts.
Hoping that you can hide better than attackers can seek is not a security strategy—it’s a form of gambling. White box testing combined with comprehensive penetration testing, can thoroughly test your applications, networks and people to tell you how robust your security posture really is, where its weak spots are, how likely it is that attackers will find them, and what it will take to fix the vulnerabilities. That’s what you want to know.
