Press Release
Last Updated on January 15, 2024
Comprehensive vs. Holistic: Not the Same
A comprehensive approach to an information security assessment sounds like a good thing, correct? After all, comprehensive means “Complete; including all or nearly all elements or aspects of something.” Having uniquely focused on conducting information security assessments for the last ten years – I have often tried to effectively communicate the difference. A recent electrical utilities project we worked on perfectly illustrates the problem with a “comprehensive” approach and why a “holistic” approach is required for critical systems.
There is a tremendous amount of energy and focus in and around Smart Grid Security. It is well-considered as the challenges are numerous; rapidly evolving technology, even more rapidly evolving standards/guidelines, the sheer number of moving parts (e.g., in home equipment, smart meter’s, demand response applications, customer facing applications, utilities/service provider transit networks, Distribution Management Systems, IP enabled SCADA devices, etc.). The client recognized these challenges and developed a comprehensive approach to assessing the security of their Demand Response initiative (Demand Response (DR) is a mechanism to manage customer consumption of electricity in response to supply conditions, for example, having electricity customers reduce their consumption at critical times or in response to market prices.)
A Comprehensive Information Security Approach
Their comprehensive approach included:
- A design review and penetration test of the In Home Device (IHD) and smart thermostat (connecting the customer to the meter)
- A design review and penetration test of the In Smart Meter (connecting the meter to the transit network)
- A design review and penetration test of the smart grid wireless network (connecting the meters to the demand response application and management networks)
- An application penetration test of the DR application and a network penetration test of systems supporting it
- A design review of the communication between the utility and the Demand Response application
In order to meet a very aggressive project deadline two different vendors were engaged in this assessment. Another vendor performed the Smart Meter testing (#2). Their testing focused on ensuring that the communications between the smart meter and the smart grid were secured appropriately (which they were). Where things jumped the track a bit is when we identified vulnerabilities in the IHD (#1) that may allow it to communicate upstream in an unintended manner. Unfortunately, the vendor assessing the smart meter had not tested the smart meter to determine whether it did/could “block” non expected (malicious) upstream communication. In fairness to the vendor, it wasn’t part of the utilities “defined” assessment scope.
Holistic means “Emphasizing the importance of the whole and the interdependence of its parts”. Interdependence is critical when evaluating complex systems. Identifying/understanding the Interdependence between varying technical solution elements and the key processes that are necessary to effectively “operationalize” it demands a holistic approach.
A Holistic Approach is Challenging
It requires “due diligence” at the project outset to really think through the requirements of the assessment. A risk-centric approach is the only way to effectively scope a holistic security assessment. We favor an ISO 27005 aligned approach that focuses on information and processes as the assets – which really focuses the assessment on information and process centric risks.
My takeaway was a better way to communicate the difference between a comprehensive and holistic information security assessment. Hopefully, yours will be the recognition that critical, complex systems with a high degree of interdependence require a holistic approach to risk assessment, risk management, and the information security assessments that support these efforts.
