January 11, 2024
Download:
Over the past many years, a group of cybersecurity rating firms have developed proprietary methodologies that produce rankings for individual enterprises relying largely on publicly available scans of their internet- facing assets. While these scores can provide important insight on a company’s risk posture, the methodologies employed to achieve these outputs have raised significant concerns when the assets evaluated are not controlled by the surveyed entity. These methodological flaws are particularly problematic for telecommunications firms and Internet Service Providers (ISPs) that provide their customers with blocks of IP addresses for use in their own organizations or for the customers. This problem has been known for many years and as the use of cybersecurity scoring services continues to grow, so have the harms associated with their often-misleading conclusions.
Since at least 2018, the rating firms have been aware of these concerns and at least one major scoring company made a substantial concession when the U.S. Chamber of Commerce and FICO Cyber Score (since acquired by ISS) revised their industry benchmarking initiative, (the Assessment of Business Cybersecurity (ABC). In a subsequent report, they acknowledged that ISPs, Infrastructure as a Service (IaaS), telecom, and cloud service providers “…with large IP address footprints controlled by IT and security teams outside their direct control could increase the likelihood of double-counting assets when such assets would be more appropriately attributed to the subscribing organizations.” They explained that “[F]or these reasons, we have elected to exclude companies in this class and have adjusted the ABC and its various sub-indices.”
This report is intended to provide an update on progress in this area and focuses on issues that continue to plague telecommunications firms and ISPs that must make substantial and ongoing investments to segment massive sets of IP addresses. The report describes the quantitative mechanisms that are used and why they continue to be a source of disagreement between telecommunications firms and security rating vendors. The rating company business model is also critiqued and while it acknowledges the value that fair and methodologically supported mechanisms can offer a variety of stakeholders from Boards to procurement specialists evaluating third-party risk, it offers three constructive recommendations to advance the interest of all parties.
First, the report notes that its purpose is not to make algorithmic design decisions for the security rating companies. Instead, it urges these companies to consider alternatives for “asset discovery” in ways that improve the validity and accuracy, and thus the utility of their products. Second, the authors explain why the rating firms should provide an option for customized risk models based on the unique characteristics of telecommunications firm’s’ threat landscape and business use cases. And third, the rating firms are encouraged to engage proactively and collaboratively with customers to improve rating accuracy. It notes that by working in such ways with the telecommunications providers, the rating firm would benefit from continuous improvement in a very dynamic cyber ecosystem.
Finally, the report notes that initial communications with leading ratings providers have already produced positive results. As evidence of such progress, both BitSight and Security Scorecard removed their industry scores from their public-facing websites industry scores for telecommunications firms.