Quantitative penetration testing with item response theory

Quantitative penetration testing with item response theory

Pieters, W.
Arnold, F.
Stoelinga, M.I.A.

Technology, Policy and Management

Engineering, Systems and Services

2013-11-04

Existing penetration testing approaches assess the vulnerability of a system by determining whether certain attack paths are possible in practice. Therefore, penetration testing has thus far been used as a qualitative research method. To enable quantitative approaches to security risk management, including decision support based on the cost-effectiveness of countermeasures, one needs quantitative measures of the feasibility of an attack. Also, when physical or social attack steps are involved, the binary view on whether a vulnerability is present or not is insucient, and one needs some viability metric. When penetration tests are performed anyway, it is very easy for the testers to keep track of, for example, the time they spend on each attack step. Therefore, this paper proposes the concept of quantitative penetration testing to determine the diculty rather than the possibility of attacks. We do this by step-wise updates of expected time and probability of success for all steps in an attack scenario. Also, the skill of the testers can be included to improve the accuracy of the metrics, based on the framework of Item Response Theory (Elo ratings). We show the feasibility of the approach by means of simulations, and discuss application possibilities.

item response theory
penetration testing
quantitative security
security metrics
socio-technical security

http://resolver.tudelft.nl/uuid:becbbecf-12c2-4b11-8ebb-3b24657bc50a

TREsPASS

TREsPASS: Technology-supported Risk Estimation By Predictive Assessement Of Socio-technical Security

Institutional Repository

report

(c) Pieters, W.
Arnold, F.
Stoelinga, M.I.A.