I'm about to order Andrew Jaquith's security metrics book from Amazon.com It is getting decent reviews and seems to resonate with my own feelings about the topic.
By the way, we've recently published a set of metrics aligned with ISO/IEC 27002 on the ISO27001security.com website (look in the white papers) and I've been working on the draft ISO standard for metrics but it's hard going. I rate metrics as one of the most difficult areas in information security.
Have had a look at the potential metrics in the 27002 white paper.
Some thoughts.
Interesting, most of the proposed measures are to do with measuring progress of the 'business process', e.g. how much of this have we done?
I see this as the 'political' type of metric. e.g. we've spent an extra $200M on health. Useful to impress the pundits, or a process oriented board (inc government departments?), but maybe less so those that are results focused.
You then have to ask the question, so what impact does this have on our risk profile and where are the metrics for this. Or perhaps, if there are no figures yet, how are we going to assess the impact on risk profile?
For some of this am sure we can learn from other 'risk fields'. For example road accidents. E.g. putting on your seat belt decreases your chance of injury by x%.which although applicable to the individual can only be gleaned by statistical analysis of thousands of cases.
So maybe we need to start correlating risk controls with security incidents over large numbers of cases. An enormous challenge. Is anybody attempting to do this do you know?
Another idea is to correlate attitudes with some of these metrics, i.e. can we predict the security culture of an organisation via some of these metrics (which ones, how strong is the correlation, and can we explain the correlation). Then to what extent does the security culture of an organisation correlate with security breaches. Are there particular dimensions of security culture that correlate more strongly than others?
In summary, I guess, its important to understand what the value of a measure.
Also be useful to assess the cost of measures, i.e. the cost of gathering, processing, presentation, plus the cost of people's time in measurement 'consumption' (now there's a concept!), vs their value as a basis for decision making.
Wed, 19/09/2007 - 4:41pm
Hi Ron.
I'm about to order Andrew Jaquith's security metrics book from Amazon.com It is getting decent reviews and seems to resonate with my own feelings about the topic.
By the way, we've recently published a set of metrics aligned with ISO/IEC 27002 on the ISO27001security.com website (look in the white papers) and I've been working on the draft ISO standard for metrics but it's hard going. I rate metrics as one of the most difficult areas in information security.
Gary Hinson
Passionate about security awareness
NoticeBored.com
ISO27001security.com
isect.com <- IT Audit FAQ here
Thu, 04/10/2007 - 3:59pm
Hi Gary
Have had a look at the potential metrics in the 27002 white paper.
Some thoughts.
Interesting, most of the proposed measures are to do with measuring progress of the 'business process', e.g. how much of this have we done?
I see this as the 'political' type of metric. e.g. we've spent an extra $200M on health. Useful to impress the pundits, or a process oriented board (inc government departments?), but maybe less so those that are results focused.
You then have to ask the question, so what impact does this have on our risk profile and where are the metrics for this. Or perhaps, if there are no figures yet, how are we going to assess the impact on risk profile?
For some of this am sure we can learn from other 'risk fields'. For example road accidents. E.g. putting on your seat belt decreases your chance of injury by x%.which although applicable to the individual can only be gleaned by statistical analysis of thousands of cases.
So maybe we need to start correlating risk controls with security incidents over large numbers of cases. An enormous challenge. Is anybody attempting to do this do you know?
Another idea is to correlate attitudes with some of these metrics, i.e. can we predict the security culture of an organisation via some of these metrics (which ones, how strong is the correlation, and can we explain the correlation). Then to what extent does the security culture of an organisation correlate with security breaches. Are there particular dimensions of security culture that correlate more strongly than others?
In summary, I guess, its important to understand what the value of a measure.
Also be useful to assess the cost of measures, i.e. the cost of gathering, processing, presentation, plus the cost of people's time in measurement 'consumption' (now there's a concept!), vs their value as a basis for decision making.
Cheers Ron