Domain 2 of 5 · Chapter 4 of 4

Measure Cybersecurity Effectiveness

Unlock the complete study guide + 12 practice exams totaling 1,500 questions.

Bundled into the existing Certified in Cybersecurity premium course — no separate purchase.

14-day money-back guarantee — no questions asked.

Included in this chapter:

  • What a security metric is for
  • Writing down how a metric is calculated
  • Checking the data before drawing conclusions
  • Qualitative ratings and quantitative measures
  • Four types of measures
  • Timing metrics: MTTD and MTTR
  • Measuring awareness: completion, clicks and reports
  • Key risk indicators and key performance indicators
  • Thresholds and escalation
  • AI-specific risk indicators
  • Presenting results: dashboards, scorecards, reports
  • Protecting measurement data
  • Trends over time
  • The corrective-action cycle

Four types of security measures and the question each answers

AspectImplementation measureEffectiveness measureEfficiency measureImpact measure
Question it answersIs the planned control in place, and how widely?Does the control achieve its intended result?How quickly is the work done?What does security mean for the mission and budget?
ExamplePercentage of eligible systems with the required control installedShare of test attacks the control stopped in a controlled testMean time to remediate a known vulnerabilityCosts incurred from addressing security incidents
Cannot show on its ownWhether the control actually worksHow fast or at what cost the result was achievedWhether the work produced the intended protectionWhich specific control needs fixing
Tempting misuseCounting installed agents as proof they stop attacksTreating money spent on a control as its protective resultAnswering how fast with a count of patches installedUsing raw alert volume as the business cost of an incident

Decision tree

Is the planned controlin place, and how widely?Implementation measurecoverage of the controlYesNoDoes the control achieveits intended result?Effectiveness measureoutcomes, e.g. test resultsYesNoHow quicklyis the work done?Efficiency measuree.g. mean time to remediateYesNoWhat does security meanfor the mission or budget?Impact measuree.g. incident costsYesNoNone of these: restate the question before choosing a metric
Cheat sheet

Cheat sheet

  • Useful security metrics answer a defined organizational question
  • Documented metric definitions make results repeatable and comparable
  • A target expresses desired performance rather than merely the current result
  • Incomplete or inconsistent input data can invalidate a security conclusion
  • Qualitative ratings and quantitative measurements convey different kinds of information
  • Security measurements need a time reference and a suitable refresh interval
  • Implementation measures show whether planned controls have been deployed
  • Effectiveness measures show whether controls achieve their intended results
  • Efficiency measures examine how quickly security work is done
  • Impact measures connect security events to organizational consequences
  • Mean time to detect measures how long problems remain undiscovered
  • Mean time to recovery measures observed restoration performance
  • Training completion and demonstrated learning answer different questions
  • Phishing click rates and reporting rates measure different user actions
  • Phishing simulation results depend on message difficulty and user context
  • KRIs track risk while KPIs track progress toward intended results
  • Risk thresholds connect indicator readings to review and escalation
  • AI-specific KRIs complement ordinary service-health reporting
  • A risk indicator can justify action before an incident is confirmed
  • A dashboard presents selected metrics for convenient monitoring
  • A scorecard organizes performance against strategic goals
  • Security reports should explain the significance of the numbers
  • The level of reporting detail should fit the recipient's responsibilities
  • Repeated comparable measurements reveal trends that a snapshot cannot
  • Security measurement data can itself be sensitive
  • Measurement should support corrective action and verification of improvement

Unlock with Premium — includes all practice exams and the complete study guide.

Also tested in

References

  1. https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-55v1.pdf
  2. https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-55v2.pdf
  3. https://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication800-30r1.pdf
  4. https://csrc.nist.gov/glossary/term/recovery_time_objective
  5. https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-50r1.pdf
  6. https://nvlpubs.nist.gov/nistpubs/TechnicalNotes/NIST.TN.2276.pdf
  7. https://csrc.nist.gov/glossary/term/risk_tolerance
  8. https://csrc.nist.gov/glossary/term/risk_appetite
  9. https://nvlpubs.nist.gov/nistpubs/ir/2020/NIST.IR.8286.pdf
  10. https://nvlpubs.nist.gov/nistpubs/ir/2022/NIST.IR.8286C.pdf
  11. https://www.isc2.org/certifications/cc/cc-certification-exam-outline
  12. https://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication800-137.pdf
  13. https://csrc.nist.gov/CSRC/media/Presentations/Creating-a-Cybersecurity-Scorecard/images-media/Developing%20a%20Cybersecurity%20Scorecard.pdf
  14. https://www.nist.gov/system/files/documents/nvl/BalancedScorecard.pdf