The Gap Between Policy and Practice: Testing What You Wrote Down

Most organizations have a security policy library. Far fewer can show that what the policies say is what the business does.

The gap is not usually dishonesty. A policy is written at a point in time. Systems change, people move on, and the written rule drifts away from the real routine. Nobody notices until an auditor, an insurer or an attacker tests it.

A small, regular test closes much of that gap. Pick a handful of controls, ask for operating evidence, and see what comes back.

TLDR

• A policy shows intent; only operating evidence shows practice.

• Test a small sample of controls on a schedule rather than everything once.

• Ask for dated artifacts such as logs, tickets and reports, not statements.

• Record the gaps you find, assign owners and retest.

• The gap list is useful to the board even when it is uncomfortable.

Why the gap opens

Policies are usually written to satisfy a requirement, then filed. The people who operate the systems rarely read them. When a new tool arrives or a team reorganizes, the practice changes and the policy stays put.

The result is a library that is accurate on the day it was approved and increasingly wrong after that.

A small test that works

You do not need a full audit. Choose five controls that matter, one from each of five areas, and ask the owner for evidence the control operated during the last quarter.

This follows a long-standing assessment idea. NIST's guide to assessing controls, SP 800-53A, describes assessment methods built around examining documents, interviewing people and testing behavior. A simple version of those three checks is enough to start.

The plan below is illustrative. Replace the rows with the controls that matter in your organization.

Control (illustrative)	What the policy says	Evidence to request	Gap found?
Access removal	Accounts are disabled on the day someone leaves	Last quarter's departures matched to disable dates	To be tested
Patching	Critical fixes are applied within a set window	Patch report for five critical items	To be tested
Backup restore	Critical systems are restored on a schedule	Dated restore test record	To be tested
Vendor review	High-risk vendors are reviewed annually	Review records for the top five vendors	To be tested
Incident drill	The response plan is exercised each year	Drill date, attendees, follow-up items	To be tested

What to do with what you find

Expect gaps. A first test almost always finds some. The right response is not blame. It is a short list: what the gap is, who owns the fix, and when you will retest.

Feed the list to whoever oversees the program. What a board needs is not a clean scorecard. It is a clear view of what is real, what is not, and what is being done about it. The post on making compliance frameworks actually work covers the same shift from paper to practice.

Make it routine

Rotate the sample every quarter so that, over a year, most important controls get tested. Keep the evidence requests the same each time so owners learn what will be asked.

Over time, people begin keeping the evidence as they go. That habit is the real benefit: the gap narrows because the test is expected.

Frequently asked questions

How many controls should we test at a time?

Five is a good start. It is small enough to finish in a few days and broad enough to show a pattern. Rotate the sample each quarter.

Who should run the test?

Someone independent of the control owner, such as internal audit, risk or an outside advisor. The person who runs a control is the wrong person to grade it.

What counts as good evidence?

A dated artifact produced by the system or process itself, such as a log, ticket, report or signed record. A statement that the control works is not evidence.

What if the test shows the policy is wrong, not the practice?

Then fix the policy. Sometimes practice has legitimately moved on. Either way, the two should match and the change should be approved.

A Defensible Next Step

Pick five controls this week and send the evidence requests. Whatever comes back, or does not, is more useful than another policy review. Book a Decision-Clarity Call if you would like an independent view of the results.