Conditional access is a policy, not a prayer

Conditional accessdefinitionRules evaluated at each sign-in that look at conditions such as user, device or location, and then allow, block or demand more proof. policies decide, on every sign-in, who gets into everything you run. In most tenants they're still edited by hand, in production, with no history and no test. The control guarding your front door deserves at least the engineering discipline you give a config file. Put it in version control, stage the rollout and rehearse the rollback. Anything less is hoping the last admin clicked the right box.

A conditional access mistake goes one of two ways. A policy that's too strict locks people out, sometimes the very admins who could fix it, and the help desk lights up. A policy with a gap lets in someone who shouldn't be there, and nothing on screen tells you.

The gap is the one that hurts. An exclusion added for a travelling executive or a broken app often stays forever. If that policy was the only thing asking for a second check, anyone who phishes that person's password gets in on the password alone. The sign-in log just records the policy as not applied.

We've argued before that zero trust is a proof obligation: every access decision is a claim you should be able to evidence. Conditional access is where that gets mechanical, since the policies are the claims. This note is about keeping them provable while you change them, which mostly comes down to three disciplines.

For the engineers, the export is the easy part. Microsoft Graph lists every policy at GET /identity/conditionalAccess/policies with the Policy.Read.All permission, so a scheduled job can pull the whole tenant into a repository. Each sign-in in the Entra sign-in logs then has Conditional Access and Report-only tabs that show which policies applied and which didn't.

The What If tool under Conditional Access simulates one sign-in for a user or service principal before you change anything. It doesn't follow service dependencies, though, so a test for Teams won't show the Exchange Online policy sitting behind it. Use it to check a single case, and use report-only data to see the whole tenant.

Policies are production code, so give them a repository. A tenant's CA configuration should live in version control, exported as code and deployed through review, so nobody has to reconstruct it from memory after an incident. Every change gets a diff, an author, a reviewer and a reason, and every past state can be restored.

After any access incident someone asks what changed, and what the policy said at the time. A portal edit history can't answer that well. Entra's audit log does record old and new values for each change, but by default it only keeps them for 30 days. A git log makes the question trivial, however long ago the change was.

Policy-as-codedefinitionKeeping policies in version control as files and deploying through review. That way each change is tracked and can be undone. also kills the classic driftdefinitionWhen a live configuration slowly wanders away from what you intended, often through emergency edits nobody tracked. failure, where the emergency exclusion added at 2 a.m. quietly becomes permanent because nothing tracked it. Exclusions deserve the same rigor. Give each one a named owner and a written reason, put an expiry date on it, and review the list on a schedule. An exclusion list that only ever grows is a policy quietly repealing itself.

If a firewall change needs a peer review, the rule set that gates your identity provider needs one more. It decides who reaches every app at once, so one bad edit there costs far more than one bad firewall rule.

Report-only mode is your staging environment, so use it like one. No policy should enforce on day one. Entra's report-only modedefinitionA setting where the policy runs against real sign-ins and logs what it would have done. Nobody actually gets blocked. evaluates a policy against real sign-in traffic and logs what it would have done. That makes it a staging environment for access decisions, fed by production load.

Run new policies there for a full business cycle. A quiet weekend tells you very little. Month-end batch jobs, the quarterly finance crunch and that one legacy client that can't satisfy a device claim all need to appear in the would-be-blocked data before enforcement.

Then read the results like a test report. Would-be blocks against service accounts mean you've found automation that authenticates like a human. Would-be blocks against an executive's unmanaged tablet mean a conversation is cheaper now than a lockout later.

Enforce only when the projected impact matches the intended impact, because that match is your evidence the policy is correct. If the policy is already live, Microsoft suggests a report-only copy with your edits first. Compare the two side by side, and only then touch the real one.

# require-compliant-device: staged enforcement
policy: require-compliant-device  // v7, in git
phase 1: report-only · all users (Mac, iOS, Android excluded) · 30 days
phase 2: enforce · pilot ring (IT + volunteers)
phase 3: enforce · staged rings → all users
excluded: break-glass ×2 · alarmed on any use
rollback: revert to v6 · rehearsed · < 5 min
✓ lockout-rate alert armed before every phase

Engineer the blast radius before you need it. The worst CA failure is self-inflicted: a policy that locks out the administrators who could fix it. Roll out in rings, starting with a pilot group that includes the people who wrote the policy, then move through staged cohorts. A bad interaction then burns one team and the tenant stays up.

Keep at least two break-glass accountsdefinitionEmergency admin accounts excluded from normal access policies. You only use them when everything else has failed, and you watch them closely. excluded from every conditional access policy that blocks or restricts sign-in. Alarm them so loudly that any use triggers an immediate investigation, because an exclusion nobody watches is a backdoor. Microsoft now recommends a passkey (FIDO2) security key for these accounts, or certificate sign-in if you already run a PKI, since those satisfy its mandatory MFA rules.

Then rehearse the rollback. Know which version you revert to, who is authorized at 3 a.m., and how long propagation takes, and measure that last one rather than guessing.

Sign-in failure telemetry with alerting on lockout spikes closes the loop. With that in place you hear about a lockout from your alerts first, and the burning help desk just confirms it.

Many teams can start on their own by exporting the policies and listing the exclusions. If you'd rather have help, Cohash's identity practice runs the same review, and our IAM assessment ends in a ranked, costed roadmap.