Microsoft Entra Conditional Access (CA) is the policy engine that decides which sign-ins are permitted, which require additional verification, and which are blocked. It is the most powerful access-control tool in the M365 stack and the easiest to misconfigure in ways that lock legitimate users out without warning.
Every MSP that has managed more than ten tenants has a CA story. Common shapes: a "block legacy authentication" policy that catches a critical line-of-business app nobody documented; a "require compliant device" policy that locks out users on personal devices over the weekend; a "block anonymous IPs" policy that stops the CFO's VPN. The stories end with an emergency call and an out-of-policy override.
The test-in-production anti-pattern
CA changes are typically tested by enabling the policy in "Report-only" mode and watching the sign-in logs. This works in principle. In practice, it has three failure modes. First, low-volume sign-in scenarios (the CFO who logs in twice a week from a specific IP) generate no signal during the report-only window. Second, the report-only mode does not capture the cross-policy interactions that emerge when the new policy is combined with existing policies. Third, "report-only" is sometimes treated as a checkbox rather than an evaluation phase, and the policy ships to enforcement after a day with insufficient coverage.
The structural fix is to simulate the policy against a representative population of users-platforms-client-apps before enabling it, including in report-only mode. This is what the CA What-If tool does.
What CA actually evaluates
CA policies are evaluated as a logical AND across assignments and conditions. The assignment dimension includes users / groups / roles / external identities. The condition dimension includes sign-in risk, user risk, device platform, network location, client app, device state, and authentication context. The grant control specifies the action — block, require MFA, require compliant device, require terms of use, custom authentication context.
A real tenant has typically six to fifteen active CA policies. Each new policy interacts with the existing set: an "allow only from trusted networks" policy combines with a "block anonymous IPs" policy in subtle ways, and an "exclude break-glass accounts" exception has to be carried through every policy. The combinatorial space gets large fast.
What-If simulation: sample users × platforms × client apps
A What-If simulator takes a proposed policy change and evaluates it against a representative sample of (user, platform, client app, network location) tuples — drawn from real sign-in history if possible, synthetic if not. The output is a per-tuple verdict: allowed, blocked, MFA-required, compliant-device-required.
The value of the simulation is the surprises. The CFO sample tuple gets blocked because the new policy excluded a network range it should have included. The mobile-app sample tuple requires MFA on every sign-in because the new policy changed the session lifetime. The Mac-OS Outlook sample tuple gets blocked because the new policy assumes Windows-only.
A rough rule of thumb: simulate at least twelve representative tuples covering the highest-risk roles (CFO, IT admin, board members) and the highest-volume sign-in scenarios (mobile app, web Outlook, Teams desktop). If twelve tuples all pass, the policy is probably safe. If even one tuple shows an unexpected verdict, the policy needs revision.
Drift detection between snapshots
CA policies drift over time even when no one is intentionally changing them. Microsoft adds new conditions to the platform; an admin enables a recommended policy without remembering doing so; a third-party tool synchronises a baseline that overlaps with existing policy. Drift detection compares the current state to a known-good snapshot and flags every change at the field level.
Drift is sometimes legitimate (a new condition was deliberately added) and sometimes not (an unauthorised change was made). The detection layer is acknowledge-or-remediate workflow, not auto-revert — auto-revert in CA is dangerous because it can override a deliberate change made under time pressure.
Bulk template deployment with rollback
For MSPs managing tens or hundreds of tenants, CA policies are best authored as versioned templates and bulk-deployed across the fleet. The pattern: an anchor tenant (the MSP's own tenant or a designated reference customer) holds the gold-standard policy set. Templates derived from the anchor are versioned and published to the fleet with a per-tenant deployment scope.
Bulk deployment requires a 4-eye approval workflow (the engineer authoring the template and the engineer reviewing it cannot be the same person) and atomic rollback (if any tenant in the deployment fails verification, the entire deployment is rolled back to the previous version). Without these, "ship the template to all 200 tenants" becomes an operational risk multiplied by 200.
POPIA Section 19 implications
POPIA Section 19 frames "appropriate, reasonable technical and organisational measures" (Republic of South Africa, 2013). Conditional Access is the technical measure that controls who can access personal information at sign-in time. A change-management posture without simulation, drift detection, and rollback is harder to defend as appropriate when the regulator asks how a CA-related lockout or breach happened.
References
Microsoft (2024) Conditional Access in Microsoft Entra. Microsoft Learn. Available at: https://learn.microsoft.com/entra/identity/conditional-access (Accessed: 8 May 2026).
Republic of South Africa (2013) Protection of Personal Information Act, No. 4 of 2013. Government Gazette of the Republic of South Africa.