A control that looks good on paper but is not actually performed reduces no risk at all. Control effectiveness is the discipline of determining whether a control is both properly designed and operating as intended. It is the difference between a control register that reflects reality and one that flatters it. This guide explains design versus operating effectiveness, the testing methods auditors use, how to rate controls, and how effectiveness drives residual risk.
What You'll Learn
By the end of this article you will understand the two dimensions of effectiveness (design and operating), the four testing methods, how to apply an effectiveness rating scale, how to sample and document evidence, and how a control's effectiveness directly determines residual risk. A worked assessment table is included.
What Is Control Effectiveness?
Control effectiveness is the degree to which a control achieves its objective, reducing the likelihood or impact of a risk to an acceptable level. Assessing it means gathering evidence to form a conclusion about whether a control can be relied upon. It is not a matter of opinion or self-attestation. It is a judgement supported by evidence.
The concept rests on a simple but important distinction. A control can fail in two different ways, and effectiveness testing has to address both:
- It can be badly designed. Even if performed perfectly, it would not adequately address the risk.
- It can be well designed but not operating. The right control exists on paper, but it is skipped, performed inconsistently, or performed by someone without the authority or information to do it properly.
This is why effectiveness is assessed in two dimensions: design effectiveness and operating effectiveness. Understanding the different types of risk controls (preventive, detective, corrective, and so on) is helpful background, because the way you test a control depends partly on what kind of control it is.
Design vs Operating Effectiveness
These two dimensions are sequential. There is little value in testing whether a control operates if it was never capable of addressing the risk in the first place, so design is assessed first.
| Dimension | Question It Answers | What You Examine |
|---|---|---|
| Design effectiveness | If performed as intended, would this control address the risk? | Control description, policy, workflow, who performs it, frequency, and what would happen if the risk event occurred |
| Operating effectiveness | Is the control actually being performed as designed, consistently, over time? | Evidence that the control ran on real transactions across the period: logs, approvals, reconciliations, sign-offs |
Assessing Design
Design assessment is a walkthrough exercise. You trace the control through its intended operation and ask whether, as designed, it would prevent or detect the risk it targets. A common red flag is a control whose design leaves a gap, for example a payment approval limit that does not cover a payment channel, or a reconciliation performed by the same person who records the transactions.
Assessing Operation
Operating effectiveness can only be tested once design is judged sound. Here you gather evidence that the control actually ran, repeatedly, throughout the period under review, on real transactions. A control that was performed in January and December but skipped for ten months in between is not operating effectively, even if its design is impeccable.
Important
Never conclude on operating effectiveness without first confirming design. If you test whether a poorly designed control operates and conclude "yes, it operates," you have validated a control that does not actually reduce the risk, which is a dangerously false sense of assurance.
Want the full framework with worked examples?
Testing Methods: Inquiry, Observation, Inspection, Re-performance
Auditors use four primary methods to test controls. They differ markedly in the strength of evidence they produce, generally increasing in reliability from inquiry to re-performance. Strong testing usually combines methods rather than relying on one.
| Method | What You Do | Evidence Strength |
|---|---|---|
| Inquiry | Ask the control owner how the control is performed | Weakest; corroborate with another method |
| Observation | Watch the control being performed in real time | Moderate; only proves it ran when watched |
| Inspection | Examine documents and records evidencing the control | Strong; durable, reviewable evidence |
| Re-performance | Independently re-execute the control to verify the result | Strongest; direct proof the control works |
Inquiry
Asking the control owner to describe the control. Useful for understanding and for design assessment, but on its own it proves nothing about operation. People describe the control as it should work, not always as it does. Inquiry must be corroborated.
Observation
Watching the control happen, such as observing a cash count or a system access review. The limitation is that people perform controls more carefully when watched, and observation only covers the moment you were present.
Inspection
Examining the documentary trail a control leaves behind: approved purchase orders, signed reconciliations, system logs, exception reports. Inspection produces durable evidence that can be re-reviewed, which is why it is the workhorse of control testing.
Re-performance
Independently redoing the control to confirm it produces the right outcome: recalculating a reconciliation, re-running an access-rights query, or re-checking that a three-way match holds. It yields the strongest evidence because you are not trusting the control owner's output. You are verifying it.
The Effectiveness Rating Scale
To make assessments comparable, controls are rated on a consistent scale. A common four-point scale combines the design and operating conclusions into a single rating:
| Rating | Meaning | Implication |
|---|---|---|
| Effective | Well designed and operating consistently across the period | Reliance justified; residual risk as assessed |
| Partially Effective | Designed adequately but operating with exceptions or gaps | Improvement needed; residual risk elevated |
| Ineffective | Operating poorly or not at all, or design is deficient | Cannot be relied upon; treat as if control is absent |
| Not Yet Operating | Newly designed control without sufficient operating history | Cannot conclude on operation; re-test after a period |
The crucial point is that an "Ineffective" rating means the control provides little to no risk reduction. So for risk purposes it should be treated almost as if it were not there at all. That has direct consequences for residual risk, covered below.
Sampling and Evidence
You rarely test every instance of a control. Instead you select a sample and infer from it. The size of the sample depends on how often the control runs and how much you intend to rely on it.
- Frequency drives sample size. A control performed daily warrants a larger sample (e.g. 25 items) than one performed monthly (e.g. all 12, or a subset).
- Risk drives rigor. Controls addressing high residual risk deserve larger samples and stronger methods (inspection and re-performance over inquiry).
- Cover the period. Spread the sample across the full review period, not just one convenient month, so seasonal or staffing gaps are caught.
- Document exceptions. Every instance where the control did not operate as designed is an exception that must be investigated and, where material, reported as a finding.
The output of testing is evidence, the work that supports your conclusion. Good evidence is sufficient (enough of it), reliable (independent and durable), and relevant (it actually addresses the control objective). Every conclusion in your assessment should be traceable back to the evidence that supports it. An exception you find during testing may become an audit finding.
Pro Tip
When a sample turns up exceptions, resist the urge to immediately expand the sample to "prove" the control works. First understand why the exception occurred. A single exception caused by a systemic design gap is far more serious than a handful of isolated human slips, and a bigger sample will not change that.
Linking Effectiveness to Residual Risk
Control effectiveness is the hinge between inherent and residual risk. Inherent risk is the exposure before controls. Residual risk is what remains after controls are applied, but only to the extent those controls actually work. A control rated "Effective" justifies the reduction from inherent to residual risk. A control rated "Ineffective" does not, and the residual risk should rise accordingly. See inherent vs residual risk for the underlying framework.
A Control Effectiveness Assessment Extract
An auditor assessing the procure-to-pay process tests three controls. The assessment table records design and operating conclusions, the methods used, and the effect on residual risk:
| Control | Methods Used | Design | Operating | Rating | Effect on Residual Risk |
|---|---|---|---|---|---|
| Three-way match before payment | Inspection, Re-performance | Sound | No exceptions in sample of 25 | Effective | Reduces to Low as assessed |
| Dual authorization over R50,000 | Inspection | Sound | 3 of 25 paid without second approver | Partially Effective | Residual risk remains Medium |
| Monthly vendor master review | Inquiry, Observation | Gap: no review of bank detail changes | Not evidenced for 8 of 12 months | Ineffective | Residual risk stays at inherent (High) |
Notice the third control. Because it is ineffective, the residual risk does not drop below the inherent level, so the organization is exposed as if the control were absent. That is the finding the report will lead with.
Common Mistakes to Avoid
1. Relying on Inquiry Alone
Asking the control owner whether the control works and recording "yes" is self-attestation, not testing. Always corroborate inquiry with inspection or re-performance.
2. Testing Operation Before Design
Confirming a control runs without first checking it is capable of addressing the risk validates a control that may reduce no risk at all.
3. Sampling From One Period
A sample drawn entirely from one month can miss seasonal spikes, staff absences, or year-end pressure. Spread the sample across the full period.
4. Treating Exceptions as Noise
Every exception is a signal. Investigate root cause before concluding; a single systemic exception can be more serious than several isolated ones.
5. Not Updating Residual Risk
If a control is rated ineffective but the risk register still shows a low residual rating, the register is lying. Effectiveness conclusions must flow through to residual risk.
Summary
- Control effectiveness is whether a control actually achieves its risk-reduction objective, evidenced not assumed
- Always assess design first, then operating effectiveness
- Four testing methods (inquiry, observation, inspection, re-performance) differ in evidence strength
- Rate controls consistently (Effective, Partially Effective, Ineffective, Not Yet Operating)
- Sample across the full period and tie every conclusion to sufficient, reliable, relevant evidence
- A control's effectiveness directly determines how far residual risk falls below inherent risk
Frequently Asked Questions
What is the difference between design and operating effectiveness?
Design effectiveness asks whether the control, if performed as intended, would adequately address the risk. Operating effectiveness asks whether it is actually being performed as designed, consistently, throughout the period. A control must pass both: a well-designed control that is not performed, and a faithfully performed control that was never capable of addressing the risk, both fail.
Which testing method gives the strongest evidence?
Re-performance, because you independently re-execute the control and verify the result rather than relying on the control owner's output. Inspection of records is the next strongest and the most commonly used. Observation is moderate, and inquiry is the weakest, useful for understanding but never sufficient on its own to conclude on operation.
How large should a control testing sample be?
It depends on how often the control runs and how much you intend to rely on it. A common rule of thumb is around 25 items for a frequently performed (daily) control, fewer for less frequent controls, and all instances for a control that only runs a handful of times a year. Higher-risk controls warrant larger samples and stronger methods.
What happens to residual risk if a control is ineffective?
An ineffective control provides little to no risk reduction, so residual risk should stay at or near the inherent level. You are exposed almost as if the control were absent. This is why control effectiveness conclusions must flow through to the risk register; otherwise the residual ratings overstate how protected the organization really is. See inherent vs residual risk.
Can management assess control effectiveness, or only internal audit?
Both. Management performs control self-assessments as part of the first and second lines of defense, and internal audit provides independent assurance over those assessments. The key difference is independence and evidence: management self-attestation is a useful input, but internal audit's value lies in testing those claims with objective evidence.
How does control effectiveness relate to the audit plan?
Control effectiveness is a primary input to risk-based planning. Areas with weak or unproven controls carry higher residual risk and rise up the audit universe rankings, earning a place in the risk-based audit plan. Effectiveness conclusions from one engagement also shape where the function looks next year.
Save this guide for later
Download the PDF version to read offline or share with your team.
