When Does a Measure of Success Become an Obstacle to It?

A Meeting Where the Numbers Succeeded

In an imagined scene, a small team opens its weekly meeting with good news: the number of closed support requests has risen. Green indicators appear on the screen, and the manager prepares to thank everyone. Then an employee says that some customers returned with the same problem after their requests were closed. A brief silence follows. The number was not necessarily false; the requests really were closed. But a question had been missed: were the problems that brought people to the team resolved?

This scene was created for the article. It describes no particular company and is not a finding from field research. It presents a distance we can easily forget: the distance between what we want and the sign by which we recognise it. We want better service, so we count requests. We want deeper knowledge, so we count pages. We want good work, so we calculate completion speed. Counting is not necessarily wrong; error begins when we forget what counting was meant to serve.

This is a reflection on the ethics and limits of measurement, not a statistical guide to designing performance indicators or a management formula suitable for every organisation. It draws on a philosophical text, an analytical report, and a documented commentary, then develops its own examples and questions. Its purpose is to restore the measure’s function: helping us see the work, rather than replacing our view of it.

Opening Questions

• What purpose does the number you follow represent?
• Can this number improve without the work improving?
• Who bears the cost of what the measure leaves out?
• When is revising an indicator faithfulness to the goal rather than retreat from it?

1. What We Want Is Not Always What We Count

Suppose a library wants to encourage reading, so it tracks the number of books borrowed. This number is useful: it tells the library something about book circulation and use of its lending service. But it does not, alone, reveal what readers understood, who found a suitable book after lengthy assistance, or who needed a quiet place to read inside the library. The information remains true without carrying the whole story.

At the opening of the Nicomachean Ethics, Aristotle mentions the variety of actions and arts and of their ends:

Aristotle is not discussing modern performance dashboards here. From his distinction we borrow a philosophical question: what are we doing this for? If opening the library is a means of making knowledge available, the easiest number to collect should not become a complete definition of knowledge. This is an editorial connection we propose, not an attribution of contemporary management theory to an ancient text.

A number can be a good window if we know its frame. When we treat the window as the whole building, however, we lose what lies outside its view. We need not measure everything every time; that may be costly and distracting. We need to name the limits of what we measure and leave a door open to what it does not show.

In the imagined meeting, the number could have appeared under a more precise label: “Requests closed,” followed by another question about actual resolution. Renaming alone does not repair the service, but it prevents a hidden shift from describing a procedure to declaring success. The language around a number is part of its honesty, not decoration added once the calculation is finished.

2. When We Reward the Sign Instead of the Goal

The distance becomes more complicated when a number is tied to reward or punishment. A worker may improve it by actually improving service, or find another way to make it look better without changing what the customer needs. In our example, one can devote time to understanding the problem, or close the request prematurely. Both actions may move the same box, but their effects differ.

An original CNA report published by Michael Stumborg and colleagues in 2022 discusses the manipulation of measures under the heading of Goodhart’s Law. It explains that linking rewards to a measure can create incentives to improve the number at the expense of system effectiveness, and distinguishes measures of performance from measures of effectiveness. The report is aimed primarily at defence analysis. We draw on its conceptual distinction without generalising its examples or treating it as an experiment proving what happens in every company.

Among the report’s recommendations:

This does not mean every activity count is useless or every employee responding to incentives is dishonest. A team may need to know how much work it actually completed. Review begins when we discover that behaviour improving the indicator weakens the purpose for which it was chosen. Then it is not enough to ask people for greater enthusiasm for the same indicator.

Imagine a team writing customer guidance whose output is measured by the number of pages it publishes. It may split one guide into many pages to make production look greater, or hesitate to delete old, confusing pages because deletion would lower the total. These are hypothetical examples, not accusations against a profession or platform. They test a simple question: what action does this number encourage, and what good action might it penalise?

Distortion can also arise without direct falsification. A worker may select easy tasks because the evaluation method allows insufficient time for complex ones. We should therefore not reduce the problem to individual ethics alone; the evaluation design itself deserves examination. Conversely, poor design does not excuse anyone from responsibility for lying or concealing harm. Both responsibilities can coexist.

3. Accurate Calculation Does Not Guarantee Fairness

Adding numbers may be entirely correct while the comparison is unfair. If one employee handles simple requests and another handles cases requiring lengthy follow-up, comparing request counts alone may conceal the difference in work. We need invent no statistical finding to see the logical possibility: identical units of counting do not mean identical effort, difficulty, or effect.

Aristotle also draws attention to differences in the precision different subjects permit:

This does not license poor calculation or vague conclusions. When we can check a number, we should. But it helps separate the accuracy of arithmetic from whether what we are counting is sufficient for the required judgement. An average can be calculated without error yet not, alone, answer a question about service quality.

Fairness thus becomes a practical question: who has authority to interpret the number? If interpretation is confined to whoever designed the table, the knowledge of the worker facing difficult cases or the customer who returned may disappear. Listening to them does not cancel measurement; it adds information about its meaning and limits. The number may prove genuinely useful, or may need to be broken down, contextualised, or defined more clearly.

But context is not an unlimited licence to justify any result. An organisation may ask for an explanation open to review, distinguishing genuine complexity from an improvised argument following an unwanted number. Fairness combines accountability with the ability to challenge, instead of treating one as the other’s adversary.

4. Reviewing the Measure Is Part of Remaining Faithful to the Goal

Sometimes we defend an indicator because we invested time in building it. It has acquired a screen, meetings, and a monthly report, and changing it seems like admitting the entire project failed. But a tool is not an ethical pledge to remain unchanged. Faithfulness to the goal may require changing a tool that no longer helps us see it.

The imagined team could retain the count of closed requests while examining a sample of resolutions and asking customers whether the problem persists. This is neither an ideal formula nor a guarantee against every distortion; it illustrates refusing to let one number monopolise judgement. Real work may require a more specialised design, depending on its purpose, resources, and risks.

Changes should also be understandable to those being evaluated. If success is redefined after the work is completed without explanation, the indicator becomes an instrument of surprise rather than learning. We can explain why it is being revised, retain the previous definition, and distinguish periods that cannot be compared directly. This protects institutional memory from a new story that erases its predecessor.

A Serious Objection: Will This Flexibility Allow Evasion?

Yes, criticism of numbers can be used as a pretext to escape accountability. Someone in authority may prefer a vague criterion because it leaves them free to reward whomever they wish. The alternative to an imperfect indicator is therefore not unregulated personal judgement. We need a public definition of what we measure, evidence that can be examined, and clear reasons for any revision.

The objection reminds us that measures can protect people too: they make some differences visible and prevent complete reliance on impressions. The problem is not their existence but their immunity from questioning. A more balanced position preserves what is useful while acknowledging what they cannot say.

In the Light of the Quran and Islamic Wisdom

Allah says: ﴿وَأَقِيمُوا الْوَزْنَ بِالْقِسْطِ وَلَا تُخْسِرُوا الْمِيزَانَ﴾TRANSLATION OF THE MEANING · COMPLETE VERSE 55:9Therefore establish weights in justice, and do not give short measure.Rowwad Translation Center · QuranEnc · v1.0.20 [Ar-Rahman 55:9]. Al-Saadi interprets the command as establishing weight justly, and his explanation prohibits short measure and injustice. Verse and commentary.

We do not present the verse as a statistical theory or a text about modern management indicators. The editorial reflection here is that honesty in measurement is inseparable from justice in its use. We should not select a number because it serves our image, or hide its limits from those affected by a judgement built on it. This is an ethical application of the meaning, keeping the text, commentary, and application on clearly distinct levels.

Justice does not demand knowledge we do not possess. We may have limited data and still need a decision. But limited knowledge calls for honesty about its limits and revision of what proves wrong, not exaggerated certainty. Saying “This indicator does not cover these cases” may be part of fulfilling a trust well, rather than a weakness in presentation.

Practical Applications: A Question Before the Dashboard

Begin by writing the goal in a sentence containing no number. Then write what the indicator actually counts. If the two sentences match only because you replaced a broad word with an easy number, pause. What does “good service” mean in the recipient’s experience? Which part of that experience does the current measure capture?

Next, test the indicator with a counterexample: how could someone raise it without advancing the goal? Do not use this question to teach manipulation or accuse workers, but to examine the design before attaching rewards. If you find a clear possibility, look for additional evidence that would reveal it, or a modification that reduces the incentive.

Also ask who falls outside the table. Do you measure completed cases and forget those unable to access the service? Do you measure response speed while leaving out clarity? These are suggested questions, not a checklist sufficient for every sector. Choosing a suitable answer requires knowing the work and the people involved.

Finally, set a date to review the indicator’s suitability, not simply to ask people to raise it. Keep a record of why it was chosen, its definition, its limits, and changes to it. Allow room in the meeting for one documented case that may reveal what totals conceal, without turning that case, in its turn, into a judgement about everyone.

Conclusion: A Number Is a Witness That Needs Questioning

We return to the opening meeting. The team can celebrate its achievements while asking what remains. A customer returning with the same problem is not an insult to the table, but information about the work the table was designed to serve. Perhaps the best decision that morning was to pause the applause briefly so the question could be heard.

A good measure does not ask us to forget the goal; it returns us to it. We need numbers to avoid remaining captive to impressions, and responsible judgement to avoid becoming captive to numbers. When a sign conflicts with what it was meant to indicate, courage lies in reviewing the sign, not demanding that reality resemble it.

Questions for Reflection

• Which number in your day has replaced the question from which it began?
• What example reveals the limits of a measure you use?
• How can you allow challenges to evaluation without cancelling accountability?
• What will you retain, and what will you revise, in your definition of success?