Thanks for reading! Subscribe for free to receive new posts.
Discussion
Be the first to share your thoughts on this article.
When “Done” Isn’t Defensible
The hidden cost of evidence debt in regulated work — and a five-minute test for whether your organization can defend why critical work was considered complete.
Not by showing a green dashboard. By tracing the requirement to the work, evidence, validation, and decision.
If a regulator, executive, client, or new program leader asked you this afternoon to prove why one of your most important requirements is complete, how long would it take?
Not to show them a green dashboard.
To actually show what the requirement was, who owned it, what work was performed, what evidence supported the result, who validated that evidence, and why the organization decided the requirement had been satisfied.
That difference matters because the cost of a weak execution record rarely appears on a project plan. It shows up later as rework, delayed decisions, disputed deliverables, regulatory exposure, lost institutional knowledge, audit findings, or money that can no longer be recovered.
Evidence debt is the future cost an organization creates when work moves faster than its ability to preserve why it believes the work was done correctly.
Like other forms of debt, it is easiest to ignore while everything appears to be going well.
I Learned This During a Medicaid System Release
I saw the distinction firsthand while supporting independent verification of a Medicaid eligibility-system release.
The testing itself was not the hardest part.
Could we demonstrate why the system was ready to be released?
A completed test schedule wasn't enough. We needed to trace requirements to test evidence, identify discrepancies, show how they were resolved, retest where necessary, and preserve the basis for the final release decision.
In other words, “done” wasn't the standard.
That changed how I look at implementation work because the same problem shows up far beyond technology.
Organizations are generally good at tracking activity. They know which milestones are late, which tasks are open, whether a vendor submitted a deliverable, and whether a project is red, yellow, or green.
What they cannot always answer is a more important question:
You Can Test for Evidence Debt Right Now
Pick one high-consequence requirement your organization says is complete. Just one.
The five-minute test
Check each question you can answer quickly, without pulling the original team into a meeting.
If the answer is, “We need to get the right people in a room,” you've learned something important.
The work may be finished. Your organization may not be able to prove why it believes that.
Then the Archaeology Begins
This is where evidence debt becomes visible.
A new regulation, policy directive, contract requirement, corrective action, grant condition, or system change arrives. Teams meet. Responsibilities are assigned. People do the work. The dashboard eventually turns green.
Six months or a year later, somebody asks why a particular decision was made.
If the organization needs the person who remembers what happened in order to explain what happened, institutional memory is filling a gap that the operating record should have filled.
The Problem Usually Isn't Missing Documentation
Most organizations have plenty of documentation. The problem is that the pieces are disconnected.
The requirement is in one document. The interpretation is in meeting notes. The task is in a project-management system. The evidence is in a shared drive. The approval is buried in email. The exception is sitting in somebody's spreadsheet.
Where did the requirement come from? What did it require us to do? Who owned it? What happened? What proves it happened correctly? Who validated that proof? What decision followed?
The ideas themselves are simple. The value comes from keeping them connected.
If those answers live in seven different places and depend on five different people remembering the context, the organization has work happening.
It does not necessarily have control of the work.
No, the Answer Isn't More Paperwork
Does every task now need an audit trail, multiple approvals, and another governance process?
Match the evidence to the consequence
Nobody needs an evidence architecture to prove that office supplies were ordered. A Medicaid eligibility determination is different. So is a production system release, a multimillion-dollar payment control, a patient-safety requirement, or a regulatory commitment.
The strength of the evidence should match the consequence of getting the decision wrong.
The goal isn't to document everything.
The work that matters most needs enough traceability that leadership doesn't have to guess.
Auditors Eventually Ask the Same Question
A 2024 review by the U.S. Department of Health and Human Services Office of Inspector General provides a useful example.
California processed Medicaid eligibility renewals or terminations for more than 1.8 million enrollees during the period OIG examined. Auditors sampled 140 cases, identified nine incorrect eligibility actions, and estimated from that sample that roughly 78,853 eligibility actions may have been completed incorrectly.
The problems were not simply documentation failures. OIG also identified training and system issues.
But one recommendation is especially relevant here: OIG recommended that California revise its guidance so counties document essential information in case files supporting continuing-eligibility decisions.
Source: U.S. Department of Health and Human Services, Office of Inspector General .
A determination existed. The auditor also wanted the record supporting that determination.
A decision and a defensible decision are not always the same thing.
Sometimes the Information Exists and the Control Still Fails
New York shows the problem from another angle.
State auditors identified $93.7 million in improper Medicaid payments during periods when beneficiaries had overlapping Medicaid and Essential Plan coverage. The state administered both programs and had enrollment information for both, but the necessary payment control was not applied correctly.
By a 2025 follow-up, auditors reported that $36.6 million of the original improper payments was no longer recoverable because applicable lookback periods had expired.
That wasn't primarily a documentation failure. It was an execution failure.
But it demonstrates the broader issue: having information somewhere is not the same as having an operating model that reliably turns that information into the right action.
Organizations often respond to problems by asking for more data.
Sometimes they already have the data.
What they are missing is the connection between the requirement, the owner, the control, the evidence, and the decision.
Policy Approval Creates the Same Challenge
I ran into the same issue from a different direction while supporting implementation of Hawaiʻi's Community Palliative Care benefit.
The benefit had an implementation start date of January 1, 2025.
But an approved benefit did not automatically create an operating model.
Health plans still had to transition members from existing supportive-care arrangements. Reimbursement rules had to work in practice. Services and supplies had to move without disrupting care. Provider and health-plan responsibilities had to become operational rather than theoretical.
The state's implementation guidance addressed those practical responsibilities, including the need to manage transitions and avoid gaps in care.
Policy can establish what should happen. It cannot, by itself, make the operating environment ready to do it.
This is where policy becomes operations.
Someone has to take “Here is what the policy requires” and turn it into “Here is who does what, under what conditions, using what process, with what evidence, and how we will know whether it is working.”
That work is easy to underestimate.
It is also where implementation succeeds or quietly begins to drift.
“Complete” and “Ready” Are Different Answers
Is it done?
Tracks whether an activity, task, deliverable, or milestone ended.
What supports our conclusion that we're ready?
Tests whether the result is valid, sufficient, traceable, and safe to rely on.
A technology team completed the configuration. Was it validated against the requirement?
A vendor submitted the deliverable. Was it actually acceptable?
Training occurred. Did the right people complete it?
A corrective action was closed. Was the underlying problem corrected?
A policy was approved. Can the operating environment actually carry it?
Those are not academic distinctions.
They are the difference between managing activity and managing execution.
Evidence Debt Doesn't Always Look Like an Audit Finding
And sometimes it is an auditor asking a question nobody expected to have to answer.
Every one of those situations has a cost.
Some are measured in dollars. Others are measured in time, capacity, confidence, or risk.
Start With One Requirement
You don't need to redesign your entire operating model tomorrow to find out whether this problem exists.
Start with one important requirement your organization says is complete.
Start with the requirement. Follow it to the obligation, the owner, the work, the evidence, the validation, and the decision.
Then ask: Could someone outside the project understand why we said this was done?
If you can follow that trail in five minutes, that's a good sign.
If the answer is, “We need to schedule a meeting with everyone who remembers what happened,” that's a different answer.
Organizations already have more documents than they know what to do with. The goal is not to create more. The goal is to preserve the logic behind the decisions that matter.
Because eventually someone will ask: How do you know?
The work may be done. But “done” and “defensible” are not the same thing.
Sources and Further Reading
See the Medicaid System Release IV&V case study for an example of evidence, exceptions, and release-readiness traceability.
Irwin James Chapman · Ono Logic
Helping Medicaid, healthcare, and public-sector teams turn complex requirements into accountable execution, readiness evidence, and defensible decisions.
Article in one sentence
High-consequence work creates evidence debt when the organization completes the activity but cannot quickly defend why it considered the requirement satisfied.