Summary
TennCare Connect, an eligibility system built by Deloitte and others on which Tennessee spent over $400 million, automatically determines eligibility for Medicaid and other programs from income and health information. According to reporting, the system sometimes failed to load the proper data, assigned recipients to the wrong household, and produced incorrect eligibility decisions. In August 2024, the federal district court for the Middle District of Tennessee ruled that these defects illegally cut thousands of people off their benefits (reporting). Without a step that checks, before an action that is hard to undo such as a termination, that an automated decision’s inputs and assumptions match the source records, the decision passes as the basis for the action even on wrong inputs.
Pre-execution check = before cutting off benefits on an automated decision, check the inputs and household assumptions it used against the source records, and keep a record. Detection = noticing the error after the termination, through appeals or litigation. Dates follow reporting.
illegally cut thousands off their benefits
What got through
- What got through
- Terminating Medicaid benefits on the basis of TennCare Connect’s automated decision of “ineligible.” According to reporting, the system (1) sometimes failed to load the proper data, (2) assigned recipients to the wrong household, and (3) produced incorrect eligibility decisions. The federal court ruled that these defects illegally cut thousands of people off their benefits (in a class action filed in 2020).
- What reached the receiving side
- In general, what reaches the step that carries out a termination is the decision “ineligible.” The decision alone does not show which inputs, and which assumptions about household composition, it was made on.
- How it was detected
- According to reporting, the errors were contested only after the terminations, when recipients appealed and a class action was filed in 2020.
- Not disclosed
- The reporting reviewed does not say which data failed to load or by what route, what review steps existed before termination, or the status of disputes and remediation after the ruling.
Pre-execution check
The condition that holds regardless of the route is to check, before cutting off benefits on an automated decision, that the inputs and household-composition assumptions the decision used match the source records.
Without a step that checks, before an action that is hard to undo such as a termination, that an automated decision's inputs and assumptions match the source records, the decision passes as the basis for the action even on wrong inputs.
- Who approves
- The person who decides on the termination, and the source records for the inputs the decision used (records of income, household and so on). The origin of the inputs is shown separately from the system that makes the decision. The decision itself is no substitute for approving the termination.
- Where it is checked
- Before moving from a decision to a termination. Compare the inputs and household assumptions the decision used against the source records; if any input does not match or was not loaded, hold the termination until it can be confirmed.
- What is recorded
- Which decision was used for the termination, on which inputs and household assumptions, and under whose approval. It gives a trail for tracing the basis of the termination later, in an appeal or an audit.
The errors were contested only after the terminations, through appeals and litigation. The benefits and the access to medical care lost in the meantime cannot be undone afterwards. A case where an AI match led to an action that could not be undone without independent corroboration is Brief 012 (a wrongful arrest from facial recognition). A case where a record of “complete” passed without being checked against what it rested on is Brief 035 (Boeing 787 inspection records).
What Lemma does
Lemma builds the pre-execution check into the step that cuts off benefits on an automated decision. It does not decide whether the decision itself is correct or fair; outside the system that makes the decision, it checks that the inputs and household assumptions the decision used match the source records, and keeps a record.
Industries and operations to watch
- IndustryPublic sector
- Government bodies that use automated decisions in actions that are hard to undo, such as terminating benefits, and the vendors contracted to run those systems.
- IndustryHealthcare
- Organizations that decide medical benefits or coverage eligibility through automated decisions.
- OperationsApproval
- Deciding to terminate benefits on the basis of a decision.
Current controls versus Lemma
The route reported in this case is column 1 (terminating benefits on an automated decision). Column 2 is the stage, after the termination, of showing what it was based on.
| Control | Terminating benefits on an automated decision | Showing the basis of the decision after termination |
|---|---|---|
| Appeals process (contesting after the termination) | Not effective | Partial |
| Lemma (pre-execution check) | Effective | Effective |
Appeals are a path to correcting a wrongful termination, but the action goes ahead until the recipient notices it and appeals. Lemma, before the termination, compares the inputs and household assumptions the decision used against the source records, and does not proceed to termination on a decision that cannot be checked. Because it keeps which decision was used on which inputs and assumptions, the basis can be traced afterwards.
Note: Lemma’s check works only when it is built into the step that moves to termination, and the source records for the inputs are managed outside the system that makes the decision. If the source records themselves are wrong, the wrong data also passes. Lemma does not fix the decision rules or their correctness or fairness. The table compares what each control checks; it does not show whether this incident would have been stopped. Lemma does not detect or prevent fraud or intrusions.
Three things to confirm for organizations that use automated decisions in actions such as terminating benefits
- Before acting on a decision, do you check that the inputs and assumptions it used, such as household, match the source records?
- Do you have a step that keeps a decision with unloaded inputs or mismatched assumptions from being used in an action until it can be confirmed?
- Do you keep a record that can show afterwards which decision was used in an action, on which inputs and assumptions, and under whose approval?
Facts and timeline
- Subject
- TennCare Connect (the eligibility system for Tennessee’s Medicaid program, TennCare). Built and operated by Deloitte and others; the state spent over $400 million on it (reporting).
- Affected people
- TennCare and other recipients and applicants. The class action filed in 2020 represents adults and children who were cut off from benefits.
- Defects reported
- The system sometimes failed to load the proper data, assigned recipients to the wrong household, and produced incorrect eligibility decisions.
- Court ruling
- In August 2024, the federal district court for the Middle District of Tennessee ruled that defects in TennCare Connect illegally cut thousands of people off their benefits (reporting).
- Other states
- KFF Health News reported that Medicaid eligibility systems Deloitte runs in multiple states have been flagged for the same kinds of errors.
Glossary
Medicaid — The US program that pays medical costs from public funds for people who meet conditions such as income. It is run state by state.
TennCare / TennCare Connect — TennCare is Tennessee's Medicaid program. TennCare Connect is the system that automatically determines eligibility from income and health information.
Eligibility determination — Deciding whether an applicant or recipient meets the conditions for benefits. It depends on inputs such as household composition and income.
Timeline
- Wrongful terminations and ineligibility decisions surface around TennCare Connect’s eligibility determinations.
- A class action representing adults and children cut off from benefits is filed.
- The federal district court for the Middle District of Tennessee rules that defects in the automated decisions illegally cut thousands of people off their benefits.
- Errors in similar systems across multiple states are reported (KFF Health News).
The facts rest on reporting by Gizmodo, KFF Health News and Texas Dentists for Medicaid Reform. The characterization “illegally” rests on the federal court ruling as reported. The status of disputes and remediation after the ruling has not been confirmed. This Brief does not blame the state, the contracted vendors or any individual. What has not been disclosed does not mean there were no controls.
Response after disclosure
- Judiciary
- The federal district court for the Middle District of Tennessee ruled that defects in TennCare Connect illegally cut thousands of people off their benefits (2024-08, in the class action filed in 2020; reporting).
- Reporting
- KFF Health News reported that Medicaid eligibility systems Deloitte runs in multiple states have been flagged for the same kinds of errors.
Sources
- Gizmodo: “Judge Rules $400 Million Algorithmic System Illegally Denied Thousands of People’s Medicaid Benefits” (2024-08) — https://gizmodo.com/judge-rules-400-million-algorithmic-system-illegally-denied-thousands-of-peoples-medicaid-benefits-2000492529
- KFF Health News: “Medicaid for Millions in America Hinges on Deloitte-Run Systems Plagued by Errors” — https://kffhealthnews.org/news/article/medicaid-deloitte-run-eligibility-systems-plagued-by-errors/
- Texas Dentists for Medicaid Reform: “Judge Rules Thousands Illegally Booted off Tennessee Medicaid by Deloitte System Similar to Texas” (2024) — https://www.tdmr.org/judge-rules-thousands-illegally-booted-off-tennessee-medicaid-by-deloitte-system-similar-to-texas/
References: “The last layer left for cyber defense in the age of AI”