Summary
According to Truffle Security’s study, the Hugging Face platform was scanning public pushes and notifying authors of the keys it found. Even so, many keys stayed unrevoked after notice, and the same keys were copied again and again into datasets derived from upstream corpora. Most of the keys that could be traced had not been leaked by their owners but had been scraped in from someone else’s corpus. The study is research, not a report of widespread exploitation. Unless the origin and terms of use of data are checked before a dataset is ingested or republished, a key leaked anywhere is republished with every derivative, and removing it from the original data leaves it in the copies.
A pre-execution check means checking the origin and terms of use of data before a dataset is ingested or republished, and keeping a record. Detection means finding keys in data after publication and notifying someone; here, the platform's scanning and notification, and Truffle Security's study, did that.
were copied into derivative datasets
What got through
- What got through
- Uploading data containing valid credentials to public datasets, and republishing it through derivatives and re-uploads. According to Truffle Security, keys leaked on GitHub, the open web, chat logs and elsewhere, entered upstream corpora such as The Stack and Common Crawl, and were republished in each dataset derived from them.
- What the receiving side gets
- In general, the side ingesting a dataset receives the files and the dataset’s name and download count, but not where the data inside came from or on what terms it was published. In general, each service that accepts a key receives the key itself, not who is using it.
- The key owners
- Of the Hugging Face tokens Truffle Security could trace, about 700 had been scraped in from someone else’s corpus; 63 were leaked by the token’s own owner. The firm says a notification only helps if someone acts on it, and the rate of action falls well short of 100%.
- Not disclosed
- Whether the keys found were actually misused (Truffle Security says this is not a report of widespread exploitation). The names of individuals, companies and datasets. The contents of the repositories whose scans did not complete.
Pre-execution check
The condition that holds whatever the route is this: before a dataset is ingested for training or republished, check the origin and terms of use of its data.
Unless the origin and terms of use of data are checked before a dataset is ingested or republished, a key leaked anywhere is republished with every derivative, and removing it from the original data leaves it in the copies.
- Who approves
- The owner of the data’s origin. A dataset’s name or download count does not stand in for its origin and terms of use.
- Where the check happens
- Just before a dataset is ingested for training, and just before it is republished as a derivative. Data whose origin and terms of use cannot be checked is not ingested or republished.
- What remains
- A record of which data went into the dataset under which origin and terms, so that when a key is found later, the derivatives it was copied into can be traced.
The platform’s scanning and Truffle Security’s study counted valid keys concretely, but they found them after the datasets had been published and copied. A case where Truffle Security counted valid credentials in a public corpus is live credentials in Common Crawl. The point that being public is not the same as being fit to train on echoes personal data in training sets.
What Lemma does
Lemma builds the pre-execution check into the point just before a dataset is ingested or republished. It does not search data for keys or revoke keys; it checks the origin and terms of use of the data before ingestion.
Industries and operations to watch
- IndustryAI
- Organizations that train models on public datasets, or that publish or redistribute datasets.
- IndustryDevelopers & agent operations
- Developers who issue and use API keys and tokens. A leaked key can be copied into datasets its owner does not know about.
- OperationDisclosure & data release
- Publishing or republishing collected data as a dataset.
- OperationAccounts & permissions
- Issuing and revoking keys.
Current controls vs Lemma
Column 1 is finding keys and notifying someone, column 2 is making keys unusable, and column 3 is ingesting or republishing a dataset. In Truffle Security’s account, column 1 worked, but many keys were never revoked and kept being copied through column 3.
| Control | Finding and notifying keys | Revoking keys | Ingesting or republishing a dataset (origin and terms of use) |
|---|---|---|---|
| Secret scanning of public pushes, with author notification | Partial | Partial | Not relevant |
| Auto-revocation of Hugging Face tokens in Enterprise organizations | Partial | Partial | Not relevant |
| Lemma (pre-execution check) | Not relevant | Not relevant | Effective |
Scanning and notification find keys that were pushed and tell the author, but for keys scraped in from someone else’s corpus, the author is not the key’s owner. Revocation moves forward only when the person notified acts. Auto-revocation is limited to Hugging Face tokens in Enterprise organizations. Neither decides whether a dataset may be ingested. Lemma checks the origin and terms of use of data before ingestion or republication. It does not find keys, revoke them, or delete data.
Note: Lemma’s check works only when it is built in just before a dataset is ingested or republished and the data’s origin and terms of use are issued and managed separately from the collector. Data that has an origin and terms of use passes this check even if it contains keys. Finding and revoking leaked keys is the job of secret scanning and the key owner’s own process; Lemma does not replace them. The table compares what each control checks; it does not show whether this incident would have been stopped. Lemma does not detect or prevent intrusions.
Three items to confirm about the datasets you use for training or distribution, and about your own keys
- Do you select datasets to ingest by a criterion other than their name or download count?
- Do you record which data went into a dataset under which origin and terms?
- When you are told one of your keys was found in public data, have you decided who carries it through to revocation?
Facts and timeline
- Target
- Every public dataset on Hugging Face: about 815,000 repositories, 186.9 million files and 7.6 PB, about nineteen times Truffle Security’s previous largest scan (~400 TB).
- Method
- Parquet, Arrow, JSONL, archives and binaries were flattened into scannable text and TruffleHog was run in verification mode, Truffle Security reports.
- What was found
- 221,303 still-valid credentials across 6,003 datasets. One of them could reach about 393 GB of personal data (by the firm’s estimate, about 3.7% of the world’s population), the firm reports.
- Breakdown
- 349 GitHub personal access tokens (223 with full repo write, 130 able to rewrite CI, 112 with
admin:org, 110 able to publish packages), 318 Docker Hub tokens, and 787 Hugging Face tokens (237 with write, 70 org-admin). 8,557 GCP service-account keys (3,811 projects), 8,594 working database logins, and 11,496 AI provider keys (1,210 datasets; OpenAI, Azure OpenAI, Anthropic, Gemini, Groq and others). npm and PyPI were checked specifically and returned no valid keys. - Replication
- 44% of live keys appeared in more than one dataset, and 19,380 in ten or more. In one reported example, an Infura key pasted once into a ChatGPT conversation passed through the WildChat chat-log dataset and was copied into 1,131 datasets across 10,162 file locations.
- Impact
- Inference that could be run with the leaked AI provider keys is estimated at no less than about $920,000 a year (Truffle Security).
- Platform measures
- Hugging Face already scans public pushes with TruffleHog and notifies the author on detection. For Enterprise organizations, a Hugging Face token pushed to a public repo or bucket is auto-revoked on the spot.
Glossary
TruffleHog — Truffle Security's tool for finding credentials in files and repositories. In verification mode it queries each service to see whether a found key still authenticates.
Live credential — An API key, token or login that verification confirmed still authenticates.
Upstream corpus — A large body of web or code data, such as The Stack or Common Crawl, from which many datasets are derived.
Timeline
- 2024-09-04
- Hugging Face announces secret scanning through its partnership with TruffleHog.
- Before publication
- Truffle Security sends the highest-impact findings (including the exposure behind the ~393 GB of personal data) to the affected parties and providers first, and holds publication until the most critical have confirmed receipt.
- 2026-06-01
- Truffle Security (Dylan Ayrey and the firm’s research team) publishes its findings.
This is research, not a report of widespread exploitation. No individuals, companies or datasets are named. Valid keys were verified against each provider, and the figures are verified ones. Reach was checked from metadata only; no data was read, copied or modified. About 670,000 of the roughly 815,000 target repositories were scanned to completion, so the figures are a lower bound for that subset. The counts and the notification picture come from Truffle Security’s report. This Brief does not blame key owners or the platform. What has not been disclosed does not mean no measures were in place.
Response after disclosure
- Hugging Face
- CTO Julien Chaumond contributed native storage-bucket scanning support to TruffleHog.
- Truffle Security
- Says a scan after that contribution has surfaced a large volume of new keys, and plans a follow-up report.
Sources
- primary, the research itselfTruffle Security Dylan Ayrey, “Scanning 7.6 Petabytes of HuggingFace Training Data for Secrets”
- primary, responseHugging Face “Secrets Scanning” (Hub documentation)
- primary, partnershipHugging Face Luc Georges, “Hugging Face partners with TruffleHog to Scan for Secrets”
References: “The last layer left for cyber defense in the age of AI”