Best local LLM for pentesting: four abliterated models for one GPU
The best local LLM for pentesting is Qwen 3.8 27B Abliterated, which fits one 24 GB GPU or a 32 GB Mac at 4-bit.
Run locally when client data must stay on your machine, and expect less reasoning than a 2,780B API model.
Four abliterated models that run at home
Hugging Face cards, read November 18, 2026. Sizes at 4-bit are estimates.
| # | Model | Parameters | Size at 4-bit | Runs on | License | Note |
|---|---|---|---|---|---|---|
| 1 | Qwen 3.8 27B Abliteratedhuihui-ai/Huihui-Qwen3.8-27B-abliterated | 27.8B | about 17 GB | one 24 GB GPU or a 32 GB Mac | Apache 2.0 | Its GGUF build had over 4.3 million downloads in the last month. |
| 2 | Qwen 3.6 35B-A3B Abliteratedhuihui-ai/Huihui-Qwen3.6-35B-A3B-abliterated | 36B total, 3B active | about 22 GB | one 24 GB GPU | Apache 2.0 | Its MTP GGUF build had over 5 million downloads in the last month. |
| 3 | Gemma 4 31B Hereticcoder3101/gemma-4-31B-it-heretic | 31.3B | about 19 GB | one 24 GB GPU | Apache 2.0 | Abliterated with the Heretic tool. |
| 4 | Llama 3.1 8B Abliteratedmlabonne/Meta-Llama-3.1-8B-Instruct-abliterated | 8B | about 5 GB | an 8 GB GPU or a laptop | Llama 3.1 Community License | Published by mlabonne, whose 2024 write-up made abliteration popular. |
When local wins, and when it does not
Qwen 3.8 27B Abliterated has 27.8B parameters and Kimi K3.1 Pro has 2,780B. Strix's docs say "Most local models, especially those under 70B parameters, struggle with these complex tasks." (Strix docs) Shannon's say a model that does not follow its tool-use constraints reliably "will produce weaker pentests than a frontier model." (Shannon docs)
Strix says to raise Ollama's num_ctx to "at least 16k–32k" for agent work. Summarize on the local model, then send that summary out for path analysis.
Connect a local model to a pentest agent
Both routes are from each tool's own docs.
Strix reads Ollama or any OpenAI-compatible server, such as LM Studio or vLLM. From its local models guide.
export STRIX_LLM="ollama/<your-model-tag>" export LLM_API_BASE="http://localhost:11434" strix --target ./app-directory
Shannon takes a local server through models.json with "api": "openai-completions". Use host.docker.internal, not localhost, because the worker runs in Docker. See Shannon setup. The same tools take an API model by changing two variables. See Kimi K3.1 Pro.