Abliterated Models on Ollama vs a Hosted API
Searching for abliterated models ollama usually means one thing: you want a model that answers without refusals and you want it on your own machine. That works, and this article shows what it costs in memory, time and upkeep, then compares it with a hosted OpenAI-compatible endpoint.
Updated


Key points
- Ollama runs abliterated weights locally and exposes an OpenAI-style endpoint on port 11434.
- VRAM, not disk space, is the first limit you will hit; quantization trades quality for memory.
- A hosted API costs $0.15 per 1M input tokens and $0.50 per 1M output tokens, so a small app can cost $12.75 a month.
- Both routes speak the same chat completions format, so switching is a two-line change.
What abliterated means and how Ollama fits
Abliteration is a weight edit. Researchers find the direction in a model's activations that correlates with refusing a request, then remove it from the weights. The result is a model that tends to answer instead of declining. It is not a retrain, so the model keeps its general skills, and quality can shift slightly in either direction.
Ollama is a local runtime. You pull a model, it downloads the weights, and it serves them on http://localhost:11434. The server also offers an OpenAI-compatible route under /v1, which is why most chat front ends and SDKs can talk to it. Community members publish abliterated builds of popular open models, usually as GGUF files that Ollama can import.
So the pipeline is simple: find a build, pull it, point your client at localhost. The hard part is not the commands. It is the hardware and the care that comes after.
Running an abliterated model locally, step by step
First install Ollama from its official site. Then either pull a published build by name or import a GGUF file you downloaded yourself.
- Pull a published build with
ollama pull some-user/some-abliterated-model(the name is an example, search for one you trust). - Or import a GGUF file: write a Modelfile whose first line is
FROM ./model.Q4_K_M.gguf, then runollama create my-abliterated -f Modelfile. - Start a chat in the terminal with
ollama run my-abliterated.
To call it from code, use any OpenAI client with the local base URL. The key can be any non-empty string, since Ollama does not check it by default.
from openai import OpenAI
# local Ollama
local = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")
# hosted endpoint: https://api.abliteratedapi.com/v1 with model "uncensored"
hosted = OpenAI(base_url="https://api.abliteratedapi.com/v1", api_key="YOUR_KEY")
r = local.chat.completions.create(
model="my-abliterated", # hosted: "uncensored"
messages=[{"role": "user", "content": "Outline a heist scene for my novel."}],
)
print(r.choices[0].message.content)
If this runs, you have a working local setup. Notice that the second client in the snippet already points at the hosted endpoint; the rest of this article explains when that is the better choice. What follows first is the part that decides whether local stays useful: memory, speed and upkeep.
VRAM, speed and the quantization trade
A model has to fit in fast memory to run at a useful speed. A rough rule: at 4-bit quantization each parameter takes about half a byte, so an 8B model needs around 4 to 5 GB for weights, and a 70B model needs around 35 to 40 GB. Add room for the KV cache, which grows with context length. Long prompts can cost several extra gigabytes.
When the model does not fit in GPU memory, Ollama spills layers to system RAM and the CPU. It still answers, but generation slows down a lot. People who see 2 tokens per second on a 24 GB card with a large model are usually in this situation.
- Smaller model: fits easily, answers fast, writes less well on long, nuanced scenes.
- Lower-bit quantization: saves memory, costs some coherence. Q4 is a common middle point; Q2 and Q3 often show visible damage.
- Shorter context: saves cache memory. Many local setups run at 8K or 16K tokens to stay fast.
Abliteration adds one more variable. Because the weights are edited, a build can be a little less stable than the original, with more rambling or odd repetition. Test any build on your own prompts before you rely on it.
Upkeep nobody lists in the tutorial
Local is free per token, but not free of work. Expect to spend time on these:
- Picking builds. Many abliterated uploads exist, with different base models, methods and quantizations. Reading the card, checking the license and testing outputs is your job.
- Updates. New base models appear, runtimes change defaults, and a build that worked can behave differently after an Ollama upgrade.
- Concurrency. One GPU serves one or two requests well. If three users chat at once, they queue.
- Availability. The app is up only while your machine is. A laptop that sleeps takes your bot offline.
- Safety around the output. A model without refusals needs your own rules about who can reach it. Keep it for lawful adult use, and never let it near anything involving minors.
One more cost is attention. When a reply looks wrong, you must work out whether the prompt, the quantization, the template or the build is at fault. With a hosted endpoint you still debug prompts, but the model file and runtime are not your problem. Keep a short log of which build, quantization and settings produced good results, because you will forget within a month.
None of this is a reason to avoid local. If you own a good GPU and enjoy tuning, it is a fine hobby and gives full privacy. The question is whether it is the cheapest route for an app other people use.
The hosted alternative and a monthly cost sketch
The hosted option is an OpenAI-compatible endpoint serving one open-weight model tuned to answer without content refusals for lawful adult use. You send requests to POST /v1/chat/completions with model uncensored. It is text only, with a 100,000 token context window, streaming and tool calling. It is not an official API of any model vendor. See the API docs for the full method list.
Pricing is $0.15 per 1M input tokens and $0.50 per 1M output tokens, from prepaid credit with no subscription. Failed or refused requests are not charged, and credit does not expire. Here is a sketch for a small chat app with 1,000 requests a day, averaging 1,500 input tokens and 400 output tokens:
| Item | Tokens per day | Rate | Cost per day |
|---|---|---|---|
| Input | 1,000 x 1,500 = 1.5M | $0.15 per 1M | $0.225 |
| Output | 1,000 x 400 = 0.4M | $0.50 per 1M | $0.20 |
| Total | $0.425 |
Over 30 days that is $12.75. Compare that with the electricity, the hardware share and your own hours for a local box. If you already own the GPU, local still wins on cash; if you would need to buy or rent one, check current rental quotes against this number. The pricing page lists top-ups, which start at $20 and add a bonus from $50.
Calling the hosted endpoint with the same client
Because Ollama and the hosted API both speak the chat completions format, the code you wrote above barely changes. Swap the base URL, the key and the model name:
from openai import OpenAI
client = OpenAI(
base_url="https://api.abliteratedapi.com/v1",
api_key="YOUR_KEY",
)
r = client.chat.completions.create(
model="uncensored",
messages=[{"role": "user", "content": "Outline a heist scene for my novel."}],
stream=False,
)
print(r.choices[0].message.content)
A practical pattern is to keep both. Develop against local Ollama when you are offline or testing prompts, and send production traffic to the hosted endpoint. Read the base URL from an environment variable and nothing else in your app needs to know.
New accounts can claim $0.50 of trial credit with one click on the API key page, valid for 7 days and without a card. Note the limits: a trial key allows 2 concurrent requests and 60 requests per minute, while a full key allows 8 and 300. The article on API rate limits explains how to handle 429 responses.
How to choose
Choose local Ollama when you already have a GPU with enough memory, you want data to stay on your machine, you tolerate slower generation on large models, or the project is personal. Choose the hosted endpoint when other people use the app, you need several requests at once, you want a 100K context without buying memory for it, or you do not want to maintain model files.
A simple test: count the hours you spent on setup this month and put a price on them. For a hobby that number is zero because you enjoy it. For a product with users, it is real money, and a pay-per-token bill of a few dollars may be smaller. Also consider the failure case. If your home machine reboots at night, a hosted endpoint keeps answering; a local server does not.
The two are not exclusive, and trying the hosted trial costs nothing. Run the same ten prompts through both and compare the answers and the time you spent.
More guides
Questions and answers
Can Ollama run abliterated models?
Yes. Ollama can pull published builds or import GGUF files you download. Once loaded, the model is served locally with an OpenAI-compatible route under /v1.
How much VRAM do I need?
At 4-bit quantization, count about half a byte per parameter plus KV cache. An 8B model fits in roughly 5 to 6 GB with modest context, while a 70B model needs well over 35 GB.
Is an abliterated model the same as the original?
No. The refusal behavior is edited out of the weights, and quality or stability can shift slightly. Test a build on your own prompts.
Does the hosted API work with Ollama-compatible clients?
Any client that lets you set an OpenAI base URL and key works. Use the base URL https://api.abliteratedapi.com/v1 and model uncensored.
Your key is one form away
Create an account, copy the key, change the base URL. That is the whole setup.
Get API key