What Is an API Rate Limit? Limits, Retries, Context
A rate limit caps how much traffic one key may send to an API in a given time. This page explains what is an api rate limit in practice, how to code around it, and what the limits are on this API.
Updated

Key points
- A rate limit restricts requests per minute or requests in flight, so one client cannot overload the service.
- Rate limits are not the same as the context window: one limits traffic, the other limits the size of a single request.
- Full key: 8 concurrent requests and 300 per minute. Trial key: 2 concurrent and 60 per minute.
- Handle limits with a concurrency cap, retries with backoff and a look at the response headers.
What a rate limit is
So, what is an api rate limit? It is a rule that says how many calls one account or key may make in a time window, or how many may run at once. If you go over it, the server refuses the extra calls instead of serving them. Most HTTP APIs signal this with the status code 429 (Too Many Requests), and many add a header telling you when to retry. Check the docs of each service for its exact behaviour.
Limits exist because a model server has a fixed amount of compute. Without them, one script in a tight loop could slow everyone else down. For you as a developer they mean two things: your code has to expect refusals and recover from them, and your design should spread work out instead of firing it all at once.
The three kinds of limit you will meet
Providers mix and match these, so read which ones apply to your key.
| Type | What it counts | Typical effect |
|---|---|---|
| Requests per minute | Calls started in a rolling minute | Bursts get refused even if each call is small |
| Concurrency | Calls in flight at the same moment | Parallel workers beyond the cap get refused or queued |
| Token or spend quota | Tokens or money used per minute, day or month | Large prompts hit the cap sooner than small ones |
Concurrency matters more for language models than for ordinary APIs, because a single completion can take many seconds. Ten workers each waiting on a long reply hold ten slots the whole time.
The limits on this API
This API limits requests per minute and concurrency, per key:
| Key type | Concurrent requests | Requests per minute |
|---|---|---|
| Full key (with credit) | 8 | 300 |
| Trial key | 2 | 60 |
Two other numbers are worth knowing. The maximum request body is 8 MB, and every response carries headers you can read: X-Request-Id to identify a call when you ask for help, X-Balance-USD for your remaining credit, and X-RateLimit-Limit-Requests and X-RateLimit-Limit-Concurrency for the limits that apply to your key. You can see them with curl:
curl -i https://api.abliteratedapi.com/v1/chat/completions -H "Authorization: Bearer $ABLITERATED_API_KEY" -H "Content-Type: application/json" -d '{"model":"uncensored","max_tokens":5,"messages":[{"role":"user","content":"Hi"}]}'
The limits are the same whichever client you use; the API is OpenAI-compatible, so any OpenAI SDK works once you change the base URL to https://api.abliteratedapi.com/v1.
Which limit will you actually hit?
Divide the two numbers. A full key can run 8 requests at once and 300 per minute, which is an average of 5 per second. To reach 300 per minute with only 8 in flight, each call would have to finish in 1.6 seconds on average (8 x 60 / 300). Replies from a language model usually take longer than that when they are more than a few sentences, so on a full key the concurrency cap tends to bind first.
Take a hypothetical case where the average call lasts 10 seconds. Eight slots then give you 8 x 60 / 10 = 48 requests per minute, well under 300. The same logic on a trial key: 2 slots and 60 per minute means calls under 2 seconds are needed before the per-minute cap matters. These durations are an illustration, not a measurement. Time your own calls and plug in the real figure.
The practical advice: size your worker pool to the concurrency limit, not to the request limit, and let the per-minute limit act as a safety net.
How to handle limits in code
Three habits cover almost every case.
- Cap parallelism yourself. A semaphore sized to your concurrency limit means you rarely get refused at all.
- Retry with exponential backoff and jitter. Wait 1 second, then 2, then 4, adding a random fraction so many workers do not retry in lockstep. Stop after a few attempts.
- Do not retry blindly. A malformed request or a bad key will fail every time. Retry only on rate-limit and temporary server errors.
The official OpenAI Python SDK already retries on 429 and server errors with backoff, and you control it with max_retries. Combined with a semaphore, a batch job looks like this:
import asyncio
import os
from openai import AsyncOpenAI
client = AsyncOpenAI(
base_url="https://api.abliteratedapi.com/v1",
api_key=os.environ["ABLITERATED_API_KEY"],
max_retries=5,
)
sem = asyncio.Semaphore(8) # use 2 on a trial key
async def ask(prompt):
async with sem:
r = await client.chat.completions.create(
model="uncensored",
messages=[{"role": "user", "content": prompt}],
)
return r.choices[0].message.content
async def main():
prompts = [f"Give one short fact about the number {n}." for n in range(40)]
answers = await asyncio.gather(*(ask(p) for p in prompts))
print(len(answers), "answers")
asyncio.run(main())
Forty prompts go through, never more than eight at a time. If you build your own client, put the same two ideas in it: a limit on in-flight calls, and a retry delay that grows.
Rate limits are not the context window
People often mix the two up, because both produce errors when you ask for too much. A rate limit controls how many requests you make. The context window controls how big one request can be.
On this API the context window is 100,000 tokens, counting the prompt and the completion together. If your prompt is 95,000 tokens, at most 5,000 are left for the reply. The max_tokens parameter is optional: leave it out and the completion can use whatever the window has left. Setting it is still a good habit when you know the answer should be short, because output tokens cost $0.50 per 1M against $0.15 per 1M for input. A full 95,000-token prompt costs 95,000 x 0.15 / 1,000,000 = $0.01425 in input.
A rate limit does not shrink the context window, and a big context does not use up extra rate limit. Each call is one request, whatever its size, as long as the body stays under 8 MB.
Streaming and rate limits
With stream: true the reply comes back over server-sent events, token by token, and the usage counts arrive at the end. A streamed call still occupies one concurrency slot until the stream finishes, so a slow reader that keeps connections open holds slots longer than you expect. Close streams you no longer need.
Common mistakes
- No worker cap. A thread pool with 50 workers against a limit of 8 means most calls get refused. Match the pool to the concurrency limit.
- Immediate retries. Retrying at once after a refusal adds load at the worst moment. Wait, and wait longer each time.
- Treating every error as a rate limit. A timeout, a bad key and an oversized body each need a different fix. Log the status code and the
X-Request-Idheader. - Sharing one key across everything. A nightly batch job and a live chat feature on the same key compete for the same 8 slots. Use separate keys, or schedule the batch for quiet hours.
- Ignoring slow streams. A client that stops reading but keeps the connection open still holds a slot.
Add a counter for refused calls to your logs. If it stays at zero, your cap is set correctly. If it climbs, lower the pool size before you add retries.
Planning around limits
Rate limits and budget are linked. Eight slots at full tilt can spend credit quickly, so decide how much you are willing to burn before you start a job. Pricing is pay as you go, with credit that does not expire and no charge for failed or refused requests. Read the pricing page for rates and top-up bonuses, and the quickstart for request parameters. To test your retry logic safely, get an API key and use the trial credit, which needs no card. Just remember it runs on the lower trial limits.
More guides
Questions and answers
What happens when I exceed a rate limit?
The server refuses the extra requests, typically with HTTP 429, and your code should wait and retry. Slow down or lower your parallelism until the errors stop.
What are the limits on this API?
A full key allows 8 concurrent requests and 300 requests per minute. A trial key allows 2 concurrent requests and 60 per minute.
Is a rate limit the same as the context window?
No. A rate limit controls how many requests you can send over time. The context window, 100,000 tokens here, controls how large one request and its reply can be.
How do I see my limits from a response?
Read the X-RateLimit-Limit-Requests and X-RateLimit-Limit-Concurrency response headers. The X-Balance-USD header shows your remaining credit, and X-Request-Id identifies a specific call.
Do refused requests cost money?
No. Failed or refused requests are not charged on this API.
Your key is one form away
Create an account, copy the key, change the base URL. That is the whole setup.
Get API key