Product
Batch work should not need a new integration
One endpoint, whether you send a list or a file.
By Ehsan Gazar, 19 September 2026
You have ten thousand documents to classify and nobody waiting on the answer. Every provider will do it for less if you let them take their time. Then you read their batch documentation, and it is a new integration each time.
The discount is the easy part. The shape is what costs you.
Every vendor, its own way in
One provider wants a JSONL file uploaded first, then a batch created from the file's id. Another takes the requests inline in one JSON body and has no file step at all. The status names differ, the result formats differ, and so does the way a single failed item is reported.
So batching stops being a switch you flip on work that can wait. It becomes a project per vendor, with its own client code, its own polling loop and its own parser for the results.
Most teams do it once, for one vendor, and never again. Work that could run at the batch price keeps running at the live price, because moving it means another integration.
That is the problem worth solving, and a lower price does not solve it.
One endpoint, two ways in
Our gateway takes every batch at one address, POST /v1/batches, with the same key you already use for chat. There are two ways to say what to run, and a batch uses exactly one of them.
The first is a list, in the body. It suits a script that already has the requests in memory.
curl https://gateway.vatan.one/v1/batches \
-H "authorization: Bearer $VATAN_API_KEY" \
-H "content-type: application/json" \
-d '{
"model": "openrouter/openai/gpt-5-nano:batch",
"requests": [
{"messages": [{"role": "user", "content": "Classify: the invoice is wrong"}], "max_tokens": 50},
{"messages": [{"role": "user", "content": "Classify: my parcel never came"}], "max_tokens": 50}
]
}'
The second is a file. You upload a JSONL file with one request per line, then name it. This is the shape every OpenAI SDK sends from files.create and batches.create, so existing batch code can point its base URL at us and keep its calls.
{"custom_id": "ticket-1", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "openrouter/openai/gpt-5-nano:batch", "messages": [{"role": "user", "content": "Classify: the invoice is wrong"}], "max_tokens": 50}}
{"custom_id": "ticket-2", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "openrouter/openai/gpt-5-nano:batch", "messages": [{"role": "user", "content": "Classify: my parcel never came"}], "max_tokens": 50}}
curl https://gateway.vatan.one/v1/files \
-H "authorization: Bearer $VATAN_API_KEY" \
-F purpose=batch -F file=@tickets.jsonl
# {"id": "file-...", "lines": 2, "status": "processed"}
curl https://gateway.vatan.one/v1/batches \
-H "authorization: Bearer $VATAN_API_KEY" \
-H "content-type: application/json" \
-d '{"input_file_id": "file-...", "endpoint": "/v1/chat/completions", "completion_window": "24h"}'
Sending both a list and a file is refused rather than resolved. Either choice would surprise half the people who did it by accident.
What stays the same behind it
Whichever way the work came in, it becomes the same batch object. The same statuses, the same counts, the same results, paged at /v1/batches/{id}/results.
A discounted batch is sent to the provider in one piece and charged once. There is no moment halfway through at which your budget can say no, so the worst case is priced and checked before anything is sent. A vendor's own batch API leaves that part to you.
The model id carries the price. Only an id ending in :batch is accepted, so a batch can never quietly run at the live rate. Ask for an ordinary id and the answer is a refusal naming the batch id to use instead.
The endpoint is declared, not guessed. The same call takes chat completions, embeddings, images, speech, transcription and video, and names which one it is running.
Being told when it is done
A batch you have to poll is a batch you have to remember. A webhook taking batch.finished is told when one ends, with its counts and how it ended.
{
"id": "evt_...",
"event": "batch.finished",
"data": {"batch_id": "...", "api_status": "completed", "total": 2, "completed": 2, "failed": 0}
}
It arrives signed, and it is sent again if your receiver is down. Your code starts the batch, forgets it, and picks up the results when it is told.
The integration you write once should be the last one the next vendor asks for.