AvailableInference · Language models

Language models

Use OpenAI Chat Completions or Responses with a Tarka base URL and a model alias returned by model discovery.

Provisioning and managementUse the console or the Control API.

Manage inference keys, model access, and organization limits in the console or through the Control API. Chat requests themselves go to the Inference API.

Open console Control API reference

Quickstart

Create a chat completion

Create a server-side key with the inference scope and set it as TARKA_API_KEY.

Pythonpip install openai

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://tarka.rest/v1",
    api_key=os.environ["TARKA_API_KEY"],
)

completion = client.chat.completions.create(
    model="himalaya-q8",
    messages=[
        {"role": "user", "content": "Reply with READY."},
    ],
)

print(completion.choices[0].message.content)

ShellPOST /chat/completions

curl "https://tarka.rest/v1/chat/completions" \
  -H "Authorization: Bearer $TARKA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "himalaya-q8",
    "messages": [
      {"role": "user", "content": "Reply with READY."}
    ]
  }'

Model discovery

Available language-model aliases

A key can call only the aliases assigned to it. Query model discovery at runtime and use the returnedid as the model value.

ShellGET /v1/models

curl "https://tarka.rest/v1/models" \
  -H "Authorization: Bearer $TARKA_API_KEY"

Family	Alias	Precision	Choose it for
GLM-5.3 Flash	`glm-5.3-flash`	—	High-volume chat, extraction, and classification
GLM-5.3 Flash	`glm-5.3-flash-eu`	—	European provider endpoint
HimalayaGPT 0.5B	`himalaya-q4`	Q4 K M	Smallest footprint
HimalayaGPT 0.5B	`himalaya-q8`	Q8	Balanced default
HimalayaGPT 0.5B	`himalaya-bf16`	BF16	Highest-precision variant
HimalayaGPT 0.5B	`himalaya-q4-private`	Q4 K M	Private · no internal retention
HimalayaGPT 0.5B	`himalaya-q8-private`	Q8	Private · no internal retention
HimalayaGPT 0.5B	`himalaya-bf16-private`	BF16	Private · no internal retention
DeepSeek V4.1 Flash	`deepseek-v4.1-flash`	—	Fast general-purpose chat, tool use, and coding
DeepSeek V4.1 Flash	`deepseek-v4.1-flash-eu`	—	European provider endpoint
Himalaya Gemma 4 E2B	`himalaya-gemma-4-q4`	Q4 K M	Smallest footprint
Himalaya Gemma 4 E2B	`himalaya-gemma-4-q8`	Q8	Balanced default
Himalaya Gemma 4 E2B	`himalaya-gemma-4-bf16`	BF16	Highest-precision variant
Himalaya Gemma 4 E2B	`himalaya-gemma-4-q4-private`	Q4 K M	Private · no internal retention
Himalaya Gemma 4 E2B	`himalaya-gemma-4-q8-private`	Q8	Private · no internal retention
Himalaya Gemma 4 E2B	`himalaya-gemma-4-bf16-private`	BF16	Private · no internal retention
Kimi K3	`kimi-k3`	—	Coding agents and long, multi-step tasks
GLM 5.3	`glm-5.3`	—	Chat and tool calls through the US provider endpoint
Qwen 3.8 Flash Next	`qwen3.8-flash-next`	—	Free chat and tool calls

Model access can be narrower for an organization or individual key. If an alias does not appear inGET /v1/models, create or update the key in the console or through the Control API before sending inference requests.

Endpoint reference

POST /chat/completions

Send an ordered conversation and receive an OpenAI-shaped completion or server-sent event stream. Generate clients and tooling from the publicTarka inference OpenAPI contract.

modelstring · required

An alias returned by GET /models.

messagesarray · required

Ordered system, user, assistant, and supported tool messages.

streamboolean · default false

Return SSE chunks when true.

max_tokensinteger · optional

Maximum generated tokens, bounded by the model.

temperaturenumber · optional

Sampling randomness. Support and range are model-specific.

top_pnumber · optional

Nucleus sampling threshold.

stopstring or array

One or more generation stop sequences.

seedinteger · optional

Requests repeatable sampling where supported.

toolsarray · optional

Tool definitions for aliases that advertise tool support.

userstring · optional

Your end-user identifier for abuse and audit correlation.

Request

JSONrequest body

{
  "model": "himalaya-q8",
  "messages": [
    {"role": "system", "content": "Answer concisely."},
    {"role": "user", "content": "Why is the sky blue?"}
  ],
  "temperature": 0.2,
  "max_tokens": 256,
  "stream": false
}

Response

JSON200 OK

{
  "id": "chatcmpl_01K3...",
  "object": "chat.completion",
  "created": 1787596800,
  "model": "himalaya-q8",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Sunlight is scattered by molecules in the atmosphere..."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 24,
    "completion_tokens": 31,
    "total_tokens": 55
  }
}

Unknown OpenAI fields are preserved for compatible providers.

Model capabilities still vary. Discover the alias first and do not assume that every model supports tools, images, structured output, or every sampling option.

Unified generation

POST /responses

Send text or message input and receive an OpenAI-shaped Response. Stored Responses can be retrieved, chained with previous_response_id, cancelled while active, and deleted.

OpenAI Python client

Pythonclient.responses.create

response = client.responses.create(
    model="himalaya-q8",
    input="Reply with READY.",
)

print(response.output_text)

HTTP request

ShellPOST /v1/responses

curl "https://tarka.rest/v1/responses" \
  -H "Authorization: Bearer $TARKA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "himalaya-q8",
    "instructions": "Answer concisely.",
    "input": "Why is the sky blue?",
    "max_output_tokens": 256,
    "store": true
  }'

modelstring · required

An alias returned by GET /models.

inputstring or input-item array · required

Text, messages, function calls, or function-call outputs.

instructionsstring · optional

Developer instructions prepended to the input.

streamboolean · default false

Return typed Responses API events as SSE.

storeboolean · default false

Explicitly retain tenant-scoped response state for 30 days or until deletion. Existing clients using previous_response_id must send store: true. This is independent of bucket capture.

previous_response_idstring · optional

Continue from a stored response owned by the same organization.

toolsarray · optional

Function and custom tool definitions. Hosted OpenAI tools are not accepted.

max_output_tokensinteger · optional

Maximum generated output tokens.

Stored response lifecycle

Shellretrieve · cancel · delete

curl "https://tarka.rest/v1/responses/resp_0123456789abcdef" \
  -H "Authorization: Bearer $TARKA_API_KEY"

curl -X POST "https://tarka.rest/v1/responses/resp_0123456789abcdef/cancel" \
  -H "Authorization: Bearer $TARKA_API_KEY"

curl -X DELETE "https://tarka.rest/v1/responses/resp_0123456789abcdef" \
  -H "Authorization: Bearer $TARKA_API_KEY"

Streaming emits typed events such as response.created, response.output_text.delta, and response.completed.
background: true, hosted tools, Conversations, and reusable prompt templates are not part of the supported contract.
Deletion follows the official OpenAI Python client contract and returns no SDK object.

Native gRPC

Call the same operations with protobuf

Use tarka.inference.v2.InferenceService at grpc.tarka.rest:443. The package name remains v2 for wire compatibility; canonical REST is /v1.

grpcurlCreateResponse

grpcurl \
  -H "authorization: Bearer $TARKA_API_KEY" \
  -d '{"body":{"model":"himalaya-q8","input":"Reply with READY."}}' \
  grpc.tarka.rest:443 \
  tarka.inference.v2.InferenceService/CreateResponse

Language-model RPCs include raw OpenAI Chat Completions and Responses unary/stream methods, typed Chat Completions methods, model discovery, and stored Response retrieve/delete/cancel methods.

Streaming

Consume server-sent events

Set stream: true. Each event contains a chat.completion.chunk and the stream ends with data: [DONE].

PythonOpenAI SDK

stream = client.chat.completions.create(
    model="himalaya-q8",
    messages=[{"role": "user", "content": "Write one short sentence."}],
    stream=True,
)

for chunk in stream:
    if not chunk.choices:
        continue
    delta = chunk.choices[0].delta.content
    if delta:
        print(delta, end="", flush=True)

Wire formattext/event-stream

data: {"id":"chatcmpl_01K3...","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":null}]}

data: {"id":"chatcmpl_01K3...","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"Hello"},"finish_reason":null}]}

data: {"id":"chatcmpl_01K3...","object":"chat.completion.chunk","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: [DONE]

Do not assume every chunk contains text; role, finish, or usage chunks may not.
Treat [DONE] as the clean end of the stream.
If the connection closes before [DONE], treat the generation as incomplete.

Data retention

See the inference capture guide for console setup, REST and MCP examples, stored formats, and delivery health.

Free Himalaya aliases retain request inputs and outputs in a Tarka-owned archive for evaluation and debugging until Tarka administrators delete them. They are not used for model training.

Aliases ending in -private have no Tarka internal retention. They use the inference key scope and are currently priced at zero. Other model families also have internal retention off.

Organization owners and admins can separately enable capture in Object Storage. All model calls then write JSON records and media into that organization’s inference-history bucket. Captures remain until deleted by default, with optional 7-, 30-, or 90-day expiry. Changing the setting affects new requests; disabling does not delete existing objects.

Storage failures do not stop inference. Delivery status shows capture gaps. Download captures through S3; permanent deletion in versioned buckets must remove historical versions as well as the current object.

Uploaded audio and inline images are saved as binary objects beside each request’s JSON record. Remote media URLs are saved as sanitized references; their remote content is not copied. Large requests and interrupted deliveries can also leave capture gaps.

The Responses API’s store flag controls only its separate conversation state. It now defaults to false. Send store: true to retrieve a response later or use it with previous_response_id; stored state expires after 30 days. This flag does not override free-Himalaya internal retention or organization bucket capture.