What is an LLM API?

An LLM API is an HTTP endpoint to a hosted language model: send a prompt, get generated text back, billed per token. It is an ordinary API with three unusual properties — it’s stateless (you resend the whole conversation each call), metered by tokens (input and output priced separately), and non-deterministic (the same prompt can return different text). Master those three and the rest is the operational discipline the ranking pages skip: where the call belongs, what it costs, and how it fails.

Key takeaways

A real call, server-side

// JavaScript — Cloud Code (cloud/main.js)
// The LLM call belongs server-side — the key never reaches the client
Parse.Cloud.define('summarize', async (req) => {
  const res = await Parse.Cloud.httpRequest({
    method: 'POST',
    url: 'https://api.llm-provider.example/v1/chat/completions',
    headers: {
      Authorization: `Bearer ${process.env.LLM_KEY}`, // server-side secret
      'Content-Type': 'application/json',
    },
    body: {
      model: 'default-chat',
      messages: [
        { role: 'system', content: 'Summarize in one sentence.' },
        { role: 'user', content: req.params.text },
      ],
      max_tokens: 80, // cost + latency guardrail, enforced by YOU
    },
  });
  return res.data.choices[0].message.content;
});

// Flutter / Dart — Back4app Flutter SDK
// The client calls YOUR function, never the LLM API directly
final summary = await ParseCloudFunction('summarize')
    .execute(parameters: {'text': longArticle});
print(summary.result);
// The model key stays on the server. If this app called the LLM API
// itself, the key would ship in the binary — one decompile from theft.

// iOS / Swift — Back4app Swift SDK
// The client calls YOUR function, never the LLM API directly
let summary: String = try await Cloud.run(
    name: "summarize", parameters: ["text": longArticle])
print(summary)
// The model key stays on the server. If this app called the LLM API
// itself, the key would ship in the IPA — one decompile from theft.

// Android / Kotlin — Back4app Android SDK
// The client calls YOUR function, never the LLM API directly
val summary = ParseCloud.callFunction<String>(
    "summarize", mapOf("text" to longArticle))
println(summary)
// The model key stays on the server. If this app called the LLM API
// itself, the key would ship in the APK — one decompile from theft.

The shape is portable — most providers accept the same chat-completions format, so the request and response below read the same whichever model you point at:

POST /v1/chat/completions            Authorization: Bearer <key>
{ "model": "default-chat",
  "messages": [
    { "role": "system", "content": "You are concise." },   ← sets behavior
    { "role": "user",   "content": "Explain tokens." } ],   ← the instruction
  "max_tokens": 200, "temperature": 0.7, "stream": false }

→ { "choices": [ { "message": { "role": "assistant",
                                "content": "A token is…" },
                   "finish_reason": "stop" } ],
    "usage": { "prompt_tokens": 24, "completion_tokens": 118,
               "total_tokens": 142 } }        ← what you're billed on

Tokens are the currency

A token is a chunk of text — roughly four characters, or three-quarters of an English word. Everything about LLM API cost reduces to counting them, and two facts surprise people. Output costs more than input — often several times more — because reading your prompt is cheap while generating each reply token is compute-intensive, one prediction at a time across the whole vocabulary. And context is paid every call: the model is stateless, so “memory” is you resending prior turns, and a long history or a big retrieved passage is billed again on every request. The rough monthly formula worth internalizing:

monthly cost ≈ ( requests/day
                 × (avg_input_tokens  × input_price_per_M  / 1_000_000
                  + avg_output_tokens × output_price_per_M / 1_000_000) )
               × 30

The levers that move it: cap max_tokens, trim the context you resend,
pick a smaller model for easy tasks, and cache reused prompt prefixes.

The parameters that matter

Streaming: why chat UIs feel fast

Set a stream flag and the reply arrives incrementally as Server-Sent Events — data: lines, one token-chunk at a time, ending in a [DONE] marker — instead of one blob after several seconds. The reason is perceptual: time-to-first-token is a fraction of time-to-full-answer, so the user watches words appear rather than a spinner spin. It’s one-directional server-to-client text over plain HTTP, which is exactly SSE’s shape and precisely why LLM streaming uses it rather than WebSockets. The backend consequence: your proxy must stream through — reading the provider’s event stream and forwarding it — rather than buffering the whole reply and defeating the point.

Tool calling and structured output

Two features turn “text in, text out” into something programmable. Tool calling: you describe available functions (name + JSON schema) in the request, and the model — instead of prose — returns a structured JSON call naming a tool and its arguments; your code executes it and feeds the result back (Fowler’s walkthrough is the clear reference). The model never runs the function; it only asks. Structured output, distinct from tool calling and often confused with it, is three things worth separating:

Tool calling is the mechanism that makes an AI agent possible — the same request/response, run in a loop until the model stops asking for tools.

LLM API vs. self-hosting an open model

The honest read: an API wins for almost everyone almost always — zero infrastructure, instant start, and cheaper until volume is genuinely large and steady. Self-hosting an open model (with Ollama, vLLM, or llama.cpp) earns its operational cost only past a high break-even, or when data residency is a hard requirement. Most products start on the API and revisit the question if scale ever forces it.

Reliability: LLM endpoints are flaky

Treat the LLM API as a slow, rate-limited, occasionally-failing remote dependency, because it is. Limits arrive as requests-per-minute and tokens-per-minute; exceed either and you get a 429. The discipline is the same one every rate-limited API demands: on a 429 or a 5xx, retry with exponential backoff plus jitter and honor Retry-After; on a 4xx — bad auth, malformed request, content-policy refusal — don’t retry, because it will fail identically. Add per-request timeouts (generation can stall), and remember that a retried non-idempotent call costs tokens twice. None of this is LLM-specific; all of it is skipped by the tutorials that stop at the happy-path curl.

The rule the tutorials bury: never call it from the client

The single most important operational fact, and the one glossary pages omit: the LLM API call belongs on your server, never in the browser or app. A model API key shipped to the client is one network-tab glance or one binary decompile from theft — and unlike a leaked publishable key, a stolen model key is a stranger with your token budget and no rate limit but your bill. The pattern is the same proxy discipline every secret-keyed service needs: the client calls your endpoint, your backend holds the key in server-side configuration and calls the model, and the reply comes back through you. That proxy is also where every other control in this article lives — cost caps, retries, streaming, prompt trimming — which is why “where does the call go?” has one answer.

Common use cases

• Summarization and extraction — turn long text into short structured data, temperature: 0.
• Chat and assistants — streamed replies over a resent conversation history.
• Classification and tagging — Structured Outputs enforcing your label schema.
• RAG answers — generation grounded in retrieved context, cost-managed by trimming that context.
• Agentic actions — tool calling in a loop, each tool a permissioned backend function.

Should you use an LLM API? A decision matrix

Limitations and trade-offs

• Non-determinism is the default. The same prompt varies run to run; anything requiring exact repeatability needs temperature: 0 and, often, validation on the output.
• Cost scales with tokens, silently. A generous context or an unbounded max_tokens turns a cheap feature expensive at volume — meter usage, don’t assume it.
• Latency is seconds, not milliseconds. LLM calls are slow by web standards; design for it with streaming, async patterns, and honest loading states.
• The dependency is external and rate-limited. Provider outages and throttling are your outages; retries, fallbacks, and caching are resilience, not polish.
• Outputs can be wrong and confident. The API returns fluent text regardless of truth; grounding (RAG) and validation are how you trust it.

LLM APIs on Back4app

Back4app is an open-source Backend-as-a-Service (BaaS) platform that combines a managed database, auto-generated REST and GraphQL APIs, authentication, file storage, and Cloud Code serverless functions. The “where does the call belong” question answers itself here: inside a Cloud Function, exactly as the code tabs show — the client calls your summarize, the function holds the model key in server-side config and calls the LLM API, and the key never touches the device. That one placement solves the whole gap list at once: cost caps (the function sets max_tokens and picks the model), retries and backoff (in the function, against the flaky endpoint), and delivery two ways — return the reply synchronously for a single answer, or write it to the database and let clients watch it fill in over Live Queries, which is streaming without a socket you had to build. The LLM API stops being an integration risk and becomes one more server-side call your backend already knows how to make safely.