What is RAG (Retrieval-Augmented Generation)?

RAG is a technique that retrieves documents at query time and adds them to the prompt, so an LLM answers from data, not memory. Introduced by Lewis et al. in 2020, it is the open-book exam to a plain model’s closed-book one: instead of relying on what it memorized during training, the model consults material you supply and answers from it — grounded, current, private-aware, and, because the sources are known, citable. For a backend developer the reassuring truth under the hype is that RAG is not a framework you must adopt; it is a for-loop you can already build.

Key takeaways

The two phases

INGESTION (offline, once per document)
  load → chunk (split into ~hundreds of tokens, with overlap)
       → embed each chunk into a vector
       → store vector + text + ACL metadata in an index

QUERY (online, per question)
  embed the question → similarity search for top-k chunks (usually 5–10)
       → filter to what THIS user may read
       → augment the prompt with the chunks
       → LLM generates an answer grounded in them (+ citations)

The whole online phase, server-side, in one function:

// JavaScript — Cloud Code (cloud/main.js): RAG is a for-loop, not a framework
Parse.Cloud.define('askDocs', async (req) => {
  // 1 · embed the question (LLM API key stays server-side)
  const qVec = await embed(req.params.question);

  // 2 · retrieve — similarity search, ACL-FILTERED to this user's docs
  const chunks = await vectorSearch('DocChunk', qVec, {
    limit: 6,
    aclUser: req.user, // never retrieve what the asker can't read
  });

  // 3 · augment + generate
  const context = chunks.map((c) => c.get('text')).join('\n---\n');
  return await complete(`Answer using ONLY this context:\n${context}\n\nQ: ${req.params.question}`);
  // Answer cites the retrieved chunks — grounded, and permission-safe.
});

// Flutter / Dart — Back4app Flutter SDK
// The client just asks — retrieval, grounding, and ACLs happen server-side
final answer = await ParseCloudFunction('askDocs')
    .execute(parameters: {'question': 'What is our refund window?'});
print(answer.result); // grounded in the user's own documents, with citations
// No embeddings, no vector math, no LLM key on the device — RAG's backend
// is ordinary backend work: store docs, index vectors, query, call the model.

// iOS / Swift — Back4app Swift SDK
// The client just asks — retrieval, grounding, and ACLs happen server-side
let answer: String = try await Cloud.run(
    name: "askDocs",
    parameters: ["question": "What is our refund window?"])
print(answer) // grounded in the user's own documents, with citations
// No embeddings, no vector math, no LLM key on the device — RAG's backend
// is ordinary backend work: store docs, index vectors, query, call the model.

// Android / Kotlin — Back4app Android SDK
// The client just asks — retrieval, grounding, and ACLs happen server-side
val params = mapOf("question" to "What is our refund window?")
val answer = ParseCloud.callFunction<String>("askDocs", params)
println(answer) // grounded in the user's own documents, with citations
// No embeddings, no vector math, no LLM key on the device — RAG's backend
// is ordinary backend work: store docs, index vectors, query, call the model.

Why RAG: the three gaps

A plain LLM has three structural weaknesses, and RAG addresses all of them without retraining. Hallucination — models answer confidently whether or not they know; grounding every claim in retrieved text gives the model something true to say. The training cutoff — a model’s knowledge freezes at training time; retrieval fetches today’s data. Private data — the model never saw your documents; retrieval is how it reads them at query time. The bonus that closed-book models can’t offer: because the retrieved chunks are known, the answer can cite its sources like footnotes — the single biggest trust advantage RAG has.

RAG vs. fine-tuning

They are not rivals: fine-tuning teaches the model how to respond, RAG supplies what to respond about, and a specialized domain assistant frequently uses both — a fine-tuned voice answering from a retrieved knowledge base.

Is RAG dead? The long-context question, answered

The honest treatment the head-term pages avoid. Context windows now reach millions of tokens, prompting the recurring “just stuff everything in the prompt” argument. Three facts keep retrieval alive. Cost and latency: paying to process a giant context on every query is dramatically more expensive and slower than fetching the relevant slice — orders of magnitude, at scale. Context rot: studies through 2025 found model accuracy degrading well before the window fills — relevant facts buried in a huge context get missed, so a bigger window is not a reliably better answer. Freshness and access control: a static mega-prompt is stale the moment data changes and blind to who may read what. The 2026 consensus is not “RAG or long context” but both — retrieve a generous, relevant, permission-filtered subset, then reason over it with a long-context model. Pure long-context is fine only for small, stable, non-sensitive corpora.

Retrieval is where RAG fails

The framing that reorganizes how you debug a RAG system: garbage retrieved is confident garbage generated. Most failures blamed on the model are retrieval failures wearing a generation costume — the wrong chunks were fetched, so a perfectly faithful answer grounds itself in the wrong material. This makes two variables the ones that actually move quality. Chunking: pieces too large dilute relevance and burn prompt budget; too small lose the context that made them meaningful — structure-aware splitting (by heading, paragraph, code block) usually beats fixed sizes (the strategies matter). Hybrid retrieval: combine keyword search (exact terms, names, IDs) with vector search (meaning), then re-rank the merged candidates with a stronger relevance model before prompting — recall from both, precision from the re-ranker. And measure the two halves separately, in the RAGAS vocabulary: retrieval metrics (did we fetch the right chunks?) versus faithfulness (is every claim supported by what we fetched?) — because they fail independently and fix differently.

The security gap nobody mentions

Vector similarity ranks by meaning and knows nothing about permissions — which makes naive RAG a data-leak engine. Embed a company’s documents into one index without access metadata, and any user’s question can retrieve any document, because the finance report and the intern’s question are just nearby points in vector space. Post-retrieval filtering is not the fix either: drop forbidden chunks after the top-k selection and you both leak their existence and break the top-k contract (you asked for six and got two). The correct pattern: store ACL metadata with each chunk and constrain retrieval to the asking user’s permitted set before similarity ranking — permission-filtered retrieval, not permission-filtered display. The code tabs’ aclUser parameter is exactly this, and it is the difference between a demo and a system you can ship.

Common use cases

• Support and knowledge chat — answer from a company’s actual docs, with citations, current as of the last ingest.
• Search over private corpora — legal, medical, internal wikis: meaning-based retrieval the keyword box never gave you.
• Customer-scoped assistants — each user’s own data, ACL-filtered so retrieval never crosses a tenant boundary.
• Grounded analytics — questions answered from live records rather than a model’s stale guess.
• Documentation and onboarding — a model that quotes the manual instead of improvising it.

Do you even need RAG? A decision matrix

Limitations and trade-offs

• RAG reduces hallucination; it doesn’t remove it. Faithfulness must be measured, not assumed — the model can still misread good context.
• Quality lives in retrieval. Most of the engineering effort — chunking, hybrid search, re-ranking — is upstream of the model, where the wins are.
• Embeddings have a version. Change the embedding model and every stored vector must be regenerated; re-embedding a large corpus is a real migration.
• It adds moving parts. Ingestion pipelines, an index to maintain, extra per-query tokens and latency — real cost that a small stable corpus may not justify.
• Permissions don’t come for free. ACL-filtered retrieval is the load-bearing security work, and it’s the part demos skip and incidents rediscover.

RAG on Back4app

Back4app is an open-source Backend-as-a-Service (BaaS) platform that combines a managed database, auto-generated REST and GraphQL APIs, authentication, file storage, and Cloud Code serverless functions. The “RAG is a for-loop” claim is literal here: documents are ordinary objects with file storage for the originals; chunk vectors live as array fields beside them, indexed for similarity search; the ACLs that already govern every query become the permission filter on retrieval for free — the same rule that stops a user reading another’s records stops the retriever fetching them; and the orchestration — embed, retrieve, augment, generate — is a Cloud Code function calling the model’s API with the key held server-side, exactly as the code tabs show. No framework, no separate vector service to secure, no LLM key on the device — RAG stops being an AI project and becomes backend engineering you already know how to do.