Basic RAG System Prompt

You are a helpful assistant that answers questions using only the provided context.

Rules:
1. Answer using ONLY the information in the context below.
2. If the context does not contain enough information to answer, say: "I don't have enough information in the provided context to answer this question."
3. Cite the source inline when referencing specific facts, like this: [Source: Document Name].
4. Never fabricate information not present in the context.
5. Keep answers concise and directly responsive to the question.

Context:
{{RETRIEVED_CONTEXT}}

Question: {{USER_QUERY}}

What this prompt does

Instructs the model to answer only from provided context, cite sources inline, and explicitly acknowledge when the answer is not in the retrieved documents.

When to use it

• When you have retrieved context from a vector store or search system
• When accuracy and source attribution matter more than creativity
• When you need to prevent the model from mixing retrieved facts with parametric knowledge

When NOT to use it

• For open-ended creative tasks where context grounding is not needed
• When you want the model to reason beyond the provided documents

The Prompt

Why it's structured this way

"Answer using ONLY" — The word "ONLY" in caps activates stronger constraint following than lowercase. Models attend to emphasis.

Rules numbered 1–5 — Numbered lists produce higher compliance rates than prose instructions. Each rule addresses one failure mode: mixing knowledge (rule 1), hallucination on gaps (rule 2), attribution (rule 3), fabrication (rule 4), verbosity (rule 5).

"I don't have enough information..." — Providing the exact fallback phrase prevents the model from improvising a weaker refusal that can still lead to hallucinated answers.

Context before Question — Models attend better to context placed before the task. Placing context last causes it to be treated as afterthought.

Variations