Prompt Engineering
Prompt engineering is the practice of designing and optimizing input prompts to elicit desired outputs from large language models. As LLMs become more capable, the quality of the prompt increasingly determines the quality of the result.
A well-crafted prompt can mean the difference between a vague, incorrect response and a precise, actionable one. This guide covers the fundamental techniques every engineer should know.
Modern LLM chat APIs use a structured message format with distinct roles. Understanding these roles is the foundation of effective prompt engineering.
| Role | Purpose | Persistence |
|---|---|---|
| system | Sets behavior, constraints, and context for the model | Persists for entire conversation |
| user | Contains the user's input or question | Per-turn |
| assistant | Contains the model's response (also used for few-shot examples) | Appended each turn |
| tool | Result from a function/tool call | Per-call |
| 1 | { |
| 2 | "model": "gpt-4o", |
| 3 | "messages": [ |
| 4 | { |
| 5 | "role": "system", |
| 6 | "content": "You are a senior software engineer. Provide concise, production-ready code with error handling. Always include type annotations." |
| 7 | }, |
| 8 | { |
| 9 | "role": "user", |
| 10 | "content": "Write a function that retries an async operation with exponential backoff." |
| 11 | }, |
| 12 | { |
| 13 | "role": "assistant", |
| 14 | "content": "Here is a TypeScript implementation with exponential backoff and jitter..." |
| 15 | } |
| 16 | ], |
| 17 | "temperature": 0.2 |
| 18 | } |
info
Research and practice have converged on several principles that consistently improve prompt quality. These apply across all model architectures and sizes.
1. Give Clear Instructions
Be explicit about what you want. Ambiguity is the most common source of poor outputs. Specify format, length, tone, and constraints.
| 1 | # Poor — ambiguous |
| 2 | Explain machine learning. |
| 3 | |
| 4 | # Good — specific |
| 5 | Explain the difference between supervised and unsupervised learning. Provide one real-world example of each. Keep your explanation under 150 words and suitable for a beginner with basic programming knowledge. |
2. Provide Examples (Few-Shot)
Examples anchor the model to the desired output pattern. Even a single well-chosen example dramatically improves consistency.
| 1 | Classify the sentiment of each tweet as Positive, Negative, or Neutral. |
| 2 | |
| 3 | Tweet: "Just got promoted! Best day ever." |
| 4 | Sentiment: Positive |
| 5 | |
| 6 | Tweet: "My flight got cancelled again." |
| 7 | Sentiment: Negative |
| 8 | |
| 9 | Tweet: "The meeting is at 3pm." |
| 10 | Sentiment: Neutral |
| 11 | |
| 12 | Tweet: "This new update broke everything." |
| 13 | Sentiment: |
3. Specify Output Format
Telling the model exactly how to format its output makes the result machine-parseable and reduces post-processing effort.
| 1 | Extract the following fields from the job posting and return them as JSON: |
| 2 | |
| 3 | { |
| 4 | "company_name": "string", |
| 5 | "role": "string", |
| 6 | "salary_range": "string | null", |
| 7 | "remote": "boolean", |
| 8 | "skills_required": ["string"], |
| 9 | "years_experience_min": "number | null" |
| 10 | } |
| 11 | |
| 12 | Job posting: |
| 13 | ``` |
| 14 | We are looking for a Senior Frontend Engineer with 5+ years of React experience to join our fully remote team at TechCorp. Salary range: $140k-$180k. Must have TypeScript, Next.js, and CSS expertise. |
| 15 | ``` |
4. Use Delimiters
Clearly separate instructions from input data using delimiters like triple backticks, XML tags, or markdown headers. This reduces prompt injection risk and clarifies boundaries.
| 1 | You are a document summarizer. Summarize the text between the <document> tags in 3 bullet points. |
| 2 | |
| 3 | <document> |
| 4 | ``` |
| 5 | Artificial intelligence has transformed industries ranging from healthcare to finance. Machine learning models can now diagnose diseases, predict market trends, and power autonomous vehicles. However, challenges remain in areas like interpretability, fairness, and robustness. |
| 6 | ``` |
| 7 | </document> |
| 8 | |
| 9 | Provide your summary in plain text bullet points prefixed with "-". |
best practice
Temperature and top-p are sampling parameters that control the randomness and diversity of model outputs. They are not just knobs for "creativity" — they are precision tools for controlling output characteristics.
| Parameter | Range | Effect | Use Case |
|---|---|---|---|
| temperature | 0.0 — 2.0 | Scales log probabilities before softmax. Low values = deterministic, high values = random | 0.0-0.3 for code/facts, 0.7-1.0 for creative |
| top_p | 0.0 — 1.0 | Nucleus sampling: only sample from tokens whose cumulative probability exceeds top_p | 0.1-0.3 for precision, 0.9-1.0 for diversity |
| frequency_penalty | -2.0 — 2.0 | Penalizes tokens based on existing frequency in the text | Positive values reduce repetition |
| presence_penalty | -2.0 — 2.0 | Penalizes tokens that have already appeared in the text | Positive values encourage topic diversity |
| 1 | import openai |
| 2 | |
| 3 | # Deterministic output — best for code, extraction, classification |
| 4 | response = openai.chat.completions.create( |
| 5 | model="gpt-4o", |
| 6 | messages=[ |
| 7 | {"role": "system", "content": "Classify the sentiment."}, |
| 8 | {"role": "user", "content": "This product is amazing!"} |
| 9 | ], |
| 10 | temperature=0.0, |
| 11 | top_p=0.1, |
| 12 | ) |
| 13 | |
| 14 | # Creative output — best for brainstorming, storytelling |
| 15 | response = openai.chat.completions.create( |
| 16 | model="gpt-4o", |
| 17 | messages=[ |
| 18 | {"role": "system", "content": "You are a creative copywriter."}, |
| 19 | {"role": "user", "content": "Write a tagline for a new AI-powered fitness app."} |
| 20 | ], |
| 21 | temperature=0.9, |
| 22 | top_p=0.95, |
| 23 | frequency_penalty=0.3, |
| 24 | ) |
warning
Effective prompts follow predictable structures. These patterns have been validated across thousands of production deployments.
Persona Pattern
Assign a role to the model to activate relevant knowledge and adjust tone.
| 1 | You are a senior systems architect with 15 years of experience at FAANG companies. You specialize in distributed systems, microservices, and cloud-native architecture. Review the following design proposal and identify: |
| 2 | 1. Scalability bottlenecks |
| 3 | 2. Single points of failure |
| 4 | 3. Security concerns |
| 5 | 4. Cost optimization opportunities |
| 6 | |
| 7 | Be specific and reference real-world patterns. |
Chain-of-Thought Pattern
Encourage step-by-step reasoning before giving the final answer.
| 1 | Solve this problem step by step: |
| 2 | |
| 3 | A store sells apples at $0.50 each and oranges at $0.75 each. |
| 4 | A customer buys 4 apples and 3 oranges. They pay with a $10 bill. |
| 5 | How much change should they receive? |
| 6 | |
| 7 | Let's work through this: |
| 8 | 1. Cost of 4 apples: 4 × $0.50 = $2.00 |
| 9 | 2. Cost of 3 oranges: 3 × $0.75 = $2.25 |
| 10 | 3. Total cost: $2.00 + $2.25 = $4.25 |
| 11 | 4. Change from $10: $10.00 - $4.25 = $5.75 |
| 12 | |
| 13 | Therefore, the customer should receive $5.75 in change. |
pro tip
✗ Overloading the System Prompt
You are a helpful assistant. Be concise. Be detailed. Use JSON. Also use markdown. Always cite sources. Never cite sources.Contradictory instructions confuse the model. Keep system prompts focused and non-contradictory.
✗ Assuming the Model Knows Context
Summarize the key findings from the report I was working on.The model does not have persistent memory of prior sessions. Always include relevant context in each interaction.
✗ Not Specifying Output Format
Give me a list of best practices.Without format constraints, the model may return bullet points, numbered lists, paragraphs, or tables inconsistently.
✗ Using Temperature > 0 for Deterministic Tasks
temperature: 0.7 // for a classification taskClassification, extraction, and code generation should use temperature=0 to ensure consistent outputs.
Prompts are not write-once artifacts. They require systematic evaluation and iteration. Treat prompt engineering as an empirical science, not an art.
| 1 | import json |
| 2 | from openai import OpenAI |
| 3 | |
| 4 | client = OpenAI() |
| 5 | |
| 6 | def evaluate_prompt(prompt_template, test_cases): |
| 7 | """Evaluate a prompt template against a set of test cases.""" |
| 8 | results = [] |
| 9 | for case in test_cases: |
| 10 | response = client.chat.completions.create( |
| 11 | model="gpt-4o", |
| 12 | messages=[ |
| 13 | {"role": "system", "content": prompt_template}, |
| 14 | {"role": "user", "content": case["input"]} |
| 15 | ], |
| 16 | temperature=0, |
| 17 | max_tokens=500, |
| 18 | ) |
| 19 | output = response.choices[0].message.content |
| 20 | passed = case["expected"] in output |
| 21 | results.append({ |
| 22 | "input": case["input"], |
| 23 | "output": output, |
| 24 | "expected": case["expected"], |
| 25 | "passed": passed, |
| 26 | "tokens": response.usage.total_tokens, |
| 27 | }) |
| 28 | accuracy = sum(r["passed"] for r in results) / len(results) |
| 29 | return {"accuracy": accuracy, "results": results} |
| 30 | |
| 31 | # Usage |
| 32 | test_cases = [ |
| 33 | {"input": "I love this!", "expected": "Positive"}, |
| 34 | {"input": "This is terrible.", "expected": "Negative"}, |
| 35 | ] |
| 36 | template = "Classify sentiment as Positive, Negative, or Neutral." |
| 37 | report = evaluate_prompt(template, test_cases) |
| 38 | print(f"Accuracy: {report['accuracy']:.0%}") |
| 39 | json.dump(report["results"], open("eval_results.json", "w"), indent=2) |
best practice