Chain-of-Thought Prompting is a technique for guiding large language models to solve complex reasoning tasks by explicitly prompting them to articulate intermediate steps. It matters when the model must infer or synthesize answers that require multiple logical or mathematical operations, such as solving a word problem, analyzing a dataset, or debugging a code snippet. Instead of a direct input-output prompt, the model is asked to "think aloud" — to generate a sequence of reasoning steps before arriving at a final answer. This approach reduces errors, improves interpretability, and increases performance on tasks requiring multi-step inference.

Generating a Chain of Thought

To generate a chain of thought, the prompt begins with a clear instruction to "think step by step" or "reason through the problem." The input includes the problem statement, and the model is expected to produce a sequence of intermediate conclusions, each on a new line or prefixed with a step number. The output is a narrative of reasoning that can be post-processed or used directly as a trace.

The most common form is a prompt template:

Solve the following problem step by step. Show your work.

Problem: A train leaves station A at 8:00 AM traveling at 60 km/h. Another train leaves station B, 300 km away, at 9:00 AM traveling at 90 km/h toward station A. At what time do they meet?

Step 1: Identify the known quantities and variables.
Step 2: Calculate the distance covered by the first train before the second train starts.
Step 3: Determine the remaining distance between the two trains when the second train starts.
Step 4: Compute the relative speed of the two trains.
Step 5: Find the time it takes for the trains to meet after the second train starts.
Step 6: Calculate the total time from 8:00 AM until they meet.
Step 7: Determine the meeting time.

The model responds with:

Step 1: The known quantities are: train A starts at 8:00 AM at 60 km/h; train B starts at 9:00 AM at 90 km/h; distance between stations is 300 km.

Step 2: In one hour (from 8:00 to 9:00), train A travels 60 km.

Step 3: After 9:00 AM, the distance between the two trains is 300 - 60 = 240 km.

Step 4: The relative speed is 60 + 90 = 150 km/h.

Step 5: Time to meet after 9:00 AM is 240 / 150 = 1.6 hours = 1 hour 36 minutes.

Step 6: Total time from 8:00 AM is 1 hour (before 9:00) + 1 hour 36 minutes = 2 hours 36 minutes.

Step 7: They meet at 8:00 AM + 2 hours 36 minutes = 10:36 AM.

This output is both the final answer and a full reasoning trace.

Structuring the Prompt with Explicit Step Instructions

When the task involves multiple phases, the prompt can guide the model using numbered steps, each with a specific instruction. This is particularly effective for tasks that require both analysis and synthesis.

For example, to analyze a SQL query for performance:

Analyze the following SQL query step by step. For each step, explain your reasoning.

1. Identify the main query and the subqueries.
2. Determine the order of execution: which parts are evaluated first?
3. Evaluate the cost of each table scan or index lookup.
4. Propose an optimization based on the current execution plan.
5. Rewrite the query using the proposed optimization.

Query:
SELECT u.name, COUNT(p.id) AS post_count
FROM users u
JOIN posts p ON u.id = p.user_id
WHERE p.created_at >= '2023-01-01'
  AND u.active = TRUE
GROUP BY u.id, u.name
HAVING COUNT(p.id) > 5
ORDER BY post_count DESC
LIMIT 10;

The model then produces a structured response that can be parsed into a JSON document or used as a report.

Using Chain-of-Thought in API Workflows

In a REST API workflow, the chain-of-thought prompt is sent as the prompt field in a request body to an LLM inference endpoint. The most common interface is a POST request to /v1/completions with JSON input.

{
  "model": "gpt-3.5-turbo",
  "prompt": "Solve the following problem step by step. Show your work.\n\nProblem: A company has 120 employees. 60% are in engineering, 30% in sales, and 10% in marketing. If 40% of the engineering team are women, and the average salary of engineering employees is $90,000, what is the total payroll for engineering employees?\n\nStep 1: Calculate the number of engineering employees.\nStep 2: Calculate the number of women in the engineering team.\nStep 3: Compute the total payroll for engineering employees.\nStep 4: Present the final answer.",
  "max_tokens": 512,
  "temperature": 0.4,
  "top_p": 0.9
}

The response contains the full chain of thought, which can be extracted using a simple regex or parsed via a JSON transformer.

Extracting Intermediate Steps Programmatically

The chain-of-thought output is not just a narrative but a structured data source. To extract the steps programmatically, a pipeline can use regular expressions or a simple parser.

For example, in Python:

import re

def extract_steps(text):
    pattern = r"Step\s+\d+[:.]\s*(.+?)(?=\n\nStep\s+\d+|$)"
    steps = re.findall(pattern, text, re.DOTALL)
    return [{"step": i+1, "content": step.strip()} for i, step in enumerate(steps)]


response = """
Step 1: The total number of employees is 120. 60% are in engineering, so 120 × 0.6 = 72 engineering employees.
Step 2: 40% of the 72 engineering employees are women, so 72 × 0.4 = 28.8 ≈ 29 women.
Step 3: The total payroll is 72 employees × $90,000 = $6,480,000.
Step 4: The total payroll for engineering employees is $6,480,000.
"""

steps = extract_steps(response)

This yields a list of step objects that can be stored in a database or used to generate reports.

Handling Step Merging and Fragmentation

A common failure mode is step fragmentation: the model generates too many small steps, or merges multiple ideas into a single step. This happens when the prompt lacks clear boundaries or when the temperature is too high.

For example, with temperature=1.0, the model may combine Step 1 and Step 2 into one long paragraph:

Step 1: The total number of employees is 120. 60% are in engineering, so 120 × 0.6 = 72 engineering employees. 40% of the engineering employees are women, so 72 × 0.4 = 28.8 ≈ 29 women.

To avoid this, the prompt should explicitly separate steps:

Step 1: Calculate the number of engineering employees.
Step 2: Calculate the number of women in the engineering team.
Step 3: Compute the total payroll for engineering employees.
Step 4: Present the final answer.

Using a consistent delimiter — such as Step X: followed by a colon and a new line — ensures that step boundaries are robust.

Step Counting and Alignment

Another subtle failure is mismatched step numbering. The model may write "Step 1" but then use "1." or "1)" or "1: " inconsistently. This leads to parsing errors when extracting steps.

To enforce consistency, the prompt should specify the exact format:

Use the following format:
Step 1: [content]
Step 2: [content]
...
Step N: [content]

Additionally, the model should be trained or prompted to use the exact capitalization and punctuation.

Handling Missing or Misordered Steps

When a step is missing or out of order, the reasoning becomes less reliable. A model might skip a key calculation or place a conclusion before its supporting evidence.

For example, a model might compute the total payroll before calculating the number of engineering employees:

Step 1: The total payroll is 72 employees × $90,000 = $6,480,000.
Step 2: 60% of 120 employees are in engineering, so 72 employees.
Step 3: 40% of 72 are women, so 29 women.

This is logically valid but suboptimal. The model should first establish the number of employees before computing the payroll.

To fix this, the prompt can include a constraint:

Ensure each step builds on the previous one. Do not make assumptions without stating them.

Alternatively, use a step dependency pattern:

Step 1: [required for Step 2]
Step 2: [depends on Step 1]
Step 3: [depends on Step 2 and Step 1]

This encourages the model to structure its reasoning hierarchically.

Combining Chain-of-Thought with Few-Shot Examples

Chain-of-thought prompting becomes even more powerful when combined with few-shot examples. Each example includes a problem, a full chain of thought, and the final answer.

Example:

Problem: A car travels 240 km in 4 hours. What is its average speed?

Step 1: The formula for average speed is total distance divided by total time.
Step 2: The total distance is 240 km.
Step 3: The total time is 4 hours.
Step 4: Average speed = 240 km / 4 h = 60 km/h.
Step 5: The average speed is 60 km/h.

Problem: A rectangle has a length of 15 cm and width of 8 cm. What is its area and perimeter?

Step 1: The area of a rectangle is length × width.
Step 2: Area = 15 cm × 8 cm = 120 cm².
Step 3: The perimeter is 2 × (length + width).
Step 4: Perimeter = 2 × (15 + 8) = 2 × 23 = 46 cm.
Step 5: The area is 120 cm² and the perimeter is 46 cm.

This format primes the model to produce structured, detailed reasoning for new inputs.

Handling Errors in the Chain

The model may produce a chain of thought with internal inconsistencies or calculation errors.

For example:

Step 1: The total number of employees is 120. 60% are in engineering, so 120 × 0.6 = 72 engineering employees.
Step 2: 40% of the engineering employees are women, so 72 × 0.4 = 28.8 ≈ 29 women.
Step 3: The total payroll is 72 employees × $90,000 = $6,480,000.
Step 4: The average salary is $90,000, so the total payroll is 72 × $90,000 = $6,480,000.

The model repeats the calculation in Step 4 but does not verify it. A better chain would include a verification step:

Step 5: Verify: 72 × 90,000 = (70 × 90,000) + (2 × 90,000) = 6,300,000 + 180,000 = 6,480,000.

This indicates that the model can be prompted to include validation steps.

To detect such errors, a post-processing step can run a validation query over the steps:

WITH steps AS (
  SELECT
    step_number,
    content,
    -- Extract numbers and formulas
    REGEXP_EXTRACT(content, '(\d+\.?\d*)') AS value,
    REGEXP_EXTRACT(content, '(\d+)[\s]+[×××]') AS operation
  FROM chain_of_thought
)
SELECT
  step_number,
  content,
  CASE
    WHEN value IS NOT NULL AND operation = '×' THEN 'Calculation'
    WHEN value IS NOT NULL AND operation LIKE '%×%' THEN 'Multiplication'
    ELSE 'General'
  END AS step_type
FROM steps;

This identifies steps that are likely calculations, allowing for automated validation.

When Chain-of-Thought Fails

Chain-of-thought prompting silently fails when the model:

  • Produces steps that are too shallow or too deep.
  • Repeats the same idea across steps.
  • Uses inconsistent terminology (e.g., "total payroll" vs. "overall salary").
  • Introduces a new variable not defined in the problem.
  • Omits a key step without justification.

These failures are often detected only under load or in production. To catch them, a pipeline should validate the output against a known ground truth, using exact match or fuzzy matching on step content.

A common cause of failure is the step collapse: the model generates only one or two steps for a complex problem, despite being prompted for five or more. This occurs when the prompt is too vague or when the temperature is too low.

To mitigate, use a step verification prompt:

Review the following chain of thought. Identify any missing steps, repeated steps, or errors in reasoning.

[Chain of thought from model]

This adds a lightweight review phase to the pipeline.

Final Answer Extraction

The final answer is not always the last step. Sometimes it is embedded in the reasoning. To extract it reliably, use a dedicated prompt:

From the following chain of thought, extract the final answer in a single sentence.

[Chain of thought]

The model then produces a concise summary.

Alternatively, use a simple rule: the final answer is the last sentence of the chain, or the last step that begins with "Final answer:" or "Answer:".

Conclusion

Chain-of-thought prompting is not merely a prompt format but a reasoning architecture. It transforms a black-box LLM into a transparent, multi-step inferencer. It succeeds when the input problem is complex, the steps are well-structured, and the output is validated. It fails silently when step boundaries are inconsistent, calculations are unverified, or the final answer is buried. To use it effectively, engineers must define the step format, enforce step numbering, and validate the output pipeline.

This page was rewritten on 11 October 2026. It replaced a templated version that was not specific to its own title. How these pages are written and checked. If anything here is wrong, tell us at [email protected] and we will correct it.