Prompt Chaining for Complex Tasks

Prompt chaining for complex tasks

Prompt chaining is the practice of sequencing two or more LLM calls so that the output of one becomes a material input for the next. It matters when a single prompt exceeds token limits, when a problem naturally decomposes into sub‑steps, when you need to validate or transform intermediate results, or when you’re building workflows that require conditional branching based on prior LLM output. This page covers the pattern, concrete invocation methods, state‑management techniques, and the failure modes you’ll encounter once chains grow beyond two steps.

Breaking a monolithic prompt into a chain

When a user request is too large for a single call, the first step is to decompose it. Instead of embedding all constraints, contexts, and desired outputs in one prompt, you ask the model to produce a structured intermediate representation—typically a list of steps, a schema, or a concise summary—and then feed each piece into subsequent calls.

The mechanics are straightforward in a shell environment. The following script demonstrates a two‑step chain using only curl and jq. The endpoint URL and auth token are sourced from environment variables so the script itself carries no vendor‑specific values; you only need to set LLM_API_URL and LLM_API_TOKEN to match the service you’re using.

#!/usr/bin/env bash
set -euo pipefail

API="${LLM_API_URL}"
TOKEN="${LLM_API_TOKEN}"


DECOMPOSED=$(curl -sS -X POST "$API" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"prompt": "Decompose the following user request into a JSON array of atomic steps. Request: {{USER_INPUT}}"}' \
  | jq -r '.steps[]')

# Step 2: iterate over the produced steps and ask the model to address each one
while IFS= read -r step; do
  RESULT=$(curl -sS -X POST "$API" \
    -H "Authorization: Bearer $TOKEN" \
    -H "Content-Type: application/json" \
    -d "{\"prompt\": \"$step\"}" \
    | jq -r '.response')
  echo ">>> $RESULT"
done <<< "$DECOMPOSED"

Key flags and their real meanings:

-sS – silent but show error messages
-X POST – HTTP method
-H "Authorization: Bearer $TOKEN" – bearer token header (the flag itself is standard curl; the variable name is placeholder)
-H "Content-Type: application/json" – content type header
-d '{"prompt": …}' – JSON data payload

The jq -r '.steps[]' and jq -r '.response' extracts array elements and string values without surrounding quotes. If the API’s response shape differs, the .steps and .response keys must be adjusted to match the actual field names; the curl command and its flags remain unchanged.

Managing state across chained calls

State in a prompt chain is rarely a persisted database record—it’s usually the textual output of the previous step, possibly transformed via jq, sed, or other Unix tools before being injected as the next prompt’s content. The most common pattern is capture‑transform‑feed: capture the raw output, normalize it (strip trailing whitespace, ensure valid JSON, map free‑form text to a schema), then feed the normalized string into the next curl invocation.

A frequent silent failure occurs when intermediate output contains characters that break the next curl -d block: unescaped double quotes, literal newlines, or control characters can cause the JSON payload to be malformed, resulting in an HTTP 400 response from the API. A robust pattern pipes the raw output through jq first, which both validates and cleans the JSON:

RAW=$(curl -sS -X POST "$API" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"prompt": "Generate a JSON object with keys $keys."}' )

NORMALIZED=$(echo "$RAW" | jq -c .)

RESULT=$(curl -sS -X POST "$API" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d "{\"prompt\": $NORMALIZED}")

If the model returns something that isn’t valid JSON, jq -c . will exit with an error and print jq: error: < to stderr. Catching that early—via set -e or an explicit || exit guard—prevents the malformed string from propagating down the chain.

Error paths and recovery in chains

Chains amplify failures. A non‑recoverable error at step 1 becomes the input to step 2, step 3, and so on, often producing a cascade of “I don’t understand” or empty responses that are hard to diagnose after the fact.

Two practical guardrails help contain the damage:

Explicit error checking after each call. After every curl | jq pipeline, inspect the exit code and the parsed output before passing it forward.

   DECOMPOSED=$(curl -sS -X POST "$API" \
     -H "Authorization: Bearer $TOKEN" \
     -H "Content-Type: application/json" \
     -d '{"prompt": "Decompose the request"}' \
     | jq -r '.steps[]') || { echo "Decomposition failed"; exit 1; }

Bounded retry with a maximum step count. If a step returns an empty string or a sentinel value like "ERROR", the chain can either halt or fall back to a simpler retry of that single step, rather than blindly forwarding the bad input.

   if [[ -z "$RESULT" || "$RESULT" == "ERROR" ]]; then
     echo "Step failed, retrying once…
     RESULT=$(curl -sS -X POST "$API" \
       -H "Authorization: Bearer $TOKEN" \
       -H "Content-Type: application/json" \
       -d "{\"prompt\": \"$step\"}" \
       | jq -r '.response')
   fi

When a chain breaks after step 3 of 5 in production, the typical discovery path is: inspect the raw curl response body before jq, check whether the intermediate string was truncated (LLM output silently cut off at the token boundary), and verify that no unexpected Unicode or control characters corrupted the JSON payload.

Sequential vs parallel chaining strategies

Not every complex task requires a strict line. Some workflows can split into independent branches that converge later. The choice between sequential and parallel chaining affects both latency and error surface area.

Sequential chaining is appropriate when step N depends on the exact output of step N‑1. The entire chain stalls if any single step fails or returns unexpected format. It’s the safest pattern for correctness but the most fragile in the face of network or model hiccups.

Parallel chaining involves issuing multiple independent prompts simultaneously, each starting from the same base context but diverging on different sub‑tasks. Results are collected after all prompts complete. This pattern reduces total wall‑clock time when sub‑tasks are truly independent, but it doubles (or triples) the number of API calls and the associated cost. It also requires a merge step—often a final LLM call or a scripted concatenation—that must handle conflicting or overlapping outputs.

A hybrid approach works well for many engineering tasks: decompose the problem into a small sequential chain for validation and preconditioning, then fan out parallel calls for independent sub‑problems, and finally converge the results with one last synthesis step.

Debugging and introspecting intermediate outputs

When a chain misbehaves, the most effective debugging technique is to “freeze” the output of each step and inspect it outside the pipeline. Replace the curl | jq chain with a temporary file write, then examine the file.

curl -sS -X POST "$API" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"prompt": "Decompose the request"}' \
  > /tmp/step1_raw.json

jq -r '.steps[]' /tmp/step1_raw.json > /tmp/step1_steps.txt

You can then cat /tmp/step1_steps.txt, run jq filters manually, or pipe the content into the next curl invocation with a different prompt to see how the model reacts. This method sidesteps the “output of one step becomes input of the next” coupling long enough to isolate whether the problem lies in the model’s response shape, the parsing logic, or the prompt text itself.

A less obvious but equally common issue: the model’s output format drifts subtly across calls (e.g., sometimes it wraps the JSON in a markdown code block, sometimes it doesn’t). A robust parser normalizes the raw curl output before jq processing:

CLEANED=$(curl -sS ... | sed 's/^```json//; s/```$//')
echo "$CLEANED" | jq -r '.steps[]'

Forgetting this normalization step is a frequent silent failure mode: the chain appears to work for several iterations, then suddenly jq reports null or parse error because the model started wrapping its output in fenced code blocks.

Sharp edges you’ll only learn after it breaks in production

Token‑budget accumulation: Each step adds its own prompt + completion to the context window. A five‑step chain of 512‑token completions quickly exhausts a 4 k‑token model, causing the last steps to receive truncated or empty context. There’s no soft warning; the API simply returns the last valid completion or an error code.

Character‑level breakage in curl -d: Shell variable expansion does not perform JSON escaping. A step that outputs a string containing unescaped double quotes will produce a malformed JSON payload, resulting in an HTTP 400 response. Always pipe through jq or a dedicated JSON‑escaper before feeding text into -d.

No built‑in rollback: Once step 1’s output is committed as step 2’s input, there’s no undo. If step 2 produces an incorrect result, you must re‑run the entire chain from the beginning. Design your workflow so that individual steps are idempotent or so that the chain can be restarted at any step without corrupting downstream state.

Prompt injection via intermediate output: If any step’s output is not properly sanitized, a malicious or inadvertently crafted LLM response can inject additional instructions into the next prompt. Treat every intermediate string as untrusted and validate its structure (e.g., with jq) before injection.

Latency amplification: Each additional call adds network round‑trip time and model processing time. A chain of four steps can take 2–3× longer than a single optimised prompt. Profile the per‑step latency in your environment and consider whether a single, well‑engineered prompt with few‑shot examples might meet the same correctness target with lower overhead.

This page was rewritten on 14 October 2026. It replaced a templated version that was not specific to its own title. How these pages are written and checked. If anything here is wrong, tell us at [email protected] and we will correct it.

Join the Newsletter

Get the latest updates and deep insights shipped directly to your inbox.