Rubber Duck AI

Rubber Duck AI is a lightweight, iterative technique for diagnosing and refining machine learning models by systematically verbalizing their behavior. It matters when a model’s predictions seem plausible but inconsistent, when a team struggles to align on expectations, or when debugging requires cross-disciplinary collaboration. The practice begins with a single, simple prompt: “Explain this model’s behavior to me like I’m a rubber duck.” The engineer—or data scientist, product owner, or frontend dev—speaks aloud, step by step, through the model’s logic, often pausing to refine the description after each iteration. The goal is not to deliver a perfect explanation, but to uncover gaps, assumptions, and hidden failures.

Preparing the Model for the Duck

Before the duck listens, the model must be structured and documented in a way that enables clear, sequential explanation. A model is ready when it can be serialized with metadata and served via a standard API endpoint.

The model must expose a /predict endpoint that accepts a JSON payload and returns a structured response:

{
  "input": {
    "text": "The weather was cold and windy, so I wore my coat.",
    "context": "User: John, Age: 34, Location: Seattle",
    "features": {
      "length": 147,
      "readability": 58.2,
      "sentiment": 0.72
    }
  },
  "output": {
    "prediction": "User will wear coat",
    "confidence": 0.89,
    "explanation": [
      "The text mentions 'cold and windy' — strong indicators for coat-wearing.",
      "User is in Seattle, known for rainy, cool weather.",
      "Sentiment is positive, suggesting intentional layering."
    ],
    "metadata": {
      "model_version": "v2.1",
      "trained_on": "2023-09-15",
      "feature_engineering": [
        "TF-IDF on input text",
        "Word2Vec embeddings",
        "Sentiment polarity score"
      ],
      "inference_time_ms": 123
    }
  }
}

This structure enables the duck to follow the model’s logic. The engineer uses curl to probe the model:

curl -X POST https://ml-api.example.com/predict \
  -H "Content-Type: application/json" \
  -d '{
    "text": "The weather was cold and windy, so I wore my coat.",
    "context": "User: John, Age: 34, Location: Seattle"
  }' \
  -v

A successful response includes inference_time_ms and explanation, both of which are essential for the duck’s understanding. If the model returns only prediction without explanation, the duck asks, “What’s the reasoning behind this prediction?” — a signal to extend the model’s output.

Explaining the Model Step-by-Step

Once the model is ready, the engineer begins the explanation. The duck listens, often silent, occasionally interjecting: “What does ‘sentiment’ mean here?” or “Why did you pick this threshold?”

The explanation follows a natural flow: input → preprocessing → feature extraction → inference → output. Each phase becomes a sub-structure in the model’s description.

For instance:

> “The input is a text string and a user context. The text is passed through a tokenizer that splits it into words, normalizes capitalization, and removes punctuation. Then, we compute TF-IDF vectors for each word. We also generate Word2Vec embeddings from a pre-trained model. These two representations are concatenated into a single feature vector. The sentiment score is derived from a separate model trained on Reddit comments. We use the average of the sentiment values across all sentences.”

The engineer uses jq to extract and inspect parts of the model output:

curl -s https://ml-api.example.com/predict \
  -d '{"text": "It’s raining heavily. I forgot my umbrella.", "context": "User: Lisa, Location: Portland"}' \
  | jq '.output.explanation[]'

This yields:

"Text contains 'raining heavily' — strong signal for umbrella usage."
"User is in Portland, known for frequent rain."
"Sentiment is 0.65, indicating mild enthusiasm about the weather."

The engineer notes that the explanation is clear but misses a key detail: the model uses a threshold of 0.6 for sentiment classification. The duck asks: “How did you decide on 0.6?” The engineer checks the model’s configuration file:

model:
  name: "coat-wear-predictor"
  version: "v2.1"
  preprocessing:
    tokenizer:
      method: "word-level"
      lowercase: true
      remove_punct: true
    feature_extraction:
      - type: "tfidf"
        ngram_range: [1, 3]
        min_df: 2
      - type: "word2vec"
        embedding_dim: 100
        window_size: 5
  inference:
    model_type: "xgboost"
    threshold: 0.6
    features:
      - "tfidf"
      - "word2vec"
      - "sentiment"
    hyperparameters:
      max_depth: 8
      learning_rate: 0.1
      n_estimators: 100

This reveals that the threshold of 0.6 was chosen during cross-validation, but not documented in the model’s API response. The engineer adds threshold to the output schema.

Iterating the Explanation

The first explanation is rarely final. The duck’s questions lead to refinements in both the model and the explanation.

After a session, the engineer realizes that the model underperforms when users are in urban areas but not in coastal cities. The duck asks: “Why isn’t location type—urban vs. coastal—explicitly modeled?” The engineer traces the data pipeline:

psql -c "SELECT location, COUNT(*) FROM user_profiles WHERE location LIKE '%coast%'"

The query reveals that “Seattle” and “Portland” are labeled as coastal, but “Denver” and “Austin” are not. The model’s feature engineering script uses a hardcoded list:

COSTAL_CITIES = [
    "Seattle", "San Francisco", "San Diego", "New York", "Boston", "Portland"
]

The engineer updates the script to use a geolocation service:

def is_coastal(city: str) -> bool:
    response = requests.get(f"https://geolocation-api.example.com/city/{city}")
    data = response.json()
    return data["is_coastal"]

The model now dynamically classifies cities. The engineer tests the new logic and updates the model’s metadata field to include location_classification_method.

Common Failure Modes

The most subtle failure in Rubber Duck AI is silent misalignment between the model’s internal logic and the explanation. The model returns a prediction with a clear explanation, but the explanation does not match the actual steps in the code.

For example, the model claims:

> “We use sentiment to predict coat-wearing.”

But the code shows:

def predict_coat_wear(input_text: str, user_context: dict) -> dict:
    # ... other logic ...
    sentiment = calculate_sentiment(input_text)  # uses VADER
    # ... more logic ...
    return {
        "prediction": "User will wear coat",
        "explanation": [
            "Sentiment score is 0.75, indicating positive mood.",
            "Text mentions 'cold and windy' — strong indicators for coat-wearing."
        ]
    }

The explanation omits that the sentiment score is computed using VADER, not a pre-trained transformer. The duck asks: “How is VADER different from a transformer?” The engineer realizes the model’s feature extraction step is missing a key step: post-processing the sentiment score with a threshold.

The model computes sentiment but does not apply a threshold to convert it into a binary signal. The engineer adds this step:

sentiment_score = calculate_sentiment(input_text)
is_positive = sentiment_score > 0.6

The explanation now includes: “We classify sentiment as positive if the score exceeds 0.6.” The duck nods.

Another failure is overloading the explanation. The engineer speaks for ten minutes, but the duck cannot follow the flow. The root cause is unstructured output. The model returns a flat list of explanation points, but the engineer needs to group them into phases.

The solution: define a structured explanation format in the API response:

"explanation": {
  "phases": [
    {
      "name": "Preprocessing",
      "steps": [
        "Tokenize input text",
        "Lowercase and remove punctuation"
      ]
    },
    {
      "name": "Feature Extraction",
      "steps": [
        "Compute TF-IDF vectors",
        "Generate Word2Vec embeddings",
        "Calculate sentiment with VADER"
      ]
    },
    {
      "name": "Inference",
      "steps": [
        "Combine features into a single vector",
        "Feed vector into XGBoost model",
        "Apply threshold of 0.6 to final score"
      ]
    }
  ]
}

This structure enables the duck to ask: “Which phase contributes most to the final prediction?” The engineer can now point to the inference phase and highlight the role of the XGBoost model.

When the Model Fails

When the model returns a prediction that surprises the duck, the engineer investigates the input data and model internals.

The duck asks: “Why did the model predict ‘User will wear coat’ for a text that says ‘It’s hot and sunny, so I wore my hat’?”

The engineer checks the input:

{
  "text": "It’s hot and sunny, so I wore my hat.",
  "context": "User: Mike, Age: 27, Location: Phoenix"
}

The model returns:

{
  "prediction": "User will wear coat",
  "confidence": 0.74,
  "explanation": [
    "Text mentions 'hot and sunny' — strong indicators for hat-wearing.",
    "User is in Phoenix, known for hot weather.",
    "Sentiment is 0.58, indicating mild positivity."
  ]
}

The duck is puzzled: “Hot and sunny suggests a hat, not a coat. Why does the model predict coat?”

The engineer runs a diagnostic:

curl -s https://ml-api.example.com/diagnostics \
  -d '{"input": {"text": "It’s hot and sunny, so I wore my hat.", "context": "User: Mike, Age: 27, Location: Phoenix"}}'

The diagnostics endpoint returns:

{
  "input_features": {
    "text_length": 58,
    "sentiment": 0.58,
    "is_coastal": false,
    "city_temperature": 35.2,
    "weather_keywords": ["hot", "sunny", "hat"]
  },
  "model_predictions": {
    "feature_importance": {
      "sentiment": 0.32,
      "text_length": 0.18,
      "city_temperature": 0.29,
      "weather_keywords": 0.21
    },
    "threshold": 0.6,
    "final_score": 0.61
  }
}

The model’s final score is 0.61, just above the threshold. The key insight: the model learned that temperature is a stronger predictor than sentiment. Even though the text says “hot and sunny,” the model values city_temperature highly.

The duck asks: “Why is Phoenix’s temperature 35.2°C important?” The engineer realizes that the model was trained on data where temperature was recorded in Celsius, but the input is in Fahrenheit. The model expects Celsius, but the input provides Fahrenheit.

The engineer adds a preprocessing step:

def preprocess_input(text: str, context: dict) -> dict:
    # ... other steps ...
    city_temp_f = context.get("temperature_f")
    city_temp_c = (city_temp_f - 32) * 5 / 9
    return {
        "text": text,
        "city_temperature_c": city_temp_c,
        # ...
    }

After this fix, the model predicts “User will wear hat” for Phoenix, and the duck is satisfied.

Key Practices

Use curl and jq to probe the model interactively. The engineer should be able to run the same command in production and development environments.
Document the explanation in the API response. The duck cannot read code; it needs clear, structured text.
Version the model and include the version in the response. The duck asks: “How do you know this model is up to date?”
Use psql to explore the feature space. The engineer should be able to query the feature database directly.
Add a diagnostics endpoint that returns input features, model scores, and feature importances.
Always explain in phases. Use the structured phases object in the explanation to guide the duck.

Rubber Duck AI is not about delivering a perfect model. It is about discovering the model’s story — its assumptions, its decisions, its failures — through a conversation that reveals what the model truly knows.

This page was rewritten on 11 October 2026. It replaced a templated version that was not specific to its own title. How these pages are written and checked. If anything here is wrong, tell us at [email protected] and we will correct it.

Join the Newsletter

Get the latest updates and deep insights shipped directly to your inbox.