How LLMs Reason and Generate Text

Introduction

Large Language Models do not “think” the way humans do — they are next-token predictors trained to produce plausible continuations of text sequences. Yet recent techniques, particularly chain-of-thought prompting, have dramatically improved their ability to solve multi-step problems. Understanding the mechanics of how LLMs generate text — and why they fail at certain tasks — is essential for building reliable AI applications.

Step-by-Step: From Prompt to Response

flowchart TD
    A["1. Auto-Regressive Token Generation"]
    B["2. Chain-of-Thought Prompting (Wei et al., 2022)"]
    C["3. Self-Consistency Sampling"]
    D["4. ReAct Pattern (Reasoning + Acting)"]
    E["5. Known Reasoning Failures"]
    A --> B
    B --> C
    C --> D
    D --> E

Step 1: Auto-Regressive Token Generation

LLMs generate text one token at a time. Each token is selected by computing a probability distribution over the entire vocabulary (~100,000 tokens) conditioned on all prior tokens:

P(token_n | token_1, token_2, ..., token_{n-1})

The model doesn’t plan the full response before generating — it commits to each token sequentially. This is why sampling parameters matter:

• temperature=0: Always pick the highest-probability token (greedy decoding) — deterministic but sometimes repetitive
• temperature=0.7: Sample from the distribution — more creative, varied
• top_p=0.95 (nucleus sampling): Sample only from tokens comprising the top 95% of probability mass

Step 2: Chain-of-Thought Prompting (Wei et al., 2022)

Standard prompting asks models to jump to the answer. Chain-of-thought asks them to show their work, dramatically improving accuracy on multi-step reasoning:

Standard prompt:
Q: A store sells apples for $0.80 each. Maria buys 12 apples and pays with a $20 bill. How much change does she get?
A: $10.40  ← Frequently wrong on standard prompting

Chain-of-thought prompt:
Q: A store sells apples for $0.80 each. Maria buys 12 apples and pays with a $20 bill. How much change does she get? Let's think step by step.
A: 
Step 1: Cost of 12 apples = 12 × $0.80 = $9.60
Step 2: Change = $20.00 - $9.60 = $10.40
The answer is $10.40  ← Correct

The key insight: generating intermediate steps forces the model to allocate more “compute” (more tokens, more transformer passes) to each reasoning step.

Step 3: Self-Consistency Sampling

A single chain-of-thought can still go wrong. Self-consistency generates multiple independent reasoning chains and takes the majority answer:

import openai
from collections import Counter

def self_consistent_answer(question, n=5):
    answers = []
    for _ in range(n):
        response = openai.chat.completions.create(
            model="gpt-4o",
            messages=[{"role": "user", "content": f"{question}
Let's think step by step."}],
            temperature=0.7  # Sample diversity
        )
        # Extract final answer from response
        answers.append(extract_answer(response.choices[0].message.content))
    
    # Majority vote
    return Counter(answers).most_common(1)[0][0]

Step 4: ReAct Pattern (Reasoning + Acting)

ReAct (Yao et al., 2022) interleaves reasoning traces with tool calls:

User: What is the population of the capital of France?

Thought: I need to find the capital of France, then look up its population.
Action: search("capital of France")
Observation: Paris is the capital of France.

Thought: Now I need the population of Paris.
Action: search("Paris population 2024")
Observation: Paris metropolitan area population is approximately 12.2 million.

Thought: I have the answer.
Answer: The capital of France is Paris, with a metropolitan population of approximately 12.2 million.

Step 5: Known Reasoning Failures

Reversal Curse: Models trained on “A is B” don’t generalize to “B is A”:

• Knows: “Tom Cruise’s mother is Mary Lee Pfeiffer”
• Fails: “Mary Lee Pfeiffer’s son is ___”

Hallucination: Models confabulate plausible-sounding but incorrect facts, especially for less common knowledge.

Compositional Failure: Simple steps combined become unreliable (e.g., “reverse this string then count its vowels”).

Solution patterns: RAG for factual grounding, code execution for arithmetic, structured output validation.

Key Takeaways

• LLMs are auto-regressive next-token predictors — they don’t plan; each token is chosen conditioned on all prior tokens.
• Chain-of-thought prompting improves multi-step reasoning by allocating more token compute to intermediate steps.
• Self-consistency (majority vote across multiple samples) significantly improves reliability on reasoning tasks.
• ReAct interleaves reasoning with tool calls — the standard architecture for LLM-powered agents.
• Known failure modes include the reversal curse, hallucination on low-frequency facts, and compositional reasoning breakdowns.