LLM (Large Language Model)

What is an LLM?

An LLM is a transformer neural network with billions to trillions of parameters, trained on internet-scale text to do one deceptively simple thing: predict the next token. From that objective — plus post-training to follow instructions — emerge translation, coding, reasoning, and conversation. Examples: GPT, Claude, Gemini, Llama, Mistral, Qwen.

How they’re built (the three stages)

1. Pretraining — predict next tokens across trillions of words. Produces a “base model”: knowledgeable but not helpful.
2. Post-training — supervised fine-tuning on instruction examples plus preference optimization (RLHF and successors) makes it follow instructions and decline harmful requests.
3. Reasoning training — reinforcement learning on verifiable problems teaches models to “think” before answering; see reasoning models.

What developers actually need to know

An LLM is stateless — it remembers nothing between API calls; all “memory” is you re-sending the conversation in the context window. It’s probabilistic — the same input can produce different outputs (see temperature). And it’s a text-in, text-out function — every capability beyond that (tools, retrieval, memory) is engineering built around the model, not inside it.

What people get wrong

• Treating it as a database. Training knowledge is compressed, dated, and unverifiable — see hallucination; facts that matter should come from retrieval.
• Assuming it understands like a human. It excels at pattern completion; failures cluster where surface patterns diverge from underlying logic (arithmetic on unusual numbers, novel puzzle variants).
• Picking by leaderboard. Benchmark rank correlates weakly with performance on your task; run your own evals on candidate models.