Tool Use (Function Calling)

What is tool use?

Tool use lets an LLM act on the world. You describe functions in JSON Schema (name, description, parameters); the model, when it decides one is needed, outputs a structured call like get_weather({"city": "Tokyo"}) instead of prose. Your code executes the function and returns the result to the model, which continues with real data. The model never runs anything itself — it only emits requests.

Why it’s the foundational primitive

Every capability layered on LLMs reduces to tool use: agents are tool use in a loop, MCP is a standard for packaging tools, agentic RAG is retrieval-as-a-tool, structured output is a tool schema with one required call. Master this primitive and the rest of the stack is composition.

Tool design is API design (for a distractible reader)

The model chooses tools based solely on names, descriptions, and parameter schemas. What works: task-shaped tools (search_orders_by_customer, not run_sql), descriptions that say when to use the tool and when not to, enums over free strings, and error messages that tell the model what to try instead — a good error turns a failed call into a self-correction.

Cost and reliability reality

Tool definitions live in the context window and are billed on every call — 30 verbose tools can add thousands of tokens before the user says a word, and selection accuracy drops as the tool count grows. Curate a small set per use case. Each round-trip (call → execute → return → continue) is a full model invocation; chatty tool loops multiply latency and spend.

What people get wrong

• Trusting arguments blindly. Model-generated parameters are untrusted input — validate before executing, especially anything touching money, deletion, or other users’ data (guardrails).
• Returning raw API dumps. A 40 KB JSON response buries the model; return the fields it needs, formatted for reading.
• Exposing every endpoint as a tool. Wrap workflows, not your OpenAPI spec.

Try it in code

The complete tool-use loop every framework wraps:

tools = [{
    "name": "get_order_status",
    "description": "Look up the current status of a customer order. "
                   "Use when the user asks where their order is.",
    "input_schema": {
        "type": "object",
        "properties": {"order_id": {"type": "string"}},
        "required": ["order_id"],
    },
}]

def run(user_message):
    messages = [{"role": "user", "content": user_message}]
    while True:
        response = llm(messages, tools=tools)        # model may request a tool
        if response.stop_reason != "tool_use":
            return response.text                     # done - final answer
        call = response.tool_call
        result = execute(call.name, validate(call.arguments))  # YOUR code runs it
        messages.append(response.to_message())
        messages.append({"role": "tool_result", "id": call.id, "content": result})

Note the validate() — model-generated arguments are untrusted input, always.