Function Calling

Source Excerpt

Function calling enables Large Language Models to interact with external tools and APIs by generating structured function calls instead of natural language responses. The model receives function schemas, determines when and how to call them, and returns structured arguments for execution.

How It Works

Flow

User prompt → Large Language Model → Function call (name + args) → Application executes → Result → Large Language Model → Final response

1. Define functions: Provide function schemas (name, description, parameters) with the prompt
2. Model decides: Large Language Model determines if a function call is needed based on the prompt
3. Generate arguments: Model produces structured JSON arguments
4. Execute: Application runs the function with the provided arguments
5. Return result: Function result is sent back to the model
6. Final response: Model incorporates the result into its response

Function Schema (OpenAI Format)

{
  "name": "get_weather",
  "description": "Get current weather for a location",
  "parameters": {
    "type": "object",
    "properties": {
      "location": {
        "type": "string",
        "description": "City and state, e.g. San Francisco, CA"
      },
      "unit": {
        "type": "string",
        "enum": ["celsius", "fahrenheit"],
        "description": "Temperature unit"
      }
    },
    "required": ["location"]
  }
}

Model Response

{
  "id": "call_abc123",
  "type": "function",
  "function": {
    "name": "get_weather",
    "arguments": "{\"location\":\"New York, NY\",\"unit\":\"fahrenheit\"}"
  }
}

Provider Implementations

OpenAI

• tools parameter in chat completions
• tool_choice: "auto" (model decides), "none", or specific function
• Parallel function calling: multiple calls in single response
• Strict mode: JSON schema validation enforced by model

Anthropic (Claude)

• tools parameter in messages API
• Tool use blocks in response
• Extended thinking + tool use supported
• Multiple tool calls per response

Google (Gemini)

• tools parameter with function declarations
• Tool calling mode: auto, any, none
• Supports parallel function calls

Best Practices

Schema Design

• Clear names: get_weather not gw or func1
• Detailed descriptions: Explain what the function does and when to use it
• Typed parameters: Use specific types (string, number, boolean, array, object)
• Required vs optional: Mark only truly required parameters
• Enum constraints: Use enums for fixed sets of values

Error Handling

• Validate arguments: Check types, ranges, required fields before execution
• Return structured errors: Send clear error messages back to the model
• Retry logic: Allow the model to retry with corrected arguments
• Timeout handling: Set reasonable timeouts for external API calls

Multiple Functions

• Parallel calling: When functions are independent, allow parallel execution
• Sequential calling: When functions depend on each other, execute sequentially
• Function selection: Help the model choose between similar functions with clear descriptions

Common Patterns

Retrieval-Augmented Generation (RAG)

Function: search_knowledge_base(query: string) → list of documents
Use: Ground Large Language Model responses in verified knowledge

Database Queries

Function: query_database(sql: string) → list of rows
Use: Dynamic data retrieval from structured databases
Caution: Validate SQL, use parameterized queries, limit result size

API Integration

Function: call_external_api(endpoint: string, method: string, body: object) → response
Use: Integrate with external services (payment, shipping, weather)

Multi-Step Workflows

Step 1: search_products(category: string) → product list
Step 2: get_product_details(product_id: string) → details
Step 3: check_inventory(product_id: string) → stock level
Step 4: Final response with synthesized information

Security Considerations

• Input validation: Never trust model-generated arguments blindly
• Output filtering: Sanitize function results before returning to model
• Access control: Restrict which functions are available based on user context
• Rate limiting: Protect external APIs from excessive calls
• Audit logging: Log all function calls for debugging and compliance

Limitations

• Hallucinated arguments: Model may generate arguments that don't match schema
• Over-calling: Model may call functions when not needed
• Under-calling: Model may skip function calls when they should be used
• Argument quality: Model may generate valid but suboptimal arguments
• Token cost: Function schemas add to context window usage

Related

• ai-ml/tool-calling
• ai-ml/structured-output-generation
• ai-ml/model-context-protocol