Command Palette

Search for a command to run...

Function Calling

Benched.ai Editorial Team

Function calling enables a language model to return a structured JSON object that describes which function to run and with what arguments. Clients then execute deterministic code, call external APIs, or query databases based on the model's plan.

  Schema Elements

FieldPurposeExample
nameUnique identifiersearch_products
descriptionNatural-language hint for model"Retrieve catalog items by keyword"
parametersJSON Schema declaring argument types{"type":"object", "properties":{ "query": {"type":"string"}}}

  End-to-End Flow

  1. Client registers one or more function schemas with the LLM endpoint.
  2. User asks: "Find me GPU servers under $2/hr."
  3. Model replies with a JSON tool call, e.g. { "name": "search_products", "arguments": {"query": "GPU server price<2"} }.
  4. Client validates arguments, performs search API call, and (optionally) feeds results back to the model for final answer.

  Comparison to Plugins

AspectFunction CallingPlugin runtimes
InvocationJSON returned inlineSeparate request URL
SecurityTyped argumentsOAuth scopes
LatencySingle round-tripExtra HTTP hops

  Design Trade-offs

  • Rich schemas reduce hallucinated keys but increase prompt tokens.
  • Two-step responses (call → results → summary) improve accuracy but double latency.
  • Auto-executing functions raises risk of unintended side effects; require user confirmation for high-impact calls.

  Current Trends (2025)

  • OpenAPI ↔ JSON Schema converters let devs reuse existing specs.
  • Rate-limit middleware throttles costly functions like place_order.
  • Execution traces collected during live traffic feed back into model fine-tuning to boost tool reliability.

  Implementation Tips

  1. Keep function names short to save tokens.
  2. Prefer enums over free-form strings for categorical arguments.
  3. Log every invocation and response pair for audit replay and debugging.