JSON mode constrains a language model to emit syntactically valid JSON that matches a predefined schema, simplifying client parsing and reducing hallucination risk.
When to Use JSON Mode
Enabling JSON Mode (examples)
Design Trade-offs
- Strict JSON forbids natural-language context; may need separate
explanation
field. - Larger token count due to quotes and braces.
- Schema too tight can cause frequent
invalid_request
errors.
Current Trends (2025)
- Streaming JSON mode with incremental tokens allows clients to parse as they arrive.
- JSON + Comments (JSONC) proposal adds developer hints stripped before validation.
- Compiler-like validators provide diff suggestions when model output deviates.
Implementation Tips
- Include a trailing newline after closing brace to detect completion.
- Reject extra keys with strict deserializers (e.g.,
pydantic.Strict
). - Log raw strings before parsing for easier debugging.