What are the most effective techniques for structured output generation without breaking JSON syntax? (Part 2 Focus)

Use a JSON‑schema‑aware prompting pattern together with function‑calling or tool‑use to enforce structure while guaranteeing the LLM emits pure JSON.

Step‑by‑step workflow
1. Define a JSON schema (Draft‑07 or OpenAI function spec). Example for a product record:

{
  "type": "object",
  "properties": {
    "id": {"type": "string"},
    "price": {"type": "number", "minimum": 0},
    "tags": {"type": "array", "items": {"type": "string"}}
  },
  "required": ["id", "price"]
}

2. Wrap the schema in a system prompt and request the model to only return JSON matching it.
- temperature=0.0‑0.2
- max_tokens=512
- response_format={"type":"json_object"} (OpenAI v1 API)
3. Enable function calling (or tool mode) so the model must invoke the defined function rather than free‑form text.

import openai

def get_product():
    return {
        "name": "function",
        "parameters": schema,
        "description": "Return a product JSON object"
    }

resp = openai.ChatCompletion.create(
    model="gpt-4o-mini",
    messages=[{"role":"system","content":"You must output JSON exactly as defined."}],
    functions=[get_product()],
    temperature=0.1,
    max_tokens=300,
)

4. Post‑process validation: run jsonschema.validate; if it fails, automatically retry with retry_on_error=True and a stricter logit_bias for {, } characters.
5. Log token usage; abort if usage.total_tokens > 0.9 * context_window (e.g., 7,200 of 8k) to keep latency predictable.

Quick comparison
| Technique | Guarantees JSON? | Latency impact | Implementation effort |
|---|---|---|---|
| Plain prompt + regex filter | ❌ | low | minimal |
| System‑prompt schema + temperature ≤ 0.2 | ✅ (high) | moderate | low |
| Function calling / tool use | ✅ (strict) | slight ↑ (extra round‑trip) | moderate |
| Post‑hoc JSON repair (e.g., json5) | ✅ (soft) | low | low |

Checklist before deployment
- [ ] Schema validated with ajv or jsonschema.
- [ ] temperature ≤ 0.2 and response_format set.
- [ ] Function spec registered and function_call="auto".
- [ ] Retry logic caps at 3 attempts.
- [ ] Token‑budget guard at 90% of context window.