Stop Parsing Broken Outputs: Generate Valid Structure Directly From Any LLM

You know the pain. You ask an LLM for JSON, and it gives you almost JSON. Or JSON with trailing commas. Or a code block that starts with json but ends abruptly. So you write regex parsers, try json.loads() with error handling, or resort to asking the model to "please fix your output" in the next turn. It works, but it's fragile and slow.

What if you could skip the parsing entirely? What if you could tell the LLM "only output a valid Python function call" or "generate a string that matches this regex" — and have it actually do that, reliably?

Outlines does exactly that. It's a library that lets you constrain any LLM's output to match a structured format — JSON, Pydantic models, regex, grammar, or even a function signature — before generation. No post-processing. No retries. Just valid output, the first time.

What It Does

Outlines is a Python library that works with any text generation model (OpenAI, Anthropic, local models via transformers or llama.cpp). Instead of letting the model generate free text and then parsing it, you define the structure you want upfront. The library uses the model's next-token probabilities to force the output to follow a schema — grammatically, structurally, and type-safely.

You can constrain output to:

• JSON matching a Pydantic model or dataclass
• A specific regex pattern
• A context-free grammar (CFG)
• A function call with typed arguments (great for tool use)
• Any combination of the above

The magic is that it works with any autoregressive model and adds minimal overhead. The constraints are applied at token sampling time, so the model never even considers invalid tokens.

Why It’s Cool

The "parse broken output" approach is a kludge. It works 80% of the time, then breaks silently. Outlines flips the script — it makes structure a first-class part of generation. Here's why that matters:

No more “please fix your JSON” prompts. You define a Pydantic model once, and Outlines guarantees the output matches it. No post-hoc validation, no fallback logic.

Faster generation. Because the model never considers invalid tokens, it can't waste time generating garbage it will have to correct later. For many use cases, this makes generation actually faster.

Tool calling without the wrapper. If you want the LLM to call send_email(to: str, body: str), Outlines can force the output to be exactly that function call — no need for custom OpenAI function-calling middleware.

Works with local models. You're not locked into commercial APIs. Outlines supports huggingface models, llama.cpp, and exllama. Your structured output pipeline works the same way whether you use GPT-4o or Llama 3.2.

Composable constraints. You can combine regex and JSON constraints, mix function calling with free text, or define a grammar that accepts certain tokens conditionally. It's surprisingly flexible.

How to Try It

Getting started takes about two minutes:

pip install outlines

Then define your structure and generate:

import outlines

# Define a Pydantic model for your output
from pydantic import BaseModel

class Person(BaseModel):
    name: str
    age: int

# Pick a model (works with OpenAI, local, etc.)
model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")

# Generate structured output
generator = outlines.generate.json(model, Person)
person = generator("Create a character for a fantasy novel")
print(person)  # Person(name="Eldrin the Swift", age=142)

That's it. No JSON parsing. No error handling. The output is already a validated Person object.

For more advanced use cases — regex constraints, grammars, function calling — check the repo's examples.

Final Thoughts

Outlines is one of those libraries that makes you wonder why everyone wasn't doing this already. It's small, focused, and solves a real pain point without adding a lot of complexity. If you're building anything where LLMs produce structured data — API responses, database entries, tool calls, form fills — give it a shot. You might be surprised how much simpler your code gets.

The project is actively maintained and has a friendly community. It's not trying to be a framework; it's a tool you reach for when you need reliable structured output. I think a lot of developers will find that's a tool they didn't know they needed.

Follow us at @githubprojects for more developer tools and libraries.