AI

How to Use Gemini Structured Output with Function Calling

Learn how to enforce strict JSON schemas in Gemini API responses while leveraging function calling for tool-based logic.

By · · 6 min read

Short answer

To enforce strict JSON output in Gemini, use the responseSchema parameter in the request configuration. Combine this with function calling to trigger external logic. By defining a clear Pydantic model or JSON schema, you ensure the API returns data that matches your expected format, allowing for type-safe integration in your applications.

Key takeaways

  • Always provide a detailed schema to minimize hallucinated fields.
  • Use responseMimeType: application/json to enforce format.
  • Combine responseSchema with function declarations for intelligent tool dispatch.
  • Validate outputs at the runtime level to catch structural drifts.

Why structured output matters for production AI

Traditional LLM responses arrive as unstructured text, which necessitates brittle regex patterns or complex string parsing to integrate into a standard application. This approach is prone to errors when the model occasionally changes its output style or includes conversational filler, leading to failed parsing logic that disrupts your data flow.

Structured output forces the model to generate responses that strictly adhere to a pre-defined JSON schema. By defining a schema, you shift from dealing with unpredictable chat bubbles to handling type-safe data objects. This transition is essential for building automated pipelines where an unexpected format would otherwise crash your backend service or cause database injection errors.

When an application relies on unstructured output, developers often find themselves writing excessive defensive code to sanitize and clean LLM responses. This creates significant technical debt. By enforcing a schema at the API level, you ensure that the AI acts as a reliable data provider, allowing your application to trust the shape of the incoming data entirely. This reliability simplifies downstream testing, as you no longer need to account for a near-infinite variety of response formats.

How to implement responseSchema in Python

The most effective way to define your schema is using Pydantic models. This approach allows you to inherit from BaseModel, define your structure with standard Python types, and easily convert them into JSON schema format for the Gemini API. By utilizing Python's native type hinting system, you maintain a consistent contract between your application code and the AI response model.

Once the schema is passed via the response_format parameter, the Gemini model constrains its output to strictly follow that structure. You can then use Pydantic's model_validate_json method to deserialize the response directly into your code logic. This process provides an immediate feedback loop; if the model violates the expected structure, your application will raise a validation error before the data enters your primary pipeline.

To maximize success, use the Field description attribute within your Pydantic models. These descriptions act as system prompts that explain to the model exactly what each field should contain. By providing high-quality descriptions, you reduce the likelihood of the model producing irrelevant content within your requested fields, effectively increasing the precision of the output without needing more complex prompt engineering.

from google import genai
from pydantic import BaseModel, Field

class PostMetadata(BaseModel):
    title: str = Field(description="The title of the post.")
    tags: list[str] = Field(description="Keywords for the blog post.")
    word_count: int

client = genai.Client()
response = client.interactions.create(
    model="gemini-4-hydrogen",
    input="Generate metadata for a blog about AI agents.",
    response_format={
        "type": "text",
        "mime_type": "application/json",
        "schema": PostMetadata.model_json_schema()
    }
)

metadata = PostMetadata.model_validate_json(response.output_text)
print(metadata.title)

Combining structured output with function calling

When you need the model to perform actions like querying a database or fetching weather data, use function calling. You provide a list of tool definitions, and the model decides if one of them is relevant to the request. This allows for modular application design where the AI acts as a router for various external logic providers.

You can combine this with structured outputs by setting the response format to match your expected return types. The model will then either return a structured data object or initiate a function call based on your prompt requirements. This is particularly useful when you want to enforce that even if the AI decides not to call a function, its final answer to the user must follow a specific organizational schema.

In practice, this means your application can support multi-stage workflows. The model might call a function to retrieve data, process that data, and then return the final result as a structured JSON object. This chain of events removes the need for custom orchestration code and allows the LLM to manage the flow of information between its capabilities and your internal systems.

Comparison: Structured Outputs vs. Function Calling

It is essential to understand when to use structured outputs versus function calling, as they solve distinct developer challenges. While they are often used together in complex pipelines, they have fundamentally different purposes.

Structured outputs handle the format of the information returned to the client, ensuring that data is consistently parsed by your backend. Function calling, in contrast, handles the operational side of executing logic outside the LLM environment. By keeping these two concepts distinct in your architecture, you gain finer control over both the quality of your data and the reliability of your tool executions.

FeatureStructured OutputsFunction Calling
Primary GoalFormatting response dataTaking external actions
Model BehaviorReturns a JSON objectRequests a tool invocation
Use CaseData extraction, classificationAPI lookups, database queries

Gotchas and technical limitations

The model's ability to generate valid JSON is bound by token state machines and finite memory during generation. If your schema is too large or deeply nested, the request will be rejected by the server to preserve stability and latency. Avoiding excessive depth in your JSON structure is the most effective way to prevent these server-side rejections.

Some constraints, such as minLength, maxLength, or specific regex patterns defined in your JSON schema, may be ignored by the model's constrained decoding engine. The model focuses on structural integrity rather than granular value validation. You must always validate these business-level constraints in your own application code after the response is received to ensure full data integrity and security.

Additionally, keep in mind that schema size consumes token space. A highly verbose schema with dozens of fields will increase the prompt size and reduce the available token budget for the actual content generation. Aim for the minimum necessary schema structure to perform your task effectively, as this will result in lower latency and higher consistency from the model.

How to debug invalid or rejected schemas

If you encounter a 400 INVALID_ARGUMENT error, it usually indicates that your schema complexity exceeds the model's limit for state transitions. Simplify the schema by flattening nested structures. If you find your models reaching these limits, consider breaking your JSON request into smaller, more granular sub-tasks that utilize smaller schemas.

Instead of deep recursion, use references or simplify the hierarchy. Use descriptive keys and ensure that all mandatory fields are clearly marked in your schema's required array to guide the model effectively. Frequently, adding a simple example or a explicit directive in your system prompt can help the model align better with a complex schema without needing to change the schema structure itself.

For difficult debugging sessions, try stripping your schema to its bare essentials and adding fields back one by one. This will help you identify which specific part of your data structure is causing the model to struggle. Testing schemas with small, controlled payloads before moving them into your full-scale production environment is a best practice that prevents sudden, large-scale failures during deployment.

Frequently asked questions

Does structured output work with streaming?

Yes, you can stream structured outputs. The API will return valid partial JSON chunks as they are generated, which can be concatenated to form the final JSON object in your client application. Be mindful that partial JSON may not be parsable until the final stream event.

Can I use Gemini structured output with REST?

Yes, you can use structured output with the REST API by setting the response_format property in the JSON request body. Ensure the schema is provided as a valid JSON Schema object exactly matching the expected format constraints of the Gemini API.

Why is my minLength constraint not working?

The constrained decoding engine strips certain constraints like minLength, maxLength, and uniqueItems. The model uses the structure for valid JSON syntax, not as a validator for individual string lengths. You should always enforce these properties in your own application logic after receiving the response.

Sources & further reading

About the author

Tushar Gautam is a full-stack & ai product engineer in India. He builds AI SaaS products such as an AI video ad generator, a RAG chatbot platform and a Razorpay-powered report service, mostly with Next.js, Flask and the Gemini API. See the projects, GitHub or LinkedIn.

This article was drafted with the help of AI (Google Gemini) from official documentation and published by Tushar Gautam. Spotted an error? Let me know.