Structured Outputs

Structured outputs are essential for ensuring that language model responses are both controlled and predictable. By defining a clear schema for the expected output, we can leverage the power of language models to generate responses that adhere to specific formats and constraints.

Consider the following example, which demonstrates how to use Pydantic models to define structured outputs in ell:

from pydantic import BaseModel, Field

class MovieReview(BaseModel):
    title: str = Field(description="The title of the movie")
    rating: int = Field(description="The rating of the movie out of 10")
    summary: str = Field(description="A brief summary of the movie")

@ell.complex(model="gpt-4o-2024-08-06", response_format=MovieReview)
def generate_movie_review(movie: str) -> MovieReview:
    """You are a movie review generator. Given the name of a movie, you need to return a structured review."""
    return f"generate a review for the movie {movie}"

By defining the MovieReview model, we ensure that the output of the generate_movie_review function adheres to a specific structure, making it easier to parse and utilize in downstream applications. This approach not only enhances the reliability of the generated content but also simplifies the integration of language model outputs into larger systems.

Once we have defined and generated structured outputs, we can easily access and manipulate the data within them. Let’s continue with our movie review example to demonstrate how to work with structured outputs:

# Generate a movie review
message = generate_movie_review("The Matrix")
review = message.parsed

# Access individual fields
print(f"Movie Title: {review.title}")
print(f"Rating: {review.rating}/10")
print(f"Summary: {review.summary}")

In this example, we first generate a movie review using our generate_movie_review function. We can then access individual fields of the structured output directly, as shown in the first part of the code.

Note

Structured outputs using Pydantic models are currently only available for the gpt-4o-2024-08-06 model. For other models, you’ll need to manually prompt the model and enable JSON mode to achieve similar functionality.

We purposefully chose to not opinionate prompting for other non-native json models because each prompt should be customized to the specific model and situation. For example if you want to get gpt-3.5-turbo to return json you should explicitly allow it by prompting the model to do so:

class MovieReview(BaseModel):
    title: str = Field(description="The title of the movie")
    rating: int = Field(description="The rating of the movie out of 10")
    summary: str = Field(description="A brief summary of the movie")

@ell.simple(model="gpt-3.5-turbo")
def generate_movie_review_manual(movie: str):
    return [
        ell.system(f"""You are a movie review generator. Given the name of a movie, you need to return a structured review in JSON format.

You must absolutely respond in this format with no exceptions.
{MovieReview.model_json_schema()}
"""),
        ell.user("Review the movie: {movie}"),
    ]

# parser support coming soon!
unparsed = generate_movie_review_manual("The Matrix")
parsed = MovieReview.model_validate_json(unparsed)