Models & API Clients¶

In language model programming, the relationship between models and API clients is crucial. ell provides a robust framework for managing this relationship, offering various ways to specify clients for models, register custom models, and leverage default configurations.

Model Registration and Default Clients¶

ell automatically registers numerous models from providers like OpenAI, Anthropic, Cohere, and Groq upon initialization. This allows you to use models without explicitly specifying a client.

If no client is found for a model, ell falls back to a default OpenAI client. This enables the utilization of newly released models without updating ell for new model registrations. If the fallback fails because the model is not available in the OpenAI API, you can register your own client for the model using the ell.config.register_model method or specify a client when calling the language model program below

Specifying Clients for Models¶

ell offers multiple methods to specify clients for models:

Decorator-level Client Specification:

import ell
import openai

client = openai.Client(api_key="your-api-key")

@ell.simple(model="gpt-next", client=client)
def my_lmp(prompt: str):
    return f"Respond to: {prompt}"

Function Call-level Client Specification:

result = my_lmp("Hello, world!", client=another_client)

Global Client Registration:

ell.config.register_model("gpt-next", my_custom_client)

Custom Model Registration¶

For custom or newer models, ell provides a straightforward registration method:

import ell
import my_custom_client

ell.config.register_model("my-custom-model", my_custom_client)

Multimodality

Configuration