LLM Wrappers

The LLM wrappers provide a unified interface for working with different language models.

They are designed for use with the Toponymy API and offer a consistent way to configure and interact with LLM providers.

Most wrappers are convenience layers around (Async)LiteLLMNamer with sensible defaults. Any model supported by LiteLLM can also be used directly via LiteLLMNamer by specifying the appropriate parameters.

class toponymy.llm_wrappers.LiteLLMNamer(api_key: str = None, model: str = 'openai/gpt-4o-mini', api_base: str = None, llm_specific_instructions=None, use_json_object: bool = None, disable_system_prompts: bool = False, max_tokens_topic_name: int = 128, max_tokens_cluster_names: int = 1024, provider_kwargs: dict[str, Any] | None = None, callback: Callable[[dict[str, Any]], None] | None = None)

Bases: LLMWrapper

Provides access to any LLM supported by LiteLLM using a unified interface. LiteLLM supports 100+ providers including OpenAI, Anthropic, Cohere, HuggingFace, Together, Replicate, and more. For more information, see https://docs.litellm.ai.

Parameters:

api_key: str, optional

The API key for the provider. If not provided, LiteLLM will look for the appropriate environment variable for the provider (e.g. OPENAI_API_KEY, ANTHROPIC_API_KEY).

model: str, optional

The LiteLLM model string, e.g. “openai/gpt-4o-mini”, “anthropic/claude-haiku-4-5-20251001”, etc.

api_base: str, optional

Optional LiteLLM/OpenAI-compatible API base. Alias-style convenience.

llm_specific_instructions: str, optional

Additional instructions appended to the user prompt.

use_json_object: bool, optional

Whether to request JSON object output via response_format={“type”: “json_object”}. If None (default), support is detected automatically by check if response_format is supported for the specified model. Set to True to force JSON object mode, or False to disable it.

disable_system_prompts: bool, False

Set to True to override to use plain calls instead of system prompts. If False (default), system prompt support is detected automatically and will flatten system prompts if unsupported for a given model.

max_tokens_topic_name: int, optional

Default maximum number of tokens for topic name generation. Default is 128. Can be overridden per-call in generate_topic_name().

max_tokens_cluster_names: int, optional

Default maximum number of tokens for cluster name generation. Default is 1024. Can be overridden per-call in generate_topic_cluster_names().

provider_kwargsdict[str, Any], optional

Additional keyword arguments passed directly to litellm.completion() / litellm.acompletion(). This allows callers to use LiteLLM-specific features such as provider routing, request timeouts, custom headers, user identifiers, or other provider parameters without modifying the wrapper.

These values are merged into the completion call arguments but may be overridden by core wrapper parameters such as model, messages, temperature, and max_tokens.

Attributes:

model: str: The LiteLLM model string being used.
extra_prompting: str: Additional instructions appended to the prompt.
max_tokens_topic_name: int: Default maximum tokens for topic name generation.
max_tokens_cluster_names: int: Default maximum tokens for cluster name generation.
use_json_object: bool: Whether response_format={“type”: “json_object”} will be sent.

FAIL_FAST_EXCEPTIONS: tuple = (<class 'litellm.exceptions.AuthenticationError'>, <class 'litellm.exceptions.PermissionDeniedError'>, <class 'litellm.exceptions.BadRequestError'>, <class 'litellm.exceptions.NotFoundError'>, <class 'litellm.exceptions.UnprocessableEntityError'>)

property supports_system_prompts: bool: Check if the LLM wrapper supports system prompts. By default, it does. Override in subclasses if not supported.

class toponymy.llm_wrappers.AsyncLiteLLMNamer(api_key: str = None, model: str = 'openai/gpt-4o-mini', api_base: str = None, llm_specific_instructions: str = None, max_concurrent_requests: int = 10, use_json_object: bool | None = None, disable_system_prompts: bool = False, max_tokens_topic_name: int = 128, max_tokens_cluster_names: int = 1024, provider_kwargs: dict[str, Any] | None = None, callback: Callable[[dict[str, Any]], None] | None = None)

Bases: AsyncLLMWrapper

Provides access to any LLM supported by LiteLLM with asynchronous support. This allows for concurrent processing of multiple prompts across 100+ providers including OpenAI, Anthropic, Cohere, HuggingFace, Together, Replicate, and more. For more information, see https://docs.litellm.ai.

As an asynchronous wrapper this will potentially speed up topic naming, particularly when you have a large number of topics. If, however, there are quirks in your data, or bugs in Toponymy’s prompt generation, you will potentially quickly spend money on API calls.

Uses litellm.acompletion() and an asyncio semaphore for bounded concurrency. Since this wrapper does not create a persistent SDK client, close() is a no-op.

FAIL_FAST_EXCEPTIONS: tuple = (<class 'litellm.exceptions.AuthenticationError'>, <class 'litellm.exceptions.PermissionDeniedError'>, <class 'litellm.exceptions.BadRequestError'>, <class 'litellm.exceptions.NotFoundError'>, <class 'litellm.exceptions.UnprocessableEntityError'>)

async close(): No-op for parity with other async wrappers.

property supports_system_prompts: bool: Check if the LLM wrapper supports system prompts. By default, it does. Override in subclasses if not supported.

LLM Convenience Wrappers

The following wrappers include helpful presets for convenience purposes (and their Async equivalents).

toponymy.llm_wrappers.OpenAINamer(model: str = 'openai/gpt-4o-mini', api_key: str | None = None, api_base: str | None = None, llm_specific_instructions: str | None = None, max_tokens_topic_name: int = 128, max_tokens_cluster_names: int = 1024, provider_kwargs: dict[str, Any] | None = None, callback: Callable[[dict[str, Any]], None] | None = None, base_url: str | None = None, http_client: Client | None = None) → LiteLLMNamer

Create a LiteLLMNamer configured for OpenAI.

All namers share the same interface once constructed — OpenAINamer is a convenience entry point, not a special case. For more information on OpenAI, see https://platform.openai.com/docs/models/overview.

Parameters:

modelstr, optional: OpenAI model to use. Default is “gpt-4o-mini”, a good balance of quality and cost for topic naming. You can use more advanced models, but they have diminishing returns for this task, and are more expensive. Must be in LiteLLM format (“openai/gpt-4o-mini”) or bare OpenAI format (“gpt-4o-mini”) — both are accepted.
api_keystr, optional: OpenAI API key. Falls back to the OPENAI_API_KEY environment variable.
api_basestr, optional: Override the OpenAI API endpoint. Useful for proxies or OpenAI-compatible local servers (e.g. vLLM, LM Studio). Can use the OPENAI_API_BASE environment variable. Default is the standard OpenAI endpoint.
use_json_objectbool, optional: Request JSON object output via response_format. If None (default), support is detected automatically for the selected model. Set to False to disable if your model doesn’t support it.
llm_specific_instructionsstr, optional: Additional instructions appended to every prompt. This can be used to provide model-specific instructions or context that may help improve the quality of the generated text.
provider_kwargsdict, optional: Additional keyword arguments passed directly to the LiteLLM completion call. Use for provider-specific features not covered by the parameters above, e.g. {"timeout": 30}.
callbackDebugCallback, optional: Optional callback function for observability. Called on each LLM request and response with a structured payload. Useful for logging, debugging, or recording prompts and responses to a file.
base_urlstr, optional: Deprecated. Use api_base instead.
http_clientoptional: Deprecated. Pass via provider_kwargs={'http_client': <client>} instead.

Returns:

LiteLLMNamer: A fully configured namer ready for use with Toponymy.

See also

LiteLLMNamer: The underlying namer, supports 100+ providers directly.

Examples

Basic usage:

namer = OpenAINamer(api_key="my-api-key")
toponymy = Toponymy(embedding_model=..., llm_namer=namer)

Using a different model:

namer = OpenAINamer(model="gpt-4o", api_key="my-api-key")

Using an OpenAI-compatible local server:

namer = OpenAINamer(model="hosted-model", api_base="http://localhost:8000/v1", api_key="none")

toponymy.llm_wrappers.AnthropicNamer(model: str = 'claude-haiku-4-5-20251001', api_key: str | None = None, api_base: str | None = None, llm_specific_instructions: str | None = None, max_tokens_topic_name: int = 128, max_tokens_cluster_names: int = 1024, provider_kwargs: dict[str, Any] | None = None, callback: Callable[[dict[str, Any]], None] | None = None) → LiteLLMNamer

Create a LiteLLMNamer configured for Anthropic with convenient defaults for topic naming. For more flexibility, use LiteLLMNamer directly with the model and parameters of your choice.

All namers share the same interface once constructed — AnthropicNamer is a convenience entry point, not a special case.

Parameters:

modelstr, optional: Anthropic model to use. Default is “claude-haiku-4-5-20251001”. May be in LiteLLM format (“anthropic/claude-haiku-4-5-20251001”)
api_keystr, optional: Anthropic API key. Falls back to the ANTHROPIC_API_KEY environment variable.
api_basestr, optional: Override the Anthropic API endpoint. Useful for proxies or Anthropic-compatible local servers (e.g. vLLM, LM Studio). Can use the ANTHROPIC_API_BASE environment variable. Default is the standard OpenAI endpoint.
llm_specific_instructionsstr, optional: Additional instructions appended to every prompt. This can be used to provide model-specific instructions or context that may help improve the quality of the generated text.
provider_kwargsdict, optional: Additional keyword arguments passed directly to the LiteLLM completion call. Use for provider-specific features not covered by the parameters above, e.g. {"timeout": 30}.
callbackDebugCallback, optional: Optional callback function for observability. Called on each LLM request and response with a structured payload. Useful for logging, debugging, or recording prompts and responses to a file.

Returns:

LiteLLMNamer: A fully configured namer ready for use with Toponymy.

See also

LiteLLMNamer: The underlying namer, supports 100+ providers directly.

Examples

Basic usage:

namer = AnthropicNamer(api_key="my-api-key")
toponymy = Toponymy(embedding_model=..., llm_namer=namer)

Using a different model:

namer = AnthropicNamer(model="claude-3-5-20251001", api_key="my-api-key")

Using an Anthropic-compatible local server:

namer = AnthropicNamer(model="hosted-model", api_base="http://localhost:8000/v1", api_key="none")

toponymy.llm_wrappers.CohereNamer(model: str = 'command-r-08-2024', api_key: str | None = None, api_base: str | None = None, llm_specific_instructions: str | None = None, max_tokens_topic_name: int = 128, max_tokens_cluster_names: int = 1024, provider_kwargs: dict[str, Any] | None = None, callback: Callable[[dict[str, Any]], None] | None = None, base_url: str | None = None, httpx_client: Client | None = None) → LiteLLMNamer

Create a LiteLLMNamer configured for Cohere with convenient defaults for topic naming. For more flexibility, use LiteLLMNamer directly with the model and parameters of your choice.

All namers share the same interface once constructed — CohereNamer is a convenience entry point, not a special case.

Parameters:

modelstr, optional: Cohere model to use. Default is “command-r-08-2024”. May be in LiteLLM format (“cohere/command-r-08-2024”)
api_keystr, optional: Cohere API key. Falls back to the COHERE_API_KEY environment variable.
api_basestr, optional: Override the Cohere API endpoint. Useful for proxies or Cohere-compatible local servers (e.g. vLLM, LM Studio). Can use the COHERE_API_BASE environment variable. Default is the standard OpenAI endpoint.
llm_specific_instructionsstr, optional: Additional instructions appended to every prompt. This can be used to provide model-specific instructions or context that may help improve the quality of the generated text.
provider_kwargsdict, optional: Additional keyword arguments passed directly to the LiteLLM completion call. Use for provider-specific features not covered by the parameters above, e.g. {"timeout": 30}.
callbackDebugCallback, optional: Optional callback function for observability. Called on each LLM request and response with a structured payload. Useful for logging, debugging, or recording prompts and responses to a file.
base_urlstr, optional: Deprecated. Use api_base instead.
httpx_clientoptional: Deprecated. Pass via provider_kwargs={'httpx_client': <client>} instead.

Returns:

LiteLLMNamer: A fully configured namer ready for use with Toponymy.

See also

LiteLLMNamer: The underlying namer, supports 100+ providers directly.

Examples

Basic usage:

namer = CohereNamer(api_key="my-api-key")
toponymy = Toponymy(embedding_model=..., llm_namer=namer)

Using a different model:

namer = CohereNamer(model="command-r-08-2024", api_key="my-api-key")

Using a Cohere-compatible local server:

namer = CohereNamer(model="hosted-model", api_base="http://localhost:8000/v1", api_key="none")

toponymy.llm_wrappers.AzureAINamer(model: str, api_key: str | None = None, api_base: str | None = None, endpoint: str | None = None, llm_specific_instructions: str | None = None, max_tokens_topic_name: int = 128, max_tokens_cluster_names: int = 1024, provider_kwargs: dict[str, Any] | None = None, callback: Callable[[dict[str, Any]], None] | None = None) → LiteLLMNamer

Create a LiteLLMNamer configured for Azure AI.

All namers share the same interface once constructed — AzureAINamer is a convenience entry point, not a special case.

Parameters:

modelstr,: The deployed model name in Azure AI Foundry. Will be prefixed with “azure_ai/” automatically (e.g. “gpt-4o” → “azure_ai/gpt-4o”).
api_keystr, optional: Azure API key. Falls back to the AZURE_AI_API_KEY environment variable.
api_basestr, optional: The Azure AI Foundry endpoint URL. Preferred over endpoint for consistency with other factory functions. Falls back to the AZURE_AI_API_BASE environment variable if not provided.
endpointstr, optional: The Azure AI Foundry endpoint URL, e.g. “https://<your-resource-name>.openai.azure.com/”. Alias for api_base; if both are provided, api_base takes precedence.
llm_specific_instructionsstr, optional: Additional instructions appended to every prompt. This can be used to provide model-specific instructions or context that may help improve the quality of the generated text.
provider_kwargsdict, optional: Additional keyword arguments passed directly to the LiteLLM completion call. Use for provider-specific features not covered by the parameters above, e.g. {"timeout": 30}.
callbackDebugCallback, optional: Optional callback function for observability. Called on each LLM request and response with a structured payload. Useful for logging, debugging, or recording prompts and responses to a file.

Returns:

LiteLLMNamer: A fully configured namer ready for use with Toponymy.

See also

LiteLLMNamer: The underlying namer, supports 100+ providers directly.

Examples

Basic usage:

namer = AzureAINamer(model="deployed-model-name", api_base="https://<your-resource-endpoint>")
toponymy = Toponymy(embedding_model=..., llm_namer=namer)

Local LLM Wrappers

These wrappers are designed to work with locally running LLMs.

toponymy.llm_wrappers.OllamaNamer(model: str = 'llama3.2', api_key: str | None = None, api_base: str | None = None, llm_specific_instructions: str | None = None, max_tokens_topic_name: int = 128, max_tokens_cluster_names: int = 1024, provider_kwargs: dict[str, Any] | None = None, callback: Callable[[dict[str, Any]], None] | None = None, host: str | None = None) → LiteLLMNamer

Convenience wrapper for a LiteLLMNamer configured for local Ollama use.

For Ollama remote API use, use LiteLLMNamer(model=”ollama_chat/<model_name>”, api_key=<api_key>).

Parameters:

modelstr, optional: Ollama model to use. Default is “llama3.2”, Must be in LiteLLM format (“ollama_chat/llama3.2”) or bare Ollama format (“llama3.2”) — both are accepted.
api_keystr, optional: Used for authentication if your Ollama server requires it. Not needed for default local setup. Falls back to the OLLAMA_API_KEY environment variable if not provided.
api_basestr, optional: Override the Ollama host URL. Default is “http://localhost:11434”. Can use the OLLAMA_API_BASE environment variable.
llm_specific_instructionsstr, optional: Additional instructions appended to every prompt. This can be used to provide model-specific instructions or context that may help improve the quality of the generated text.
provider_kwargsdict, optional: Additional keyword arguments passed directly to the LiteLLM completion call. Use for provider-specific features not covered by the parameters above, e.g. {"timeout": 30}.
callbackDebugCallback, optional: Optional callback function for observability. Called on each LLM request and response with a structured payload. Useful for logging, debugging, or recording prompts and responses to a file.
hoststr, optional: Deprecated. Use api_base instead.

Returns:

LiteLLMNamer: A fully configured namer ready for use with Toponymy.

See also

LiteLLMNamer: The underlying namer, supports 100+ providers directly.

Examples

Basic usage:

namer = OllamaNamer()
toponymy = Toponymy(embedding_model=..., llm_namer=namer)

Using a different model:

namer = OllamaNamer(model="llama3.2")

class toponymy.llm_wrappers.HuggingFaceNamer(model: str, llm_specific_instructions=None, callback: Callable[[dict[str, Any]], None] | None = None, **kwargs)

Bases: LLMWrapper

Provides access to Huggingface models from Huggingface Hub with the Toponymy framework. For more information on Huggingface, see https://huggingface.co/docs/transformers/index. You will need the transformers library installed to make use of this wrapper, and you will need a model available on Huggingface Hub. This wrapper allows you to use models hosted on Huggingface Hub, rather than requiring a service API key. However, this does require you to have access to the model and suitable hardware to run it.

class toponymy.llm_wrappers.LlamaCppNamer(model_path: str, llm_specific_instructions=None, callback: Callable[[dict[str, Any]], None] | None = None, **kwargs)

Bases: LLMWrapper

Provides Access to LlamaCpp models with the Toponymy framework. For more information on LlamaCpp, see https://github.com/abetlen/llama-cpp-python. You will need llamma-cpp-python installed to make use of this wrapper, and you will need a local model file downloaded to use it. This Wrapper allows you to use local models, rather than requiring a service API key. However this does require you to have the model and suitable hardware to run it.

Note: This wrapper does not support system prompts, as LlamaCpp does not support them.

property supports_system_prompts: bool: Check if the LLM wrapper supports system prompts. By default, it does. Override in subclasses if not supported.