LiteLLM

Component Category

LLM proxy and gateway

Component Description

LiteLLM provides a unified gateway for accessing language models and model providers through a consistent interface.

Why It Is Used

It simplifies how CoreAI services call models, apply shared access patterns, and manage model routing and credentials across different providers and endpoints.

Learn More

• LiteLLM Documentation
• LiteLLM on GitHub

Position In CoreAI

LiteLLM is an important internal CoreAI dependency, but it should not usually be treated as the primary long-term integration surface for product developers. The preferred stable product-facing contract is the CoreAI API.

Interacts With

• CoreAI Web Portal for user-facing model access
• CoreAI API for application-level LLM calls
• Model Installer for shared model access patterns
• OpenTelemetry and MLflow for tracing and GenAI observability
• Runtime inference services such as vLLM and KubeAI
• External model providers where applicable