Model as a Service

Deploy models on the platform, add external ones, and serve both through the BSQAI API.

Agentic Friendly

BullSequana AI serves every model through the BSQAI API under a model name. Applications, Portal chat, Build, and Agents send that name, and the BSQAI API forwards the request to the server that runs the model. The model server's address and key stay in the BSQAI API.

How a model request is served

On every model request, the BSQAI API:

  1. authenticates the caller with a platform token or an API key;
  2. resolves the model name to its model route, and the route to its connection;
  3. checks that an allow rule in the caller's organization lets them call the model, and answers as if the model did not exist when none does;
  4. checks the caller's monthly BCU cap, and the cap of the group that funds their API key; and
  5. forwards the request over the connection, to a model on the platform or to an external server, and records the call's usage for the tenant, the member, and the API key.

The BSQAI API keeps routes and connections in its own database and reads them on every request, so a change applies to the next request.

Where models come from

SourceWho adds itHow it is served
A model deployed with the Model Installer API or StudioPlatform administrators in the Global organization; members with an Inference role in a customer organizationA KServe LLMInferenceService for an LLM, an embedding model, a reranker, or LoRA adapters, or a KServe InferenceService for a predictive model. The installer adds its route and connection.
A model at a cloud provider or on another serverPlatform administratorsA model route and connection that an administrator adds. See Cloud and external models.

The Global organization's models run in the kserve namespace and keep their bare names. A customer organization deploys only when a platform administrator has enabled inference for it, and only within the deployment policy and GPU allocation the administrator sets. Its models run in its own namespace, tenant-<slug>, and callers name them <slug>/<name>.

Who may call a model

Seeing a model and calling it are separate. Calling a model needs an allow rule in the caller's organization, for LLMs and predictive models alike, and for API keys as for the Portal. The person who deploys a model in a customer organization owns it and calls it without a rule. A model that a platform administrator adds, or that the Global organization deploys, starts with no rule.

A customer organization's model serves only that organization until its tenant administrators publish it to every tenant, which a platform administrator must allow first. Only language models are published; predictive models stay in their organization. A platform administrator chooses which tenants may use a Global model, and can narrow a published model to chosen tenants. Each tenant then decides with its own allow rules which of its members may call a model offered to it. The owning tenant keeps running and paying for a published model, and other tenants see the model, never its deployment.

In this section

On this page