Use local models via API

Connect local tools and applications to BullSequana AI models through the BSQAI API.

Agentic Friendly

When developing against BullSequana AI, the preferred integration point is the BSQAI API.

Do not integrate directly with the model servers behind it, such as KServe. They are internal platform components and may change over time. The BSQAI API is the stable interface that BullSequana AI exposes to developers. It serves every model under its model name, whether the model runs on the platform or at a cloud provider.

This page is primarily for application developers and AI engineers integrating external tools or custom applications with the platform.

Helpful experience includes:

  • API-based application development
  • bearer-token or API-key authentication
  • local development with Docker when reproducing platform-adjacent flows
  • basic understanding of OpenAI-compatible clients and SDKs

What to use

Use the API endpoint exposed by your BullSequana AI deployment:

export BSQAI_BASE_URL="https://api.<platform-domain>/v1"

BullSequana AI uses api as the default public hostname prefix. A deployment can expose a site-specific hostname, so use the endpoint supplied by the platform operator when it differs from this example.

This API already handles the platform integration behind the scenes:

  • model routing
  • authentication
  • platform policy enforcement
  • future component evolution behind the API boundary

Authentication options

The backend supports both of these:

  • JWT bearer tokens
  • BullSequana API keys in the sk-bsq-... format

For interactive user access, JWT is a good fit.

For local developer tools, scripts, IDE plugins, and long-lived integrations, API keys are usually the cleaner option when your deployment exposes API key management.

How to pass credentials

API keys can be sent in either of these ways:

  • Authorization: Bearer sk-bsq-... — the same header used for JWTs
  • X-Api-Key: sk-bsq-... — a dedicated API-key header

Both are accepted. JWT tokens always use Authorization: Bearer <jwt-token>.

When using OpenAI-compatible SDKs or tools, the Authorization: Bearer form is the most convenient because most clients already send the configured key in that header. The X-Api-Key header is available as an alternative when the Authorization header is reserved for another purpose or when you want an explicit separation between JWT and API-key authentication.

Option 1: Authenticate with JWT

If your platform uses Keycloak-backed interactive access, obtain a JWT and pass it as the bearer token.

Example:

export BSQAI_TOKEN="<jwt-token>"

curl "$BSQAI_BASE_URL/models" \
  -H "Authorization: Bearer $BSQAI_TOKEN"

Select a tenant for JWT requests

The Keycloak organization claim contains the tenants available to the user. When the user belongs to more than one organization, select one tenant by sending its Organization UUID:

export BSQAI_TENANT_ID="<keycloak-organization-uuid>"

curl "$BSQAI_BASE_URL/models" \
  -H "Authorization: Bearer $BSQAI_TOKEN" \
  -H "X-Tenant-Id: $BSQAI_TENANT_ID"

The header is optional when the JWT has exactly one tenant membership. Browser WebSocket clients use the tenant_id query parameter because they cannot set custom headers during the handshake.

A JWT comes from your deployment's Keycloak realm through its normal sign-in flow, and it expires. For scripts and tools that run unattended, use an API key instead.

Option 2: Create and use an API key

API keys are created through the BSQAI API itself, or in the Portal, and are returned only once when created. Each key carries a fixed set of permissions chosen at creation. A request made with the key must be allowed both by those permissions and by the owner's current access.

First list the permissions you can give a key, calling the API with a valid JWT:

curl "https://api.<platform-domain>/v1/api-keys/options" \
  -H "Authorization: Bearer <jwt-token>"

The response lists the permissions from your current access in the active tenant that a key can carry. Each entry's implied_permission_ids are added automatically when you select it.

The response also lists funding_groups: the groups you belong to in this tenant, each with an id and a name. The group you choose as the key's funding_group_id is the team its usage counts for, in usage reports and against that group's monthly BCU cap. When you belong to any group, you must choose one. Omit it only when you belong to none. In the Portal, this is the key's Team field. It cannot be changed later.

Then create the key with a name, at least one permission ID and, when you belong to a group, its funding group:

curl -X POST "https://api.<platform-domain>/v1/api-keys" \
  -H "Authorization: Bearer <jwt-token>" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Local Development",
    "permission_ids": ["tenant.read", "chat.message.create"],
    "funding_group_id": "<group-id-from-options>"
  }'

Typical response shape:

{
  "id": "a37e6b2e-728e-4ffc-aea4-e5415616a96a",
  "name": "Local Development",
  "key": "sk-bsq-v1-...",
  "masked": "sk-bsq-v1-********-e5b4a3",
  "created_at": "2024-01-15T10:30:00Z",
  "permission_ids": ["chat.message.create", "tenant.read"],
  "effective_permission_ids": ["chat.message.create", "tenant.read"],
  "funding_group_id": "<group-id-from-options>"
}

effective_permission_ids holds the selected permissions plus the permissions they imply. Choose the permissions for the requests the key will make:

RequestPermission
List models (/v1/models)tenant.read
Generate responses (/v1/responses)chat.message.create
Generate chat completions (/v1/chat/completions)chat.message.create
Use the Anthropic-compatible routes (/anthropic/v1/...), the model list includedchat.message.create
Read model details (/v1/models/info)tenant.read
Create embeddings (/v1/embeddings)tenant.read
Rerank documents (/v1/rerank)tenant.read
Call predictive models (/v1/serving-endpoints/...)tenant.read
Hold voice conversations, transcribe, and generate speech (/v1/voice, /v1/stt, /v1/tts, and their subpaths)chat.message.create
Read your own usage and monthly caps (/v1/usage/me, /v1/usage/budgets/me)tenant.read
Use an Agent through agent_idagent.configuration.use
Search files with file_searchstorage.object.read
Use Connectorsmcp.server.use

A request without permission_ids is rejected with 422. A selection that includes a permission you do not hold, or one a key cannot carry, is rejected with 400 and the error code api_key_scope_invalid. A missing funding_group_id while you belong to a group, or a group you do not belong to, is rejected with 400 and the error code api_key_funding_group_invalid, with param set to funding_group_id. Credential management and most tenant administration cannot be assigned to a key. A key's permissions cannot be changed; create a new key instead.

Then use that key for requests. Both header styles are accepted:

export BSQAI_API_KEY="sk-bsq-v1-..."

# Option A: Bearer header (works with most OpenAI-compatible SDKs)
curl "$BSQAI_BASE_URL/models" \
  -H "Authorization: Bearer $BSQAI_API_KEY"

# Option B: Dedicated API-key header
curl "$BSQAI_BASE_URL/models" \
  -H "X-Api-Key: $BSQAI_API_KEY"

An API key is bound to the active tenant at creation time. It cannot be reused to access a different tenant.

Discover available models

Before wiring an application, check which models your deployment exposes:

Available models are also visible in the Portal.

If you want the API-level source of truth for the current environment, use:

curl "$BSQAI_BASE_URL/models" \
  -H "Authorization: Bearer $BSQAI_API_KEY"

This is the correct source of truth for model names in your environment.

The response uses OpenAI's list shape: {"object": "list", "data": [...]}. Each entry carries the model name in id and model_name, its mode, and context_length and max_completion_tokens when the model's route states them.

The list holds only the models you may use. Permissions decide which requests a key may make; allow rules decide which models it may call, and a key uses the models its owner may use. A model without a rule for you fails with 404 and model_not_found, like an unknown one. The people who manage a model's rules set them with Access on the model in Studio → Available models: Tenant Administrators, the model's owner and its Grant Managers, and platform administrators for the Global Organization's models. See Model access.

Use the OpenAI-compatible API

The BSQAI API implements the OpenAI Responses API and OpenAI Chat Completions on the same base URL:

  • /v1/models
  • /v1/responses
  • /v1/chat/completions

That means SDKs and tools that speak either API can be pointed to BullSequana AI with only a base URL and token change. The official OpenAI SDKs use the Responses API by default.

The Responses API takes an input field, not a messages array — sending messages to /v1/responses returns a user_message_missing error. Agents, file_search, web search, and Connectors are available only through the Responses API.

The legacy text completions endpoint, /v1/completions, is not served.

Responses API example (curl)

curl -X POST "$BSQAI_BASE_URL/responses" \
  -H "Authorization: Bearer $BSQAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<model-name-from-/v1/models>",
    "input": [
      { "role": "user", "content": "Hello from BullSequana AI" }
    ]
  }'

Responses API example (Python)

from openai import OpenAI

client = OpenAI(
    api_key="sk-bsq-v1-...",
    base_url="https://api.<platform-domain>/v1",
)

response = client.responses.create(
    model="<model-name-from-/v1/models>",
    input="Explain the deployment model of BullSequana AI."
)

print(response)

When a Responses request uses the file_search tool, the answer text carries one file_citation annotation for each source passage. A file appears more than once when several of its passages were used.

FieldMeaning
file_idID of the cited file
filenameFile name as shown in Files
pagePage number, for documents with pages; null otherwise
indexPosition in the text the citation applies to; citations refer to the whole answer, so this is the end of the text
{
  "type": "output_text",
  "text": "Revenue grew by 12% in the third quarter.",
  "annotations": [
    { "type": "file_citation", "index": 41, "file_id": "<file-id>", "filename": "quarterly-report.pdf", "page": 3 }
  ]
}

In streaming responses, the annotations arrive with the completed text, in the response.content_part.done and response.output_item.done events and in the final response, not with individual text deltas.

Control reasoning effort

Set reasoning.effort to none, low, medium, high, or xhigh to trade answer depth for speed and output tokens on models that reason:

{
  "model": "<model-name-from-/v1/models>",
  "input": "Summarize the incident report.",
  "reasoning": { "effort": "low" }
}
  • Omit reasoning.effort to use the model's own default. Some models default to their highest effort.
  • Which values a model honours depends on the model. The platform rejects a value that the model's route rules out with 400 and the error code reasoning_effort_unsupported, with param set to reasoning.effort. For example, it rejects low for a model that does not reason. none is always accepted for such a model.
  • A model that has no route, or that no allow rule lets you use, fails with 404 and model_not_found.

A model can still map a value it does not support to its default. Check the reasoning_tokens in usage.output_tokens_details when the effort matters.

Invalid tool definitions

The platform validates every tool of a supported type, such as function, file_search, and custom, before it calls the model. A tool whose definition is invalid fails the request with 400, the error type invalid_request_error, and the error code invalid_tool_definition. The error's param names the field to fix, for example tools[0].parameters.required. This applies to streamed and non-streamed requests.

For example, parameters.required must be a list, a function name cannot contain spaces, and a file_search tool needs vector_store_ids. Tools sent in an additional_tools input item are checked the same way, and param points into that item, for example input[2].tools[0].name. Tools of types the platform does not support, such as mcp, are still accepted and ignored.

Handle incomplete responses

An HTTP 200 does not always mean the answer is complete. When the model reaches max_output_tokens, or a provider content filter stops the output, the response has status: "incomplete":

{
  "status": "incomplete",
  "incomplete_details": { "reason": "max_output_tokens" },
  "output": [
    {
      "type": "message",
      "status": "incomplete",
      "content": [{ "type": "output_text", "text": "..." }]
    }
  ],
  "usage": {
    "output_tokens": 8192,
    "output_tokens_details": { "reasoning_tokens": 6776 }
  }
}
  • incomplete_details.reason is max_output_tokens or content_filter.
  • Partial output and token usage are kept. The cut-off item has status: "incomplete". Reasoning models can run out of tokens while reasoning, so output can hold only a reasoning item, or nothing.
  • Streaming requests end with a single response.incomplete event instead of response.completed, followed by [DONE].
  • Never execute a function_call with status: "incomplete". Its arguments can be cut off.

Check the status before you use the answer:

response = client.responses.create(
    model="<model-name-from-/v1/models>",
    input="Write the migration plan.",
    max_output_tokens=2048,
)

if response.status == "incomplete":
    reason = response.incomplete_details.reason
    if reason == "max_output_tokens":
        print("Cut off. Partial answer:", response.output_text or "(none)")
    else:
        print("Stopped by a content filter:", reason)
else:
    print(response.output_text)

To avoid max_output_tokens cut-offs, raise max_output_tokens or lower reasoning.effort. Reasoning tokens count toward the same limit.

Chat Completions example (curl)

curl -X POST "$BSQAI_BASE_URL/chat/completions" \
  -H "Authorization: Bearer $BSQAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<model-name-from-/v1/models>",
    "messages": [
      { "role": "user", "content": "Hello from BullSequana AI" }
    ]
  }'

Chat Completions is stateless: send the whole conversation in messages, and run tool calls in your client.

  • Streaming sends text, reasoning_content, and tool calls, and ends with [DONE]. With stream_options.include_usage, a usage chunk precedes [DONE].
  • Function tools, tool_choice, and parallel tool calls are supported. When output is cut off inside a tool call, finish_reason is length.
  • response_format accepts json_schema and json_object.
  • reasoning_effort is passed to models that accept it and ignored for others.
  • model names a model, not an Agent.
  • n above 1, log probabilities, and audio are refused with 400, and the error names the field.

Errors on the OpenAI-compatible routes

/v1/responses, /v1/chat/completions, /v1/embeddings, and the /v1/models list return errors in OpenAI's format:

{
  "error": {
    "message": "The model rejected the request.",
    "type": "invalid_request_error",
    "code": "upstream_rejected",
    "param": "messages",
    "request_id": "<request-id>"
  }
}

On /v1/responses and /v1/chat/completions, a failed model call is classified as follows:

StatuscodeMeaning
400noneThe request body is invalid; param names the first field to fix
400upstream_rejectedThe model rejected the request
400context_length_exceededThe prompt exceeds the model's context window; the message states the window when it is known
400turn_limit_reachedThe response used all of its model calls, or reached max_tool_calls, before it finished; for max_tool_calls, param is max_tool_calls
400content_unsupportedChat Completions only: a message holds an image or file that is neither a base64 data: URI nor, for an image, an http(s) URL; param is messages
404model_not_foundThe model is unknown, or no allow rule lets you use it
429rate_limit_exceededThe model provider is rate-limiting; retry after the retry-after header when it is present
429usage_cap_reachedA monthly BCU cap is reached; the error type is insufficient_quota
503upstream_unavailableThe model is unavailable, for example while no pod serves it
500internal_error or noneThe platform failed

Error messages never contain the provider's text.

A usage_cap_reached answer carries the header x-should-retry: false. Retrying does not help until a tenant administrator raises the cap or the month ends. See What a refused call looks like.

Generate embeddings

Use POST /v1/embeddings with an embedding model exposed by your deployment. input accepts one nonempty string or a list of 1 to 2,048 nonempty strings. Token-ID arrays are not supported. The endpoint requires tenant.read in the authenticated tenant context.

curl "$BSQAI_BASE_URL/embeddings" \
  -H "Authorization: Bearer $BSQAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<embedding-model-from-/v1/models>",
    "input": ["Annual maintenance contract", "Replacement part availability"],
    "encoding_format": "float"
  }'

The response contains object: "list", the model name, a data array, and token usage. Each data item has object: "embedding", an index matching its input position, and an embedding vector. Read data[0].embedding for the first input, not a top-level embedding field.

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["BSQAI_API_KEY"],
    base_url=os.environ["BSQAI_BASE_URL"],
)
texts = ["Annual maintenance contract", "Replacement part availability"]
result = client.embeddings.create(
    model="<embedding-model-from-/v1/models>",
    input=texts,
    encoding_format="float",
)
for item in result.data:
    print(item.index, texts[item.index], len(item.embedding))
print(result.usage.prompt_tokens, result.usage.total_tokens)

For one text, pass a string instead of the list; the response still has a data array. encoding_format: "base64" returns each vector as a base64-encoded little-endian float32 byte sequence. Optional dimensions requests a positive output dimension only when the selected model supports it. Do not assume that arbitrary dimensions are accepted by every model.

Generating embeddings does not upload a file, create a collection, or add data to Portal RAG. Use the Files pipeline for those operations.

Track usage and monthly caps

BullSequana AI records every model call with the user and, for a call made with an API key, the key and its funding group. Usage is measured in BCU.

  • GET /v1/usage/me returns your own usage in the active tenant, including the calls made with your API keys. It takes first_day and last_day, and group_by breaks the period down, for example by api_key, model or application.
  • GET /v1/usage/budgets/me returns your monthly BCU cap for the current month and the caps of the groups that fund your API keys.

Both need only tenant.read, so a key that carries tenant.read can read its owner's usage:

curl "$BSQAI_BASE_URL/usage/budgets/me" \
  -H "Authorization: Bearer $BSQAI_API_KEY"

A call with an API key is checked against the owner's cap and against the cap of the key's funding group before it starts. When either is reached, the call is refused with 429 and usage_cap_reached. Calls already running finish. The same figures appear in the Portal under My usage. See Review your usage and Manage monthly BCU allowances and caps.

Inspect AI-output provenance

Generative HTTP responses include headers that identify AI-generated output:

X-AI-Generated: true
X-AI-Provenance: model="<resolved-model>", generated-at="<timestamp>", producer="coreai-llm-backend"

producer is a fixed identifier of the BSQAI API. The headers are present on Responses API and Chat Completions output, Anthropic-compatible Messages, Portal chat responses, and chat-history reads that contain generated output. Streaming responses send them before the first event. Error responses and non-generative endpoints do not include them.

IDE and tooling integrations

For tools such as OpenCode or custom internal applications, point the tool to the BSQAI API, not to a model server directly.

The correct pattern is:

  • provider type: OpenAI (Responses API or Chat Completions)
  • base URL: your BSQAI API base URL
  • bearer token: JWT or sk-bsq-... API key

Tools that call Chat Completions for the platform's model names, such as Continue, work with the same base URL as tools that use the Responses API. See Developer Tools.

GitHub Copilot BYOK in VS Code

To connect Copilot through Custom Endpoint, follow Configure GitHub Copilot in VS Code. The guide covers secret storage, stateless Responses requests, and a file tool-call check.

Serving-route boundaries

Model workloads remain behind the internal inference gateway. Its explicit path allowlist exposes supported inference calls, not arbitrary runtime administration, model-loading, or file-management endpoints. A route supported by a serving engine is not automatically a public BSQAI API route. Keep applications on the public API contracts documented here; use Studio → Deploy a model or the authorized Model Installer operations for deployment changes.

Use this rule for all developer-facing integrations:

Application or tool -> BSQAI API -> platform components

Not:

Application or tool -> model server

That keeps the developer contract stable even if the platform team changes the internal inference or proxy layer later.

On this page