Track experiments

Record runs, metrics, parameters and artifacts in MLflow from Studio or a BullSequana AI dev environment.

Agentic Friendly

Every run belongs to an experiment. Create experiments and runs under Models > Experiments in Studio, or through the BSQAI API from a dev environment, a pipeline step or CI. Either way the experiment is yours and private until you share it.

The MLflow SDK's tracking functions (mlflow.start_run(), mlflow.log_metric(), autologging) do not work from a dev environment: MLflow is reachable only through the BSQAI API. Use the bsqai_mlflow client or plain HTTP requests instead.

Log a run with the bsqai_mlflow client

The Python JupyterLab image (jupyterlab:python3.12) includes bsqai_mlflow, a client for the BSQAI MLflow API. Its MlflowClient has the methods of mlflow.MlflowClient, with the same names and argument order, and raises MlflowApiError when the API refuses a call.

Create an API key with the MLflow permissions your code needs, then log in once from a notebook. The client saves the address and key in ~/.config/bsqai/credentials.json on your workspace volume, readable only by you, so later sessions on the same dev environment need no login. BSQAI_API_URL and BSQAI_API_KEY take precedence over the saved values.

import os
import bsqai_mlflow

bsqai_mlflow.login(os.environ["PLATFORM_API_URL"])  # prompts for the API key
from bsqai_mlflow import MlflowClient

client = MlflowClient()
experiment = client.create_experiment("my-exp")
run = client.create_run(experiment.experiment_id, run_name="baseline")

# ... your training code here, saving the model to the local folder model_out ...

client.log_param(run.run_id, "learning_rate", "0.01")
client.log_metric(run.run_id, "accuracy", 0.95)
client.log_artifacts(run.run_id, "model_out", "model")
client.set_terminated(run.run_id)

Results are plain objects (run.run_id, not run.info.run_id), search_* methods take a free-text search instead of filter_string, and search_runs takes one experiment. Each BSQAI API serves the client's reference at https://<api-host>/docs/sdk/python/mlflow/.

Record work as a service account

For unattended pipelines or CI, a tenant service account can own the experiment and its runs instead of a person. A tenant administrator creates it under Administration > Access Control > Service accounts.

  • From a notebook, a member who may act as the service account passes its ID: MlflowClient(service_account_id="<service account ID>"). The client exchanges your notebook session for a short-lived token of the service account.
  • In a job, give the client the service account's credentials in BSQAI_CLIENT_ID, BSQAI_CLIENT_SECRET and BSQAI_TOKEN_URL, for example from a Kubernetes Secret, and create MlflowClient() as usual.

What the client creates as a service account belongs to the service account, not to the member acting as it.

Log a run through the BSQAI API

Any HTTP client can call the same routes. Create an API key with the MLflow permissions your code needs and make it available to your notebook, for example as BSQAI_API_KEY. The key belongs to the organization it was created in, so runs land in that organization's MLflow. Every dev environment already has the API address in PLATFORM_API_URL.

import os
import requests

api = os.environ["PLATFORM_API_URL"]
headers = {"X-Api-Key": os.environ["BSQAI_API_KEY"]}
experiment_id = "123"  # from Studio, or the response that created the experiment

run = requests.post(
    f"{api}/v1/mlflow/experiments/{experiment_id}/runs",
    json={"run_name": "baseline"},
    headers=headers,
)
run.raise_for_status()
run_id = run.json()["run_id"]

# ... your training code here ...

requests.post(
    f"{api}/v1/mlflow/runs/{run_id}/batch",
    json={
        "params": [{"key": "learning_rate", "value": "0.01"}],
        "metrics": [{"key": "accuracy", "value": 0.95, "step": 1}],
    },
    headers=headers,
).raise_for_status()

with open("results.txt", "rb") as artifact:
    requests.put(
        f"{api}/v1/mlflow/runs/{run_id}/artifacts/content",
        params={"path": "results.txt"},
        data=artifact,
        headers=headers,
    ).raise_for_status()

requests.post(
    f"{api}/v1/mlflow/runs/{run_id}/terminate",
    json={"status": "FINISHED"},
    headers=headers,
).raise_for_status()

A batch carries up to 1,000 metrics, 100 parameters and 100 tags, and at most 1,000 entries in total. Ending the run through terminate also records its end time, so Studio shows its duration; use FAILED or KILLED when the training did not finish. Create an experiment with POST /v1/mlflow/experiments and a name. You need write access on the experiment to add runs to it.

Manage records in the Portal

Use Experiments in Studio to create experiments and run records, compare runs, update names, descriptions, statuses and tags, and delete or restore entries. Save each field before leaving edit mode; Done does not save drafts. Starting a run in the Portal does not execute notebook or training code.

Next steps

On this page