Inference / API

Warning

Do not upload sensitive or personal data. You are responsible for how you use the platform and its outputs. The MLMP team does not guarantee accuracy and is not liable for misuse.

Accessing the LLM inference API

1. Log in to https://inference.mlmp.ti.bfh.ch
2. Navigate to:
   Settings → Account → API keys
3. Press "Show" and copy the API key (NOT the JWT token).
4. Securely store your token — you will use this to authenticate requests.

API keys expire after 180 days

Make sure to create a new API key to ensure continous access to our services.

Feature Overview

Feature	Status
Function calling	✅ Supported (via /chat/completions)
Web search	✅ Supported (via /openai/responses endpoint)
Remote MCP tool calling	✅ Supported (via /openai/responses endpoint)
Speech to Text (STT model Whisper)	✅ Supported (via /openai/audio/transcriptions endpoint)
Text to Speech (TTS model Qwen3)	✅ Supported (via /openai/audio/speech endpoint)
PDF Upload	✅ Supported
Streaming	✅ Supported
Embeddings	✅ Supported
Vision / Image Support	✅ Supported (check specific models below)
Batch Requests	❓ Unknown
Custom Models	⏳ Beta Access

• ✅ Supported - Feature is available and working
• ❓ Unknown - Please test and report back
• ⏳ Beta Access - Send us a message if you need exclusive beta access to features under development
• ❌ Not Planned - Not currently planned (convince us if you need it)

Available Resources & Models

• Base URL for SDKs: https://inference.mlmp.ti.bfh.ch
• Proxy Base URL for API access from outside the BFH network: https://inference.proxy.ti.bfh.ch
• Endpoints: /api/v1/chat/completions - /api/v1/audio/transcriptions Experimental: /openai/responses (not supported via proxy)
• Serving Framework: LMStudio

Models

• gpt-oss:120b (Tool Use, Reasoning)
• gemma-4-e4b (Vision, Tool Use, Reasoning)
• embeddinggemma:300m (Embedding)
• glm5.2 (Tool Use, Reasoning)
• deepseek-v4 (Reasoning)
• qwen3.5-397b-a17b-mlx (Vision, Tool Use, (Reasoning - unverified))
• qwen3.6-35b-a3b (Vision, Tool Use, Reasoning)
• qwen3.6-27b (Vision, Tool Use, Reasoning)
• whisperx (Speech-to-text)
• qwen3 (Text-to-speech)

We are open for model suggestions!

Feel free to contact us if you want to use a specific model for your project.

Rate limits

Please consider the following rate limits that apply to every user: - Maximum number of concurrent requests: 2 - Tokens per Minute: 150k - Requests per Minute: 60

System status

You can check for system outages here.

Live latency

The panel below refreshes every minute and shows recent inference latency (last hour, only visible when connected to the BFH network).

---

Examples

Sending Requests to the API

You can use any OpenAI-API-compatible package. Example code snippet:

from openai import OpenAI

class LLMService:

    def __init__(self):
        self.client = OpenAI(
            base_url='https://inference.mlmp.ti.bfh.ch/api/v1',
            api_key='sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'  # your PAT 
        )

    def generate_completion(
        self,
        system_prompt: str,
        user_prompt: str,
        temperature: float = settings.DEFAULT_TEMPERATURE
    ) -> Dict[str, Any]:
        try:
            response = self.client.chat.completions.create(
                model="gpt-oss:120b",
                messages=[
                    {"role": "system", "content": system_prompt},
                    {"role": "user", "content": user_prompt}
                ],
                temperature=temperature
            )

            return {
                "text": response.choices[0].message.content.strip(),
            }

        except Exception as e:
            raise ServiceConnectionError(f"Error connecting to Ollama service: {str(e)}")

---

Create Embeddings

from openai import OpenAI

class EmbeddingService:

    def __init__(self):
        self.client = OpenAI(
            base_url='https://inference.mlmp.ti.bfh.ch/api/v1',
            api_key='sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'  # your PAT 
        )

    def create_embedding(
        self,
        text: str
    ) -> list[float]:
        try:
            response = self.client.embeddings.create(
                model="embeddinggemma:300m",
                input=text
            )

            return response.data[0].embedding

        except Exception as e:
            raise ServiceConnectionError(f"Error creating embedding: {str(e)}")

Create Transcriptions (No Diarization)

import requests
from typing import Dict, Any


class AudioTranscriptionService:

from openai import OpenAI
from typing import Dict, Any

class AudioTranscriptionService:

    def __init__(self):
        self.client = OpenAI(
            base_url='https://inference.mlmp.ti.bfh.ch/api/v1',
            api_key='sk----------------------------------'
        )

    def transcribe(
        self,
        file_path: str,
        model: str = "whisperx"
    ) -> Dict[str, Any]:
        try:
            with open(file_path, "rb") as audio_file:
                response = self.client.audio.transcriptions.create(
                    file=audio_file,
                    model=model
                )

            return {
                "text": response.text
            }

        except Exception as e:
            raise Exception(f"Transcription failed: {str(e)}")

# Usage example:       
service = AudioTranscriptionService()
result = service.transcribe("output.mp3")
print(result)

Create Transcriptions + Diarization

Diarization data is dropped by our standard endpoint. Therefore, we have to grant you explicit access to a separate API. Please let us know in advance if you require diarization so we can provide you with the right API key.

There are more fields that you can use additional to 'diarize': 'true', For details go to the API reference section here: WhisperX Documentation

import requests
from typing import Dict, Any


class AudioTranscriptionService:

    def __init__(self, base_url: str = "https://tracking.mlmp.ti.bfh.ch/v1", api_key: str = "sk--------------------------"):
        self.base_url = base_url.rstrip("/")
        self.api_key = api_key

    def transcribe(self, file_path: str, model: str = "whisperx", diarize: str = "true") -> Dict[str, Any]:
        url = self.base_url + "/audio/transcriptions"
        headers = {"Authorization": "Bearer " + self.api_key}

        with open(file_path, "rb") as audio_file:
            files = {"file": audio_file}
            data = {"model": model, "diarize": diarize}
            response = requests.post(url, headers=headers, files=files, data=data, timeout=300)

        response.raise_for_status()
        return response.json()  # full raw JSON, nothing dropped


# Usage example:
if __name__ == "__main__":
    service = AudioTranscriptionService()
    result = service.transcribe("output.mp3")
    print(result)

Create Audio

For details, visit the Qwen3 Documentation

from openai import OpenAI
from typing import Dict, Any

class AudioGenerationService:

    def __init__(self):
        self.client = OpenAI(
            base_url='https://inference.mlmp.ti.bfh.ch/api/v1',
            api_key='sk-------------------------------------',

        )

    def generate(
        self,
        text: str,
        model: str = "qwen3-tts",
        voice: str = "Vivian",
        language: str = "English",
        instruct: str = "neutral, no emotion",
        output_file: str = "output.mp3"


    ) -> Dict[str, Any]:
        try:
            response = self.client.audio.speech.create(
                model=model,
                voice=voice,
                input=text,

            )

            # Save audio to file
            with open(output_file, "wb") as f:
                f.write(response.content)

            return {
                "file": output_file
            }

        except Exception as e:
            raise Exception(f"TTS generation failed: {str(e)}")

# Usage example:       
service = AudioGenerationService()
result = service.generate(text="Do not upload sensitive or personal data. You are responsible for how you use the platform and its outputs. The MLMP team does not guarantee accuracy and is not liable for misuse.")
print(f"Audio saved to: {result['file']}")

Function calling

{
    "model": "gpt-oss:120b",
    "messages": [
        {
          "role": "user",
          "content": "What is the weather like in Paris today?"
        }
      ],
"tools": [
    {
      "type": "function",
      "function": {
        "name": "get_current_weather",
        "description": "Get the current weather",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {
                "type": "string",
                "description": "The city and country, eg. San Francisco, USA"
            },
            "format": { "type": "string", "enum": ["celsius", "fahrenheit"] }
          },
          "required": ["location", "format"]
        }
      }
    }
   ]
  }

Source

Remote MCP tool calling

{
    "model": "gpt-oss:120b",
    "tools": [{
      "type": "mcp",
      "server_label": "tiktoken",
      "server_url": "https://gitmcp.io/openai/tiktoken",
      "allowed_tools": ["fetch_tiktoken_documentation"]
    }],
    "input": "What is the first sentence of the tiktoken documentation?"
}

Source

Web search

{
    "model": "gpt-oss:120b",
    "input": "What is happening in Switzerland today?",
    "tools": [
        {
            "type": "mcp",
            "server_label": "litellm",
            "server_url": "litellm_proxy",
            "require_approval": "never"
        }
    ],
    "stream": false,
    "tool_choice": "required"
}

Postman Examples

You can find Postman requests examples JSON for every endpoint in the Teams Inference Channel on the "Released" section