Local LLM (Chatbot-Tool)
AnythingLLM is a free desktop chat application that can be connected to the BFH-TI MLMP Inference service as its model backend. You get the same models and API access as the web interface, while your conversations, workspace settings, and chat history stay on your machine.
Feature Overview
| Feature | Description |
|---|---|
| Workspaces | Isolated projects, each with its own documents, system prompt, model settings, and chat history |
| Document chat (RAG) | Upload PDFs, Office files, text, web pages, and more; answers can cite your sources |
| Chat modes | Chat (documents + general knowledge), Query (documents only), and Agent (tools and skills) |
| AI agents | Multi-step tasks using built-in skills, custom tools, and MCP integrations |
| Embeddings & vector search | Built-in local embedder and LanceDB — no extra database setup |
| Privacy | Single-user, runs on your machine; data is not sent to AnythingLLM servers |
When you connect AnythingLLM to the MLMP inference API, only chat prompts and model responses travel over the network. Your documents, embeddings, and conversation history stay local.
Setup Guides
Follow the guide for your platform to install AnythingLLM and connect it to the MLMP OpenAI-compatible API. You will need:
- Access to the BFH network (on campus or via VPN)
-
A personal API key from the Inference service
- Linux