About
The MaluOS Local LLM Server: a node that runs local language models for a business and reports its queue, runs, models and usage through a node API. The Sovereign LLM Manager application in the Business OS manages a fleet of these nodes and their policies; it never serves an inference itself.
What it does
- Node API for queue, runs and usage
- Managed as a fleet from the Business OS
Install
From source
git clone https://github.com/maludb/maluos-local-llm-server && cd maluos-local-llm-server- Follow the repository's instructions, then register the node in the Sovereign LLM Manager.
Downloads
| File | For | Size | |
|---|---|---|---|
| Source (zip) | any | Get | |
| Source (tar.gz) | any | Get |