Quick answer

Ollama is still maintained, so you only need to leave if one of its documented limits bites. Use llama.cpp’s server for a lean, scriptable runtime, LM Studio or Jan for a desktop app, vLLM for concurrent serving on Linux, LocalAI for one API across text, speech and images, and KoboldCpp or TextGen for roleplay and writing. All seven expose an OpenAI-compatible API.

Contents

At a glance

Class
Self-Hosted

Provenance

  • Nov 18, 2026Published

Ollama is still maintained, so the reason to leave is never “it’s dead”. On 2026-11-17 its repository was not archived and had been updated that day. Move only when one of the things below applies to you, and pick the replacement by the job.

If you wantUseThe documented reason
A lean server you script, with the engine’s own flagsllama.cpp (llama-server)OpenAI-compatible routes, parallel decoding, continuous batching and a model router are in its server README
A desktop app with a model browserLM Studio, or Jan if you want open sourceLM Studio’s app is closed source; Jan’s README says Apache 2.0
Concurrent requests from many clients, on LinuxvLLMContinuous batching is its headline feature; no native Windows
One API for text, speech, images and embeddingsLocalAIIts README lists more than 80 backends behind OpenAI-format endpoints
Chat, story or roleplay front ends in one fileKoboldCpp or TextGenBoth bundle a web UI and an OpenAI-compatible API

The sections below give each project’s licence, platforms, model formats, API and GPU support as its own documentation states them. I did not run any of them for this page. The speed numbers I do quote come from the three measured comparisons I already published on one RTX 5080, linked in each section, and I give their conditions every time.

What Ollama is, and why people move off it

Ollama’s README opens with “Start building with open models” and shows a CLI (ollama run gemma4) plus a REST API on localhost:11434. It runs on macOS, Windows, Linux and in an official Docker image, is MIT licensed per its repository, and lists llama.cpp as its supported backend. Its docs describe an OpenAI-compatible API, import of Safetensors and GGUF models through a Modelfile, and GPU support for Nvidia (compute capability 5.0 and up with driver 550 or newer, and driver 570 or newer for compute capability 5.0 through 6.2), AMD through ROCm, Apple’s Metal, and Vulkan on Windows and Linux.

I found five reasons in the documentation that could justify a move. None of them is a verdict on Ollama.

  1. Concurrency defaults. Ollama’s FAQ says OLLAMA_NUM_PARALLEL, the number of parallel requests each model handles, defaults to 1, and that the RAM needed scales with that number times OLLAMA_CONTEXT_LENGTH. llama.cpp’s server README lists parallel decoding with multi-user support and continuous batching, vLLM’s README lists continuous batching, and LM Studio’s docs say Max Concurrent Predictions defaults to 4.
  2. Context window. Ollama’s context-length page sets the default by VRAM: 4k under 24 GiB, 32k from 24 to 48 GiB, 256k at 48 GiB or more (its FAQ gives 4096 tokens as the default and the OLLAMA_CONTEXT_LENGTH override). llama.cpp’s --ctx-size defaults to 0, meaning the size is loaded from the model. On my LM Studio comparison, an Ollama app context slider left at 256k (clamped to the model’s 131,072 maximum) took Llama 3.2 3B from 351 to 102 tokens per second on a 16 GB RTX 5080, so check this setting before you blame the runtime.
  3. The cloud features. Ollama’s docs describe cloud models and web search, and the FAQ gives a switch to turn them off: OLLAMA_NO_CLOUD=1, or disable_ollama_cloud in ~/.ollama/server.json. If you want a runtime with no hosted side at all, that setting does it inside Ollama. Moving does not obviously help: LM Studio’s pricing page and Jan’s README both describe optional hosted or remote models too.
  4. Formats. Ollama imports GGUF and Safetensors. If your model ships as something else, vLLM’s docs cover Hugging Face models and quantization formats such as AWQ and GPTQ, and TextGen’s README lists ExLlamaV3 and Transformers backends.
  5. Open-source GUI. If you want a desktop app and want the source, that is Jan, not LM Studio.

If none of those describes you, stay.

llama.cpp and its server

llama.cpp’s README calls it “LLM inference in C/C++” and states the goal as minimal setup and state-of-the-art performance on a wide range of hardware. It is MIT licensed. The README’s quick start now uses llama cli and llama serve, with Docker images and pre-built releases alongside; the server README and its Docker examples still call the binary llama-server.

  • Model formats: GGUF. The models doc says llama.cpp requires GGUF and that other formats convert with the repo’s convert_*.py scripts. -hf <user>/<model> downloads a GGUF from Hugging Face directly.
  • OpenAI-compatible API: yes. The server README lists chat completions, responses and embeddings routes, plus an Anthropic Messages-compatible route, and says that while no strong claims of compatibility are made, it supports many apps.
  • Model switching: a router mode (--models-dir, --models-max, default 4 loaded at once, autoload on by default) is documented. It is the nearest thing I found to choosing the model by name in each request, and you configure it yourself.
  • GPU: the build docs cover CUDA (Nvidia), HIP (AMD, with ROCm), Vulkan, SYCL (Intel), Metal (on by default on macOS), CPU and several other backends.
  • Suits: someone comfortable with flags who wants the engine without a model library in the way.

My measured comparison, Ollama vs llama.cpp on an RTX 5080, used Ollama 0.32.1 against llama.cpp b10507 (llama-server), full GPU offload, 4096 context, 256-token generations, decode medians of five runs:

ModelOllama decodellama.cpp decode
Llama 3.2 3B319 t/s338 t/s
Qwen 2.5 14B99.4 t/s101.4 t/s

The page also reports Ollama prefilling 4-7% faster at prompts of about 420 tokens, and a portability catch: Ollama’s gpt-oss file is tagged “gptoss”, which upstream llama.cpp refuses, so that comparison used different conversions of the model.

LM Studio

LM Studio is a desktop application, not open source. Its App Terms of Service (dated August 23, 2026) grant a non-exclusive licence for personal or internal business use and describe the software’s source code as a trade secret. Its homepage now markets the app as LM Studio Bionic, an agent for open models, and its pricing page lists a free plan that includes local models plus paid plans for hosted inference.

  • Platforms: Apple Silicon Macs on macOS 14 or newer (Intel Macs are not supported), Windows on x64 and ARM, and Linux on x64 and ARM64 as an AppImage.
  • Model formats: GGUF through llama.cpp on all three platforms; MLX models as well on Apple Silicon.
  • OpenAI-compatible API: yes: /v1/responses, /v1/chat/completions, /v1/completions, /v1/embeddings and /v1/models, plus an Anthropic-compatible route. The docs’ examples use port 1234.
  • Headless: llmster is documented as the GUI-free version, started with lms daemon up.
  • GPU: the system requirements page recommends at least 4 GB of dedicated VRAM on Windows. It does not enumerate GPU vendors.
  • Suits: a model browser and per-model tuning in a GUI, with the server a click away.

On my Ollama vs LM Studio bench (Ollama 0.32.15, LM Studio 0.4.21, one RTX 5080, context 4096, full GPU offload, temperature 0, seed 42, one serving slot, medians of five fresh-prompt runs), Ollama decoded 350.5 t/s and LM Studio 316.0 t/s on Llama 3.2 3B; on Qwen 2.5 14B it was 102.2 against 97.6. LM Studio prefilled faster on both at prompts of about 450 tokens, 20,337 against 17,862 t/s on the 3B model. Those are different Ollama and llama.cpp builds from the table above, so do not compare numbers across the two captures.

vLLM

vLLM’s README describes “a fast and easy-to-use library for LLM inference and serving”, built around PagedAttention, continuous batching and chunked prefill. The repository’s licence is Apache 2.0.

  • Platforms: the GPU install docs list Linux as the OS, Python 3.11 to 3.14, and state that vLLM does not support Windows natively; they point to WSL or community forks. The quickstart adds macOS on Apple Silicon through vLLM-Metal.
  • Model formats: Hugging Face models, with quantization including FP8, GPTQ/AWQ and others. The GGUF page says GGUF support is “highly experimental and under-optimized” and now lives in a separate plugin.
  • OpenAI-compatible API: yes, plus an Anthropic Messages API.
  • GPU: NVIDIA GPUs with compute capability 7.5 or higher, AMD GPUs (the docs list MI200, MI300, MI350, Radeon RX 7900 and 9000 series, and Ryzen AI parts, with ROCm 6.3 or newer and ROCm 7 for some), Intel GPUs, and several other accelerators.
  • Suits: a Linux box serving several clients at once. For one person chatting from a Windows desktop it is the wrong tool.

I have not measured vLLM on my RTX 5080, so there is no number for it here. The vLLM vs Ollama page works from the documentation for the same reason.

LocalAI

LocalAI’s README describes it as running open models “for chat, images, speech and video on your own machines”, with an API in the OpenAI and Anthropic formats and more than 80 backends. It is MIT licensed, and the README says the name and logo are not covered by that licence.

  • Platforms and install: the README lists an install script for Linux and macOS on Apple Silicon, a container (Docker, podman), and a desktop app for macOS and Linux that the README says arrives with its next release. It does not list a native Windows installer.
  • Model formats and sources: models from its gallery, Hugging Face GGUF files, the Ollama registry (ollama://) and OCI registries. Backends include llama.cpp, vLLM, MLX and whisper.cpp.
  • GPU: NVIDIA (CUDA 12 and 13), AMD (ROCm), Intel (oneAPI/SYCL), Apple Silicon, Vulkan, NVIDIA Jetson, or CPU only.
  • Suits: a homelab that wants one endpoint for text, speech and image jobs. The README also notes that some features it describes need a development build, so read it against the version you install.

Jan

Jan’s README calls it an open-source ChatGPT replacement that runs models locally, and the README’s licence section says Apache 2.0. The LICENSE file in the repository is an Apache 2.0 notice, though GitHub’s detector does not label it.

  • Platforms: Windows, macOS and Linux desktop builds. Its README lists GPU support for NVIDIA, AMD and Intel Arc on Windows and says acceleration is available on Linux.
  • Model formats: GGUF through its llama.cpp engine, which can also import a local GGUF file in place. MLX models on Apple Silicon, which Jan’s docs label experimental, with no embeddings and some newer architectures failing to load.
  • OpenAI-compatible API: yes, a local server at 127.0.0.1:1337 by default, with an optional API key and a setting to listen on 0.0.0.0.
  • Also: MCP integration, and connections to hosted providers such as OpenAI and Anthropic.
  • Suits: a desktop chat app with the source open, that can also act as a local server.

KoboldCpp

KoboldCpp calls itself free and open-source software for running GGUF models on your own computer, built on llama.cpp. It is AGPL-3.0 licensed, with bundled components under their own licences.

  • Platforms: a single executable for Windows, Linux (x64) and macOS (Apple Silicon); Intel Macs need a source build.
  • Model formats: GGUF.
  • API: an OpenAI-compatible API at localhost:5001/v1, KoboldAI’s own API, and endpoints compatible with Ollama.
  • GPU: --usecuda for Nvidia only, --usevulkan for AMD, Nvidia and Intel; a CPU-only mode as well.
  • Suits: chat, story writing and roleplay, which its bundled KoboldAI Lite web UI is built for. The README also lists image, video, speech and music generation.

The README warns that koboldcpp.com is a fake site and that the only official downloads are the GitHub releases page and koboldcpp.net.

TextGen (formerly text-generation-webui)

text-generation-webui now lives at github.com/oobabooga/textgen: the old repository URL answered with a permanent redirect when I requested it, and the README is headed “TextGen”. It is AGPL-3.0 licensed. The latest GitHub release I could see was v4.9, dated May 20, 2026, and the repository’s last push was August 17, 2026, while Ollama, llama.cpp, vLLM, LocalAI, Jan and KoboldCpp all had a push on 2026-11-17, so check it is keeping up with the models you want.

  • Platforms: portable builds for Linux, Windows and macOS with CUDA, Vulkan, ROCm and CPU-only options.
  • Model formats: GGUF through llama.cpp in the portable build. The full installation adds ExLlamaV3 and Transformers backends, which take multi-file Safetensors and EXL3 models.
  • OpenAI-compatible API: yes, OpenAI and Anthropic-compatible chat, completions and messages endpoints. On the manual install the README launches the server with --api.
  • Suits: someone who wants a chat UI and options to train a LoRA or swap between several backends in one program.

How to choose, and what to do first

I would work through it in this order:

  1. Check Ollama’s context and parallelism settings and, if the cloud is the issue, OLLAMA_NO_CLOUD=1. Those are one-line changes.
  2. If you want a GUI, install LM Studio or Jan next to Ollama and point it at a GGUF you already have. My LM Studio page ran both on ports 11434 and 1234 at once with two 3B models loaded.
  3. If you want the engine itself, run llama.cpp’s server. It loads GGUF files, either a local path or -hf from Hugging Face. Ollama’s docs say where models are stored but, in what I read, not that llama.cpp can load them from there. My own comparison did it: ollama show <model> --modelfile gave the path, and llama-server loaded the Llama 3.2 3B and Qwen 2.5 14B blobs from it, while the gpt-oss:20b blob was refused. That is one machine and three models, so try it on yours before you delete anything.
  4. If you serve many clients, run vLLM on Linux.

Before you swap anything, check what fits in your card with the LLM VRAM calculator and sanity-check expected speed with the LLM speed calculator; the VRAM-tier guide maps model sizes to cards. If you keep Ollama and run it in a container, the Ollama in Docker guide covers binding it to loopback.

Frequently asked questions

What is Ollama and is it safe to use?
Ollama is a runtime for open models. Its README describes it as a way to run open models locally or in its cloud, and lists llama.cpp as its supported backend. Its FAQ says the server binds 127.0.0.1 port 11434 by default and that Ollama does not see your prompts when you run locally; cloud-hosted models are processed on Ollama’s service, and the FAQ gives a setting (OLLAMA_NO_CLOUD=1) to disable the cloud features. I have not audited the code, and the docs do not describe a security audit, so that is the vendor’s description, not a safety verdict.
Is Ollama deprecated?
No. On 2026-11-17 the ollama/ollama repository on GitHub was not archived, carried the MIT licence, and had been updated that day. Its current README advertises local and cloud use of open models.
What is faster than Ollama?
It depends on which half of the work you time. On my RTX 5080, llama.cpp’s server decoded 2-6% faster than Ollama on the two dense models I tested (Ollama 0.32.1 against llama.cpp b10507, 4096 context, 256-token generations, medians of five), while Ollama prefilled 4-7% faster at about 420-token prompts; against LM Studio (Ollama 0.32.15, LM Studio 0.4.21, 4096 context, about 450-token prompts), Ollama decoded 5-11% faster on the dense models and LM Studio prefilled 5-14% faster. Those figures come from my two measured pages, one RTX 5080, and are not comparable across the two captures. I have not benchmarked vLLM, which the vLLM docs aim at concurrent serving rather than a single chat.
Is vLLM better than Ollama?
They target different jobs. vLLM’s README describes high-throughput serving with continuous batching and an OpenAI-compatible server; its docs list Linux as the supported OS, with no native Windows support, and call GGUF support highly experimental. Ollama runs natively on macOS, Windows and Linux and imports GGUF and Safetensors models. I have not measured vLLM on my RTX 5080, so I will not claim a speed winner; the site’s vLLM vs Ollama page works from the documentation.
Which is the best Ollama model to use?
That depends on your VRAM and the task. For coding on a 16 GB card, my August 2026 test on an RTX 5080 found gpt-oss:20b passed 8 of 8 tasks at a measured 178-190 tokens per second; the full test and the other four models are on the coding-model page, and the VRAM-tier guide maps models to card sizes.
Is Ollama better than GPT?
They are not the same kind of thing. Ollama is software that runs open models on your machine or in Ollama’s cloud; GPT is a family of models from OpenAI. Ollama’s docs show an OpenAI-compatible API, so a client written for OpenAI’s API can point at Ollama, but no source I opened ranks the models you would run through it against GPT, and I am not going to guess.

Sources and corrections

Last updated
Methodology
See our methodology for research and review standards. It draws on 41 cited sources, listed below, each checked against the original page on 2026-11-17. On 2026-11-17 I read each project’s own README, documentation or repository, fetched live with curl, and nothing on this page comes from a listicle or a forum. First-hand reads on this desktop (Windows 11 25H2, build 26200.9974, read with reg query on HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion; RTX 5080, driver 621.24 and 16303 MiB, read with nvidia-smi –query-gpu=name,driver_version,memory.total): where.exe found none of ollama, llama-server, lms, docker or vllm on PATH today, so I did not run any of these runtimes for this page. Every speed figure is quoted from my two earlier measured pages, with the conditions those pages give; I did not re-run them and did not benchmark vLLM, LocalAI, Jan, KoboldCpp or TextGen. Licences come from each repository’s LICENSE file (MIT for Ollama, llama.cpp and LocalAI; Apache 2.0 for vLLM and Jan; AGPL-3.0 for KoboldCpp and TextGen), cross-checked against the README where it states one and, for Ollama and vLLM, against GitHub’s repository metadata. Repository status (not archived, last push) was read from the GitHub API on 2026-11-17. The vLLM documentation I read is labelled by its own site as the latest developer preview docs. The LM Studio licence line is from its App Terms of Service dated August 23, 2026. Judgments about who each runtime suits are mine, built on those documented features.
Sources
Corrections
Spotted an error? Email a correction with the affected line and what you found. The draft includes this page’s address. Verified changes are recorded here. Contact details.

About the author

I am Lowell K. Wood IV. I research, write and maintain everything on TechFuelHQ from St. Louis County, Missouri.