Local AI: Run LLMs with Ollama and llama.cpp

Local AI means running language models on hardware you control instead of sending every prompt to a cloud service. This section covers the tools that make that practical, such as Ollama, llama.cpp, Open WebUI and vLLM, and the questions that come with them: which model fits your memory, whether a CPU-only server is enough, how to run everything in Docker, and how to keep the setup private and up to date.

These articles are documentation and community roundups, not lab reviews. We do not download large models on our own server, so speed and hardware figures come from official pages or from user reports that we quote and link. Each article separates official facts, what users report, our own take and what we did not test, so you can judge every claim by its source.

If you are new to containers, the Docker section explains the basics used in these setups. For announcements of new open-weight models and the companies behind them, see the AI News section.

What this section covers

  • Ollama setup on Linux and in Docker, CPU-only and with a GPU
  • llama.cpp, model formats and quantization in plain language
  • Open WebUI and other interfaces for local models
  • Choosing a model for your RAM and VRAM
  • Privacy, updates and running local models as a service

Start here

Latest Local AI posts

Browse all Local AI posts

How we research, test and update articles: our editorial policy.