How to Run a Local LLM on Windows 11: A Step-by-Step Guide

Quick start

To run your first local LLM on Windows 11:

1. Confirm that the PC has enough RAM and storage.
2. Install Ollama from its official website.
3. Open PowerShell or Windows Terminal.
4. Choose a model that matches the available memory.
5. Run ollama run model-name.
6. Wait for the download and enter your first prompt.
7. Use Task Manager to monitor RAM, CPU, and GPU activity.

An NPU is not required. A local LLM can run on a CPU, a compatible GPU, or a combination of CPU, GPU, and system memory. The main differences are speed, efficiency, and the model size that the computer can handle.

What is a local LLM?

A local LLM is a language model whose main files and computations run on the device itself. Once downloaded, many tasks can be completed without a permanent internet connection.

Potential advantages include:

• prompts and documents can remain on the computer;
• there is no per-token cloud charge for local generation;
• the model can remain available when the connection fails;
• users can choose among different models and quantizations;
• developers can connect applications to a local endpoint.

There are limitations. Local models can be slower, occupy many gigabytes, and sometimes provide weaker answers than larger cloud systems. Privacy also depends on the complete configuration: web search, extensions, telemetry, remote models, and external integrations may still transmit data.

Before installing anything

Choose a model that fits your PC

Your first model should leave headroom. Starting with the largest model that might barely fit often creates a slow and unstable first experience.

These are guidelines, not guarantees. Consumption changes with architecture, quantization, context length, runtime, and background applications. For a full calculation, see the related NextTechSearch guide, How to Calculate Local LLM RAM Requirements for Your PC.

Main method: install Ollama on Windows 11

Ollama handles three useful tasks: it downloads models, manages installed files, and provides a local interface for conversations and application integrations.

ollama --version

ollama run model-name

ollama run model:3b

Check whether the CPU or GPU is being used

While the model generates a response, open Task Manager and select Performance. Observe the CPU, Memory, and GPU panels. If needed, change one GPU graph to a compute engine and check both dedicated and shared GPU memory.

Ollama can also show active models:

ollama ps

The output may help indicate whether the model is loaded on the CPU, GPU, or across both resources. Exact reporting depends on the hardware, drivers, and runtime version.

Low activity on the default GPU graph does not necessarily indicate a failure. Task Manager may be showing the 3D engine rather than the engine used for compute, and only part of the model may have been offloaded to the GPU.

List, download, and remove models

List installed models:

ollama list

Download a model without immediately opening a chat:

ollama pull model-name

Remove a model you no longer need:

ollama rm model-name

Confirm the name carefully before removal. Deleting a model frees its storage, but you will need to download it again if you want to reuse it.

Control the context window

The context window is the amount of information a model can consider during a session. A larger context does not automatically produce better answers. It increases memory use and may reduce speed.

The maximum advertised by a model describes a supported capability, not necessarily the best setting for every PC. For a first test:

• use the default value;
• avoid pasting very long documents;
• monitor RAM consumption;
• increase context only for a real need;
• start a fresh conversation when old information is no longer useful.

On a 16 GB PC, an 8B model may work well with a moderate context and become unstable with a much larger one. The model file remains the same, but its KV cache grows with the tokens processed.

A practical 16 GB test plan

This sequence works well for an ordinary business laptop with 16 GB of RAM and integrated graphics:

1. Restart the PC to create a predictable baseline.
2. Close games, video editors, and unnecessary browser tabs.
3. Open Task Manager and record existing memory use.
4. Install Ollama from the official page.
5. Choose a quantized 3B model.
6. Ask three short questions.
7. Repeat one question with a 7B or 8B model.
8. Compare speed, quality, and memory consumption.
9. Keep only the model that offers the best balance for your needs.

This comparison is more useful than immediately loading the largest available model. A smaller model may respond faster, while a larger one may follow complex instructions better. The right choice depends on the task.

Common problems and fixes

Privacy: local does not automatically mean isolated

A downloaded model can process prompts without sending them to an external provider, but the complete configuration still matters. Before using confidential documents:

• confirm that the selected model is local rather than a cloud option;
• check whether the interface has web search or telemetry enabled;
• inspect extensions and external integrations;
• do not expose a local server to the internet without proper authentication;
• keep Windows, the runtime, and drivers updated;
• read the tool's licence and privacy policy.

localhost normally refers to the same computer, but network configuration can change the service's reach. Developers who make a local API available across a network must secure the port and understand origin, firewall, and authentication settings.

Alternative 1: llama.cpp

llama.cpp suits users who want direct control over GGUF files, context settings, GPU layers, and runtime parameters. Its official documentation includes installation routes for Windows, including winget where available:

winget install llama.cpp

The project includes command-line tools and a local server. A generic Windows command is:

llama-server.exe -m path\model.gguf -c 2048

This route is flexible, but the user must obtain the correct GGUF file and understand more execution options. It is a logical next step after learning the basics; compiling source code is not required for every installation, although acceleration support varies by package and hardware.

Alternative 2: Microsoft Foundry Local

Foundry Local is part of Microsoft's tooling for running models on-device and integrating them into applications. It is particularly relevant to developers using a CLI, SDK, local API, or Windows application project.

Check the current Microsoft documentation for supported Windows versions and hardware before installation. The CLI installation command is:

winget install Microsoft.FoundryLocal

After reopening the terminal, verify it with:

foundry --version

Foundry Local is not mandatory for everyone. Ollama is more direct for a reader who simply wants a first local conversation; Microsoft's ecosystem may be more relevant for Windows application development.

Compare the three routes

There is no universally best tool. Ollama lowers the initial complexity, llama.cpp offers control, and Foundry Local brings local inference closer to Windows application development.

Final checklist

• The runtime came from its official source.
• The model fits the available RAM.
• The SSD has enough free space.
• The first answer ran without excessive paging.
• CPU, RAM, and GPU use were observed in Task Manager.
• Context was not set higher than necessary.
• You know how to list and remove models.
• You confirmed whether the selected model is local or cloud-based.
• The local API was not exposed to the internet.
• You recorded the name, version, and size of the tested model.

Conclusion

Running an LLM on Windows 11 is now approachable: install a runtime, select a compatible model, and start a session. The most important decision happens before the first command—choosing a model that respects the limits of the machine's memory and hardware.

For most users with 16 GB of RAM, a quantized 3B–8B model is a sensible start. A 32 GB system offers more freedom with medium models, while 64 GB or more opens the door to larger workloads. Context length, quantization, and background applications still influence every result.

An NPU is not essential. It may accelerate compatible workloads and improve efficiency, but CPU and GPU execution remain valid. The practical goal is to find the right balance among quality, speed, memory, and privacy—not simply to select the largest parameter count.

After completing this guide, a useful next step is to compare the same model on CPU and GPU, measure tokens per second, and observe how different quantizations affect speed and output quality. Start small, test gradually, and understand the performance of your current computer before buying new hardware.

NTS View

Local AI is turning the personal computer into an inference platform rather than merely a terminal for remote services. That shift gives users more autonomy, but it also transfers decisions once made by cloud providers: which model to use, how much memory to allocate, what quantization is acceptable, and which external connections should be permitted.

That is precisely where Practical Guides have value. It is not enough to describe a PC as “AI-ready.” Readers need to see what it can run, how performance can be measured, and where the real limits are.

NTS Verification

This article separates official instructions from editorial recommendations. Commands, interfaces, and requirements can change, so they should be checked again against primary documentation immediately before publication.

Primary documentation supports the following points:

• Ollama provides an official Windows installer and documentation for local model use.
• llama.cpp supports GGUF model execution and includes command-line and local-server tools.
• Microsoft documents Foundry Local for on-device model execution and Windows application integration.
• GPU and NPU compatibility depends on the hardware, driver, model, runtime, and execution provider.