Running Local LLMs

What Are Local LLMs?

A Large Language Model (LLM) is the brain behind tools like ChatGPT. It reads text, thinks, and writes text back. Normally, these models run on giant computer servers somewhere on the internet — and you pay monthly fees to access them.

A local LLM is exactly the same kind of AI brain, but it runs on your own computer. No internet needed. No monthly bill. Your data never leaves your machine.

You install a small app, download a model file (like a big brain file), and then you can chat with AI completely offline. Think of it like owning a robot brain instead of renting one by the month.

Why Run AI On Your Own Machine?

Most people use AI through websites that send their questions to big servers online. That means your data travels over the internet and gets stored on someone else's computers. For most people, that's fine. But for builders, privacy lovers, and businesses, local AI opens up something powerful.

Here is why local matters:

• Privacy — Your prompts and data never leave your computer. Great for working with sensitive code, business info, or personal documents.
• Cost — Pay once for a good graphics card, then use AI unlimited times for free. No $20/month subscription.
• Offline use — Works on a plane, in a basement, or anywhere without internet.
• Customization — Swap models, tweak settings, and fine-tune the AI to your exact needs.

Getting Started With Ollama

Two tools make running local LLMs easy: Ollama and LM Studio. Ollama is the most popular and works on Mac, Windows, and Linux. LM Studio gives you a nice point-and-click interface. Both do the same job — let's look at Ollama.

Here's how it works in three steps:

That's it. No cloud accounts. No API keys. No internet. Just you and your local AI brain.

Your First Local AI Chat

Open your terminal (the command prompt on Windows, or Terminal app on Mac) and run these commands:

# Step 1: Download a model (Llama 3.2 — small, fast, smart)
$ ollama pull llama3.2

# Step 2: Chat with it right in your terminal
$ ollama run llama3.2

# Now type your question and press Enter. Try:
→ Explain local LLMs to a 5th grader

# Press Ctrl+D to exit

Want to use it from Python? Here's a tiny script:

import ollama

response = ollama.chat(
    model='llama3.2',
    messages=[
        {'role': 'user', 'content': 'What is a local LLM in one sentence?'}
    ]
)

print(response['message']['content'])

Knowledge Check

Test what you learned with this quick quiz.