Local AI
- Your prompt
- Your machine
- Response
Runs on your own computer or compatible local hardware. Supported inputs can stay within your environment.
AI Locally / An independent guide
AI, on
Your Terms.
Explore local AI, understand how it works, and discover what you can run on your own machine. Learn about models, hardware requirements, privacy, and the tools that make local AI possible.
Model inference inside your environment
01 / The basics
Same idea.
Different place to run.
Local AI runs inference on your computer or compatible local hardware. Cloud AI runs it remotely. Both can be useful; the right choice depends on the task and configuration.
Cloud AI
- Your prompt
- Remote service
- Response
Runs on remote infrastructure. Prompts or data may be sent to an external service.
Privacy depends on software configuration, integrations and data handling—not just where the model runs.
02 / Local AI explorer
Three pieces.
One working setup.
Capability starts with the model.
A model is a set of learned weights. Parameter count describes its scale—not whether it is right for your task. Quantization stores weights at lower precision, reducing memory use with possible quality trade-offs.
Compare task quality, model license, supported languages and context limits—not parameter count alone.
In practice
- Model weights
- Quantization
- Runtime memory
Illustrative example: a 7-billion-parameter model at 4 bits needs roughly 3.5 GB for weights alone, before metadata and runtime overhead. Its working memory will be higher.
03 / Your setup journey
Start with curiosity.
Build understanding.
Five steps, at your own pace. This guide explains the process; it does not install software or run a model.
On Windows, open Settings → System → About for installed RAM and Task Manager → Performance for memory and GPU details. On Mac, use Apple menu → About This Mac and Activity Monitor → Memory. On Linux, use the system information and system monitor apps. Note your operating system, free disk space, available RAM and GPU memory. These are manual checks, not a compatibility guarantee.
04 / Hardware & models
File size is not
working memory.
Model weights are only the starting point. Runtime memory also includes the context cache, computation buffers and software overhead. Your operating system and other applications need their own headroom.
- Weights
- + Context cache
- + Runtime overhead
Longer context often needs more memory. Actual use and speed depend on model architecture, runtime, precision and hardware.
Illustrative weight sizes / 7B parameters
- 16-bit weights
- ~14 GB
- 8-bit weights
- ~7 GB
- 4-bit weights
- ~3.5 GB
Simple parameter-count × bit-width arithmetic in decimal GB, before format overhead. Not total runtime memory or a compatibility recommendation. Lower precision can reduce memory use, but may affect quality; larger models generally require more resources.
05 / Practical experiments
What will you
try first?
Writing & brainstorming
Explore outlines, rewrite a paragraph or generate ideas with a chat-capable model and interface. Tone and factual quality vary; review the result.
Document summaries
Summarize supported text with a model and document parser. Long files may need chunking or retrieval; scans need OCR, and important details can be omitted.
Coding assistance
Try explanations and code suggestions with a code-capable model or compatible editor integration. Test outputs; editor plugins may still call external services.
Personal knowledge bases
Combine local embeddings, document indexing and retrieval to find relevant passages. Configure permissions and citations; retrieval does not guarantee accurate answers.
AI agent experiments
Connect a capable model to a controlled tool runner. Start with read-only tools and require approval for actions. Model support and tool-call reliability vary.
Offline AI workflows
Explore a downloaded model with locally available inputs and dependencies. Web search, hosted APIs and some plugins still require a connection.
06 / Playground preview
Follow a prompt
through the process.
A lightweight, prewritten demonstration of a possible workflow—not a model running on your device.
Illustrative options
These are conceptual configurations, not tested hardware recommendations.
Simulation · no model connected
Prompt
Summarize this note in one sentence: We will test a small model locally, review its answers, and check whether the workflow needs a network connection.
Example response appears after the simulation.
No software is installed, no hardware is inspected, and nothing is sent to a model. Options change the process labels, not the prewritten answer.
07 / Frequently asked questions
A few good questions.
A clearer starting point.
Keep exploring
Your Machine. Your Models.
Your Experiments.
Explore local AI at your own pace and discover what is possible with the hardware you already have.