we ran out token!
Artificial Intelligence is no longer a cloud-only technology. Today, developers can run powerful AI models directly on their laptops, workstations, and developer machines without sending data to external services.
Local AI brings three major benefits:
- Privacy: Source code and business data stay on your device.
- Lower cost: No per-token cloud charges.
- Offline development: Continue working without Internet access.
Microsoft has significantly improved local AI support across Visual Studio 2026, Visual Studio Code, GitHub Copilot, Foundry Toolkit, Foundry Local, Windows AI, and ONNX Runtime. Microsoft officially supports multiple approaches for running models locally, including Ollama, Foundry Local, ONNX models, and Windows AI APIs. [code.visua…studio.com], [developer….rosoft.com], [learn.microsoft.com]
This article explains how to get started step by step.
Understanding the Local AI Architecture
Before installing anything, understand the architecture:
Microsoft’s current tooling allows local models to appear alongside cloud-hosted models inside the development experience. [code.visua…studio.com], [code.visua…studio.com], [code.visua…studio.com]
Option 1: Run Local AI Using Ollama
This is the easiest approach.
Step 1. Install Ollama
Download and install Ollama:
Verify installation:
ollama –version
Step 2. Download a Model
Example:
ollama pull phi3:mini
Microsoft documentation uses Phi-3 Mini as a local AI example. [learn.microsoft.com]
You can also install:
ollama pull qwen3
ollama pull deepseek-r1
ollama pull gemma3
Step 3. Run the Model
ollama run phi3:mini
You now have a local LLM running on:
Option 2: Use Local AI in Visual Studio Code
Microsoft has evolved AI Toolkit into Foundry Toolkit.
Foundry Toolkit lets developers:
- Download local models
- Test prompts
- Compare models
- Build AI agents
- Connect local and cloud AI resources
[code.visua…studio.com], [learn.microsoft.com]
Step 1. Install Visual Studio Code
Download latest VS Code:
Step 2. Install Foundry Toolkit Extension
Open:
Extensions
Search:
Foundry Toolkit
Install.
Microsoft describes Foundry Toolkit as the primary AI development extension for VS Code. [code.visua…studio.com], [learn.microsoft.com]
Step 3. Connect to Local Models
In Foundry Toolkit:
Model Catalog
→ Add Model
→ Ollama
or
Add Custom Endpoint
Point to:
for Ollama.
Supported local runtimes include:
- Ollama
- Foundry Local
- ONNX
- Windows AI APIs
[code.visua…studio.com], [learn.microsoft.com]
Step 4. Open Playground
Foundry Toolkit includes a local playground.
You can:
- Chat with models
- Test prompts
- Evaluate responses
- Compare models
without writing code.
[code.visua…studio.com], [adolfi.dev]
Using Local AI with GitHub Copilot in VS Code
Many developers assume GitHub Copilot requires Microsoft-hosted models.
That is no longer true.
VS Code can now connect chat experiences to:
- Ollama
- Foundry Local
- Azure AI Foundry
- OpenAI-compatible endpoints
[code.visua…studio.com], [devblogs.m…rosoft.com]
Step 1. Install GitHub Copilot
run GitHub Copilot.
Step 2. Add Local Model
Open Copilot Chat.
Choose:
Model Picker
Then:
Add Model
Select:
Foundry Local
or
Ollama
Microsoft documents support for local models inside the model picker experience. [code.visua…studio.com], [devblogs.m…rosoft.com]
Step 3. Start Coding
Example prompt:
Create a .NET Web API using Minimal APIs
and add JWT authentication.
Everything can be processed by your local model.
Source code never leaves your machine.
Using Local AI in Visual Studio 2026
Visual Studio 2026 introduces Bring Your Own Model (BYOM).
Developers can connect local AI models to Visual Studio’s Agent experience.
Supported providers include:
- Microsoft Foundry
- OpenAI
- Anthropic
- Ollama
Step 1. Install Visual Studio 2026
Install:
- ASP.NET workload
- Python workload
- AI Development tools
Step 2. Start Ollama
Example:
ollama run qwen3
Step 3. Open Agent Mode
Visual Studio Agent Mode provides:
- AI-assisted development
- File analysis
- Code generation
- Refactoring
Step 4. Connect Local Endpoint
Configure:
Provider = Ollama
Endpoint = localhost
Model = qwen3
Microsoft’s BYOM preview supports local Ollama endpoints within Agent Mode. [visualstud…gazine.com]
Building .NET Applications with Local AI
Microsoft provides official support for local AI development using:
- Microsoft.Extensions.AI
- Ollama
- Foundry Local
[learn.microsoft.com], [learn.microsoft.com]
Create Project
dotnet new console
Add package:
dotnet add package OllamaSharp
Sample Code
using Microsoft.Extensions.AI;
using OllamaSharp;
IChatClient client =
new OllamaApiClient(
new Uri(“http://localhost:11434“),
“phi3:mini”);
This connects directly to a local model. [learn.microsoft.com]
Building Python Applications with Local AI
Install:
pip install ollama
Example:
from ollama import chat
response = chat(
model=”phi3″,
messages=[
{“role”: “user”,
“content”: “Explain dependency injection”}
]
)
print(response)
The application runs completely on local hardware.
Using ONNX Models in Visual Studio
For production scenarios, many enterprises prefer ONNX models.
Advantages:
- Smaller deployment size
- Hardware acceleration
- CPU, GPU, and NPU support
- Windows-native optimization
Microsoft recommends ONNX Runtime and Windows ML for local model execution. [learn.microsoft.com], [learn.microsoft.com], [onnxruntime.ai]
Install Package
dotnet add package Microsoft.ML.OnnxRuntime
[c-sharpcorner.com], [onnxruntime.ai]
Load Model
using Microsoft.ML.OnnxRuntime;
var session =
new InferenceSession(
“model.onnx”);
Recommended Models for Developers
| Model | Memory | Best For |
|---|---|---|
| Phi-3 Mini | 4-8 GB | .NET development |
| Phi-4 | 8-16 GB | reasoning |
| Qwen3 | 8-16 GB | code generation |
| Gemma 3 | 8-16 GB | chat and coding |
| DeepSeek-R1 Distilled | 16+ GB | advanced reasoning |
Microsoft specifically highlights Phi family models and Qwen variants for local AI development scenarios. [devblogs.m…rosoft.com], [learn.microsoft.com]
Best Practices
For Laptops
Use:
- Phi-3 Mini
- Phi-4 Mini
8 GB to 16 GB RAM is often sufficient.
For Developer Workstations
Use:
- Qwen3
- DeepSeek Distilled
- Larger Phi variants
Recommended:
- 64 GB RAM
- NVIDIA RTX GPU
For Enterprises
Use:
- Foundry Local
- ONNX Runtime
- Windows AI APIs
This provides better governance and privacy controls. [developer….rosoft.com], [learn.microsoft.com]
Conclusion
Local AI is rapidly becoming a first-class development experience within Microsoft’s ecosystem. With Visual Studio 2026, Visual Studio Code, Foundry Toolkit, Foundry Local, GitHub Copilot, ONNX Runtime, and Windows AI, developers can build intelligent applications while keeping their source code and data entirely on-device.
For most developers, the recommended path is:
- Install Ollama
- Download Phi or Qwen models
- Install Foundry Toolkit in VS Code
- Connect GitHub Copilot to local models
- Use Visual Studio 2026 BYOM for agent-based development
- Move production workloads to ONNX Runtime or Foundry Local
This approach delivers privacy, lower costs, faster iteration, and complete control over your AI development environment.
References
- Microsoft Learn – Local AI with .NET: [learn.microsoft.com]
- Microsoft Foundry Local SDK: [learn.microsoft.com]
- VS Code Foundry Toolkit Documentation: [code.visua…studio.com], [learn.microsoft.com]
- VS Code Model Management: [code.visua…studio.com]
- Windows Local AI Developer Hub: [developer….rosoft.com]
- Visual Studio 2026 BYOM Preview: [visualstud…gazine.com]
- ONNX Runtime Documentation: [onnxruntime.ai], [learn.microsoft.com], [learn.microsoft.com]
- Microsoft Foundry Blog – Local Models with GitHub Copilot: [devblogs.m…rosoft.com]
- Foundry Toolkit Overview: [learn.microsoft.com], [code.visua…studio.com]
Leave a Reply