ninja nerdingRun your AI at home: Ahmad Osman on local AI
Run your AI at home: Ahmad Osman on local AI
Ahmad Osman runs Osmantic, which makes an installer for running AI on your own machine. In a 96-minute talk with David Ondrej he argues most people should own their AI instead of renting it, and demos it on a laptop. The short of it: keep your data and most of the work on hardware you own, and ask the big cloud models only small, scrubbed questions.
Jump to the stack map, what to buy, or try it on what you already have.
The video
96 minutes, posted Oct 20, 2026, on David Ondrej's channel. The video has no chapters, so these are ours. Every timestamp on this page opens the video at that moment. Screen demos run at 19:30, 31:40, 65:40 and 85:40.
- 0:00 Cold open: whoever owns the hardware
- 1:38 Will everything get rebuilt? Polish still wins
- 5:16 Sponsor read: CodeRabbit (until 6:24)
- 6:33 Backdoored models and the supply chain
- 8:12 Sandbox your agents, cut your dependencies
- 10:32 Use tokens to build your own software
- 15:32 Ondrej's own product: Cloudroom (until 16:45)
- 16:46 The RTX Spark laptop and ODS
- 17:53 A local agent needs search too
- 19:30 Live: Portal builds a page; the dashboard
- 21:16 Mac Studio vs DGX Spark: a software problem
- 28:00 Markdown to Word, PDF and slides; taxes
- 30:58 Why the big labs won't tune a model for you
- 31:35 Live: a game built fully local
- 34:22 Hybrid: Opus 5.5 is best, local covers 95%
- 37:15 Intelligence keeps shrinking
- 40:03 Your data is your moat; the masking layer
- 44:47 What to buy with $10,000
- 48:54 NVIDIA, Mac, Intel: pick the software
- 53:26 Windows or Linux? Let the agent handle it
- 57:57 Touch grass; the work above the agents
- 1:00:53 Agents need new computers and better sandboxes
- 1:05:39 Live: a new game prompt, 72 tokens a second
- 1:08:24 Prices, idle GPUs, his 2023 bet
- 1:12:29 Privacy hygiene: trust but verify
- 1:15:45 Hybrid like an outside consultant
- 1:19:46 Ondrej's office: two DGX Sparks for private questions
- 1:20:57 $7,000 once vs a subscription
- 1:25:36 Live: extensions, Hermes, a sandbox block
- 1:27:49 Own it: the risk of a few gatekeepers
Ten signals
The whole talk in ten boxes. Each links to the part of this page that explains it.
- 011:28:55
If someone can take it away, you don't own it
A cloud model can be rate-limited, priced up, swapped for a weaker version or switched off. A model on your own machine can't. That's his whole case.
More - 0236:40
Local already covers most of the work
He says Claude Opus 5.5 is the best model out there. He also says the model on this laptop covers about 95% of everyday tasks. That 95% is his estimate, not a test.
More - 031:16:28
Hybrid: treat the cloud like a consultant
Keep your files, customers and money on your machine. Send the big cloud model one small, scrubbed question. Let the local model do the work with the answer.
More - 0421:16
Same chip, five times slower: software matters
On paper a Mac Studio should crush a DGX Spark. In his own test, two Sparks answered more than twice as fast. His point: hardware, model and software must be tuned together.
More - 0550:12
New to this? Buy for the software
If you don't know what a kernel is, pick NVIDIA. Millions of people already tune software for it. A Mac works but takes learning. Intel only if you'll tinker for days.
More - 0616:46
One laptop now runs a big model
The demo laptop has 128 GB of shared memory and runs a 180-billion-parameter model at 72 tokens a second, he says. This one costs about $7,000.
More - 0717:53
A local agent needs tools, not just a model
A friend's local agent couldn't change his new graphics card's lights: the model was older than the card. One search endpoint later, it worked.
More - 088:12
Lock it in a box
Run agents in a sandbox: a separate user with no admin rights, no internet for private work, and fewer outside libraries. His live demo hit his own sandbox, and that was the point.
More - 0928:00
Spend tokens on your own tools
He hasn't opened Word in about nine months. Agents turn files to Markdown and back, set up a new PC in hours, even tune his headphones.
More - 1041:15
Prices are going up. Don't borrow
A DGX Spark launched at $3,999 and now costs about $6,950. Still, in his words: please don't take a loan to buy a GPU. Not financial advice.
More
Words on this page
27 terms in plain words. Skip what you know.
- Local AI
- An AI model that runs on a computer you own, with no internet needed once it's set up. Nothing you type leaves the machine.
- Frontier model
- The newest, strongest models from the big labs, like Claude Opus 5.5 or GPT-6.1 Sol. They run only in the lab's cloud.
- Open-weights model
- A model whose files anyone can download and run, like Qwen or DeepSeek. The weights are the model's learned numbers.
- Parameters (B)
- The count of learned numbers in a model. 27B means 27 billion. More usually means smarter but needs more memory.
- Quantization
- Storing a model's numbers with fewer digits so it fits in less memory and runs faster, for a small loss in quality. Osman's fear: a cloud provider can do this to you quietly.
- NVFP4
- NVIDIA's 4-bit number format for its Blackwell chips. About 3.5 times smaller than the usual 16-bit format. The demo model runs in it.
- Token
- The small piece of text a model reads and writes, about three-quarters of a word. Speed is measured in tokens per second.
- Decode speed
- How fast the model writes its answer, in tokens per second. The demo: 72 for code, about 50 when thinking or calling tools.
- VRAM
- The graphics card's own memory. The model has to fit in it to run fast.
- Unified memory
- One pool of memory shared by the main chip and the graphics chip, as in Apple Silicon Macs and the DGX Spark. Lets a big model fit without a giant graphics card.
- Memory bandwidth
- How fast data moves out of memory. For writing tokens it's often the speed limit, which is why the Mac should have won on paper.
- Inference engine
- The program that loads a model and runs it, like llama.cpp, vLLM or Ollama. A tuned engine can make the same hardware several times faster.
- Kernel
- Here: a small, low-level piece of code that does the model's math on a specific chip. Good kernels are where the speed comes from.
- CUDA
- NVIDIA's software layer for running math on its chips. Most AI software is written for it first, which is why he says beginners should buy NVIDIA.
- Harness
- The program around a model that turns it into an agent: it sends your request, runs the tools the model asks for and loops. Claude Code, Pi and Hermes are harnesses.
- Sandbox
- A closed-off space where an agent can work without touching the rest of your computer: its own user account, its own files, no internet if you want.
- Air-gapped
- Fully cut off from the internet. He'd run sensitive work this way.
- Supply-chain attack
- Hiding something bad in a part you trust: a library, a graphics card, or a downloaded model trained to misbehave when it sees a secret phrase.
- Hybrid AI
- Using local models for most work and a cloud model only for the hard parts, without handing over your private data.
- Stateless question
- A request that carries no history and no private context, just the one well-scoped problem. The cloud model can't learn who you are from it.
- Masking proxy
- A program that sits between you and the cloud model, strips or swaps private details on the way out and puts them back on the way in.
- RAG and embeddings
- A way to let a model look things up in your own files. Embeddings turn text into numbers so similar passages can be found fast.
- Engram
- A design from DeepSeek research that keeps part of a model's knowledge in a big lookup table outside the graphics memory. The demo model uses the idea. Officially the table sits in RAM; Osman says theirs is on the SSD.
- WSL
- Windows Subsystem for Linux: real Linux running inside Windows. ODS uses it in the background so Windows users don't have to switch.
- SearXNG
- A free search engine you host yourself. It gives a local agent a way to look things up without a paid search API.
- Subsidized subscription
- A plan priced below what it costs the lab to serve you. His bet: those prices go up once the money runs out.
- RSI
- Recursive self-improvement: a model that helps train a better version of itself. He uses it to explain why labs are buying up chips.
What local AI is, and what it does
Local AI means the model runs on a computer you own. Your questions and files never leave it, nobody can change the model under you, and there's no monthly bill once you've bought the hardware. The trade: you buy the hardware, and the best models still live in the cloud.
Cloud AI
- Your prompt and files go to the lab's servers.
- The lab sets the price, the limits and which version you get.
- The strongest models, with nothing to buy up front.
- Needs the internet. Can be slowed, rationed or switched off.
Local AI
- Everything stays on your machine. Works on a plane.
- You pick the model and it never changes unless you change it.
- Smaller models, but he says good enough for most tasks.
- You pay for hardware once, plus electricity.
Three basic uses
- 01He does this himself29:48
Do your taxes without uploading your statements
Put bank and card statements in a folder. A local agent sorts every line, finds the deductions and writes a cover letter for your accountant as a normal Word file.
- 02He does this himself28:00
Skip Word, keep the template
The agent turns a .docx you were sent into Markdown, you edit plain text, and a small renderer puts it back into the exact same layout. Same for PDFs and slides.
- 03He does this himself13:24
Set up a new computer by asking
Change the taskbar, remove the ads, set your preferences. He says a week of fiddling took about three hours, mostly while he worked on another machine.
Three advanced uses
- 01Ondrej's setup1:19:46
A private box for legal and tax questions
Ondrej's office runs an uncensored DeepSeek fine-tune on two DGX Sparks. When a question is private, they switch the provider in their Pi harness to the office machine.
- 02Osman's design1:15:45
Ask the cloud without showing your business
Your local model writes one scoped question, a masking layer strips names and numbers, a frontier model answers, and the local model does the work. ODS has a basic version built in; his company sells more for businesses.
- 03Shown on screen31:35
Build apps and games with no cloud at all
On the laptop, ODS's own agent built a focus-timer app and a full fantasy dungeon game with levels and health bars. The game ran as a long goal of about 21 minutes, not one prompt.
Everything named
The demo ran on a Windows laptop using his company's installer, ODS. The setup group is read off the screen. Speeds and prices are his unless a line says otherwise.
His setup on the laptop (read from the screen)
Installer and control roomOsmantic · Apache-2.0
ODS
Osmantic Deployment System. One command installs everything, checks your hardware, picks a model for it and adds an app store of extensions. 3.0.0 on screen. Apache-2.0, except the bundled agent code, which has its own licence.
The built-in agentInside ODS
Portal
"Your local ODS owner agent". A chat that plans, asks you questions first, writes code and shows a preview. It has a cloud icon; he calls it the cloud at home. It has memory and a permissions switch.
Runs the modelllama.cpp, open source
llama-server
Listed in the ODS sidebar as the LLM inference app. He also says they built their own tuned engine for this laptop that beats vLLM; the benchmarks weren't shown.
Web search you hostOpen source
SearXNG
Gives the local agent a search engine, no paid API needed. This is the piece his friend's setup was missing.
Rules for tool callsODS extension
APE (Agent Policy Engine)
On screen: checks and logs every tool call an agent makes before it runs, with allow and deny rules. ODS also has Privacy Shield, which swaps private details for placeholders before a cloud call.
A second agent, one clickNous Research
Hermes Agent
Installed from the ODS library and wired to the same local model with no setup. On screen: v0.21.5, 18 tools and 53 skills.
Usage monitorODS app
Token Spy
In the sidebar. Not opened on camera.
Paid web searchBrave
Brave Search API
A core extension in the library, for when you want a commercial search index instead of SearXNG.
Hardware he compares
The demo machineNVIDIA + Microsoft, Windows
RTX Spark laptop (Asus ProArt)
NVIDIA's new chip for Windows laptops, announced June 1 with Microsoft. He calls it a DGX Spark in a laptop. This one has 128 GB of shared memory; the dashboard shows 96 GB for the GPU. RTX Spark laptops start at $2,599; the 128 GB ProArt P16 is about $7,000.
Small desktop AI boxNVIDIA, Linux
DGX Spark
Same idea on a desk, running Linux. Until recently he preferred graphics cards. Now he calls it a real option for running three or four agents. Launched at $3,999 in Oct 2025; the 128 GB model is about $6,950 now.
The paper winnerApple
Mac Studio M5 Ultra (256 GB)
About five times the memory speed of a DGX Spark, he says. In his Sept test, two Sparks reached the first word 2.4 times faster; the Mac wrote 6% faster after that. He agrees with DHH: Apple has a software problem, not a hardware one.
Top gaming cardNVIDIA
RTX 5090
The card in the friend's story. He says it went from about $2,000 a year ago to $5,000 or $6,000 last month, and can run four agents with a 27B model.
The old favouriteNVIDIA, 2020
RTX 3090
His basement rig grew to 14 of them (his blog shows 8 in 2024 and 14 by early 2025). He paid $500 to $600 each and says they go for $1,600 to $1,700 now. He keeps his idle for when smaller models get better.
Workstation cardNVIDIA
RTX Pro 6000
The yardstick for his predictions: Opus 4.5-level on one card this year (he says it came by March), Opus 5.5-level on two of them within eight months.
Good specs, young softwareIntel
Intel graphics cards
Fine on paper. Only for you if you'll spend days tuning one model on one card.
Models he names
The demo modelAlibaba Qwen, open weights
Qwen 3.8 Flash Next (NVFP4)
Released Aug 26 as a preview of Qwen 4. About 180B parameters counting its 51B lookup table, in 4-bit NVFP4. He says 72 tokens a second for code, about 50 while thinking or calling tools. Qwen's own licence, not Apache.
The small onesAlibaba Qwen
Qwen 3.5 27B and 4B
27B (out Feb 24) is the model in the friend's story. 4B runs on a 12 GB phone and, he says, beats GPT-4o, which once needed a data centre.
Open models getting strongerDeepSeek, open weights
DeepSeek V4 Pro and V4.1 Flash
His example of how fast open models improve: V4 Pro's full release on Aug 13, then V4.1 Flash on Sept 10.
The best model, he saysAnthropic, cloud only
Claude Opus 5.5
"Currently the best model out there." He thinks Anthropic cracked synthetic training data, and that open research will match it within a year.
The other frontierOpenAI, cloud only
GPT-6 Astra and GPT-6.1 Sol
Named with Opus 5.5 as the models you'd ask for advice through a masking layer.
Agents and engines
Harness, pointed at a local modelAnthropic
Claude Code
The friend ran it against a local Qwen with an environment variable. It works with local models when you give it an endpoint.
Harness Ondrej usesOpen source
Pi
Ondrej switches its provider to the office DGX Sparks for private questions. See our Pi page.
The engine to beatOpen source
vLLM
The well-known fast inference engine. He says their tuned engine for this laptop beats it.
Linux inside WindowsMicrosoft
WSL
ODS runs its Linux parts here so Windows users never have to switch systems.
Also named, not part of the setup
- CodeRabbit The sponsor. An AI code reviewer with a security scan. Not part of the local setup.5:16
- Cloudroom Ondrej's own product for cloud coding agents, plugged mid-video.15:32
- Project Discovery Security researchers who added a hidden trigger to a 7B Qwen model for about $8 of compute. Inside a coding agent, the trigger phrase made it send out the user's keys.6:33
- BlenderHis example of open source winning over time against paid 3D tools.4:06
- NVIDIA VeraA CPU NVIDIA built for the agent era. His point: computers were designed for human speed.1:00:53
- Grok CLI Ondrej's example of a cloud tool sending more than you expect. xAI's Grok Build was shown uploading the whole project folder it was started in.1:34:36
- OpenRouterNamed to say he's not attacking any one provider: if you don't own it end to end, you can't guarantee it.1:19:00
Model facts from the Qwen 3.8 Flash Next card, the NVFP4 explainer and DeepSeek's Engram repo. Prices from iTWire on RTX Spark laptops, reviews of the 128 GB ProArt P16, and ServeTheHome.
The local AI stack
Nine layers. Seven sit on your machine; two make an optional cloud lane for hard questions. Click a layer to read it, or pick a story and press Start to follow one job through the stack.
On your machine · top to bottom
Optional cloud lane
The cloud lane is for the hard questions only. In Osman's setup nothing private crosses into it: the mask decides what leaves.
Follow one job through the stack
A story Osman tells from March: a friend's brand-new graphics card, a local model, and one missing piece.
On your machine · layer 2 of 9 · you buy it once
Watch at 16:46The box
- RTX Spark laptop
- DGX Spark
- RTX 3090 / 5090
- Mac
The model has to fit in fast memory: a graphics card's VRAM, or a shared (unified) pool. More memory means a bigger, smarter model.
His setup: An Asus ProArt RTX Spark laptop, 128 GB shared memory, 96 GB for the GPU. At home: a basement rig of 14 RTX 3090s.
Start here
Use what you already have first. A recent laptop runs a 4B model. Buy only after you hit a wall.
Watch out
Buying on specs. The Mac wins on paper and still lost on time to first answer in his test, because the software isn't as tuned.
The layers and their order are FakeNerd's way of drawing what Osman describes. The lights and taxes stories are his; the hybrid story is his design in our words.
Prompts from the video
Three were typed on screen and are copied exactly. The rest he describes out loud, so the wording is ours. Each spoken one has a fill-in version: replace the [brackets].
The first test
19:38Typed into Portal on the laptop. It thought for 34 seconds, then asked what kind of page first (personal, landing, mini-app or surprise me).
As typed · read from the screen
Create a nice web page for me
Keep a long job going
1:06:05From the thread that built the game. The goal stopped once and asked for review; the next run worked for 20 minutes 57 seconds.
As typed · read from the screen
/goal Continue the plan for the game until complete
A game, live, in one sentence
1:05:50He typed it while talking. The finished game shown earlier came from a longer goal run, not this one line.
Said on camera · his words, lightly cleaned up
Can you make me a hack-and-slash, dungeon-crawler-style game themed on The Lord of the Rings? Make it cartoon: a hobbit slashing his way across Mordor.
FakeNerd fill-in
Make me a [genre] game themed on [setting]. Style: [cartoon / pixel / minimal]. The player is [who] trying to [goal]. Arrow keys to move, space to act. Show health, levels and a choice of upgrades on level up. Keep it to one page I can open in a browser.
Fix a setting without the app
11:43He says it took about 20 minutes in the background and beat opening the app and moving sliders.
Said on camera · our wording
I just connected my headphones over Bluetooth. Figure out how to get into their settings and set the equalizer to match this screenshot.
FakeNerd fill-in
I just connected [device] over [Bluetooth / USB]. Find out how to read and change its settings from this computer. Then apply these settings: [attach a screenshot or list them]. Tell me what you changed and how to undo it.
Set up a new computer
13:24He did this on the RTX Spark with ODS and a local model. About three hours, mostly unattended.
Said on camera · our wording
Change the taskbar, change the settings, remove this bar, remove these ads.
FakeNerd fill-in
This is a fresh [Windows / Mac] machine. Set it up the way I like: [taskbar position, theme, startup apps to remove, ads and suggestions off, default browser]. Ask before anything that needs admin rights. Give me a list of every change.
Turn your taste into a skill
14:30His point: once it's a skill, every new machine or tool gets set up the same way.
Said on camera · our wording
This is what I like. Find the taste in the customizations I made, then create a skill based on it.
FakeNerd fill-in
Look at the changes I've made on this machine: [list or point to settings]. Describe the taste behind them in five short rules (for example minimal, keyboard-first). Save those rules as a skill called [name] so you apply them next time without asking.
Word files without Word
28:00He says he hasn't used Microsoft Word in eight or nine months because of this.
Said on camera · our wording
Reverse-engineer this .docx into a Python program that converts it to Markdown, and a renderer that turns the Markdown back into the same template with the same formatting.
FakeNerd fill-in
Here is a [.docx / .pptx / PDF] I was sent: [file]. Write a small script that turns it into Markdown I can edit, and a second script that renders my edited Markdown back into the same template and formatting. Test both on this file and show me the round trip.
Taxes, locally
29:48Run this only on a local model, offline. That's the point of it.
Said on camera · our wording
Go through my bank statements and credit card statements, find out what each charge was for, and prepare everything for my tax accountant, with a cover letter as a normal Word document.
FakeNerd fill-in
In [folder] are my [bank / card] statements for [year]. Run offline. Categorise every charge, flag likely deductions for [country] with a reason, and write a one-page cover letter for my accountant as a .docx. Don't send anything anywhere. List anything you weren't sure about.
His posts named in the video
- Renderers instead of Word“Tell your agent to write HTML / DOCX / PDF renderers for your MD files”Oct 12, 2026
- Cut your libraries“Libraries are a supply chain disaster. Point an agent at a library / API and tell it to extract / rewrite what it needs to get the job done.”Oct 13, 2026
- Unified memory, reconsidered“One thing I discounted was that Unified Memory might actually play very well for small and efficient models”Oct 7, 2026
- Touch grass“If you wake up in the middle of the night and your first instinct is to check on your agents that's a sign to use the weekend to touch some grass”Oct 10, 2026
- No loans“please don't take a loan to buy a gpu”Oct 14, 2026
What to buy, and when not to
His rule: buy for the software, not the specs. Pick your situation to see his answer. Not financial advice; he says so twice.
His answer
NVIDIA
The most mature software. Millions of people already tune it, and most tools run on it first. "If you have no idea what a kernel is, just go with NVIDIA."
Watch at 50:12Specs vs software: his own Mac test
In September he ran DeepSeek V4 Flash on a Mac Studio M5 Ultra and on two DGX Sparks. His post:
Time to the first word
Sparks 1.3 s, Mac 2.9 s. The Sparks were 2.4 times faster.
Writing speed after that
The Mac was 6% faster.
On paper the Mac has about five times the memory speed. In the video he sums it up as the Spark winning; the post shows it's the first-word wait where it wins. His point stands: tuned software beats raw specs. 21:16
How much memory, which model: ODS's default picks
From the ODS README, checked Oct 21. A sensible starting size for any app, not only ODS. The 180B model in the demo is their tuned build for the 128 GB laptop.
Windows, Mac or Linux?
Windows is fine
54:09ODS runs on Windows first, with Linux in the background through WSL. He won't tell a cousin to switch systems to get local AI.
Linux for builders
1:01:31If you're building agent infrastructure and need to get close to the hardware, Linux is still the fastest place to work.
Soon the agent won't care
54:51Ondrej's point: if agents do the clicking, the operating system is more for them than for you. Osman agrees it should disappear for the user.
Hybrid: use the cloud like an outside consultant
Ondrej's picture, and Osman agrees: you'd hire a consultant for one topic, but you wouldn't hand them every employee's medical records. 1:16:28
Before: cloud for everything
- You paste the whole document, codebase or spreadsheet.
- The cloud model sees your clients, numbers and plans.
- It does the work, and the terms decide what's kept.
After: hybrid
- Your local model finds the one part that needs outside help.
- It writes a scoped question with no names, numbers or history.
- The frontier model answers that question only.
- The local model applies the answer to the real files and checks it.
Why he cares
- Labs want data to train the next model.
- Some tools' fine print lets them use what you send.
- Companies can be pressured to hand data over.
- He says some CEOs keep a local model in the office just for their email, because they won't trust it anywhere else. 1:18:27
Do it with software
ODS includes Privacy Shield, a proxy that swaps emails, phone and card numbers, IP addresses and API keys for placeholders before a request reaches a cloud model. He describes a fuller version for business, ODS Enterprise; we found no public docs for it.
FakeNerd starter · the mask, by hand
You are my local assistant. I need advice from a stronger cloud model, but nothing private may leave this machine. 1. Read the problem below and find the smallest part that needs outside help. 2. Write ONE question for the cloud model about only that part. - No company, client, product or person names. Use [Company], [Client], [Product]. - No numbers that identify us: prices, revenue, account IDs, dates. - No file paths, repo names, internal URLs or keys. - Code only if it's generic. Rename anything that hints at the business. 3. Show me the question and a list of every detail you removed. Wait for my OK. 4. When I paste the answer back, apply it to the real files here and tell me what changed. The problem: [describe it]
Run this in your local agent, not the cloud one. Osman describes the idea; this prompt is ours.
Lock it in a box
Local doesn't mean safe by itself. A downloaded model is one more part of your supply chain, and an agent can change your computer. His answer: sandbox everything. 8:12
- 011:31:08
Give agents their own user
A standard account with no admin rights. His live task tried something it wasn't allowed to do and was blocked.
- 027:37
Cut the internet for private work
"Have the entire thing air-gapped and sandboxed." A sandbox that has no network module can't send anything out.
- 036:33
Download models from known publishers
A backdoored fine-tune looks normal until it sees its trigger. Researchers built one that sent a coding agent's keys out. Treat models like any other part of your supply chain.
- 049:24
Fewer dependencies
Every library that updates itself is an attack surface. Tokens are cheap: have the agent write the small part you need instead of importing a package.
- 051:26:00
Check every tool call
A policy engine (APE in ODS) reads each command before it runs and applies allow and deny rules.
- 061:34:36
Read what a cloud tool sends
A cloud coding CLI was caught uploading the whole folder it started in. Open source lets you check what a tool sends.
What happened live · 91:08
He asked the agent to clone and set up another coding tool. It ran 53 actions, then a command was blocked and it stopped: the Windows user it runs as has no admin rights. “At least the sandboxing works.” The task didn't finish on camera. 1:31:08
The money
His argument: a top subscription costs about as much each year as a laptop that runs local AI forever. And subscriptions are priced below cost today, so they'll rise. 1:20:57
What the math leaves out
- Electricity, and the time to set it up.
- You'll still want a frontier model for the hardest work.
- Most people pay $20 a month, not $500. The math is for heavy users.
What it leaves in your favour
- No rate limits, no quiet downgrades, works offline.
- The same hardware runs better models every few months.
- Hardware has held its value lately: 3090s and Sparks cost more now than new.
His predictions, with dates
He asks to be held to these. So here they are, with when to check.
- Claude Code + Opus 4.5 quality running at home on a single RTX Pro 6000 before the end of the year.His post, Jan 16, 202647:31 PostHe says it came true by March. Qwen's own numbers have Qwen3.6-27B (April) beating Opus 4.5 on two of three tests, behind on one.
- Opus 5.5 running on two RTX Pro 6000s within less than eight months.His post, Oct 13, 202636:06 PostOpen. Check by June 2027. In the video he says one card; his post says two.
- The equivalent of Opus 5.5 running on this RTX Spark laptop within a year.Said in this video1:24:30Open. "Come back and check on this in a year." Check Oct 2027.
- The local model covers about 95% of tasks today; maybe 98% in six months.Osman, then Ondrej36:40Estimate. No test shown.
- Every three and a half months, models get about half the size for the same intelligence.Osman37:46His trend line. No source shown.
- Local AI becomes the default for most people; hardware prices stay high for at least two years.Osman1:09:41Opinion. Not financial advice, he says.
Try it, smallest version first
- 01
See what your machine can run
freeAsk your agent to read your hardware and recommend a model size. It installs nothing. Works in any agent chat on your own computer.
Step 1 · in agent chat
Look at this computer without installing anything: graphics card and its memory (VRAM), unified memory if it has it, system RAM, free disk space, operating system. Tell me: 1. The largest open model that fits with room for a 32k-token context. 2. A smaller model that will feel fast. 3. One app to run them with, for a beginner. Plain words, short.
- 02
Install one local app
freePick a beginner app that chooses the engine for you, such as Ollama or LM Studio. Download it from the app's own site.
Ollama - 03
Or: the full stack in one command (his ODS)
freeODS adds search, agents, a policy engine and Privacy Shield on top. Linux and Mac need Docker; Windows uses a different PowerShell block from the README that sets up WSL2, run in a normal (not admin) window. Needs about 40 GB free. Read the script before you run it.
ODS README (Windows steps)Step 3 · Linux or macOS terminal · from the ODS README
curl -fsSL https://install.osmantic.com/ods.sh | bash
- 04
Run a small model and ask something private
freeStart with a 4B-class model. Turn off Wi-Fi and ask it a question you'd never paste into a cloud chat. That's the feeling he's selling.
- 05
Give it search
free or a small API feeThe lesson of the RGB lights: a model can't know what's newer than it. Add a search tool, either SearXNG on your own machine or a search API key.
SearXNG docs - 06
Make a fence before you give it real work
freeCreate a standard (non-admin) user for agent work and keep private files in one folder. Leave the network off for private jobs.
- 07
Try one real job offline
freeThe Word round trip or your own statements. Use the fill-in prompts above.
The prompts - 08
Go hybrid by hand
your current subscriptionWhen the local model is stuck, have it write a scrubbed question with the starter below, check it, and only then paste it into your cloud model.
The mask starter
The install line is from the ODS README. The step order and the prompts are FakeNerd's.
Next to the others
The other David Ondrej interviews on this site, and pages that make the opposite bet.
- Pi workflow Ondrej with Armin Ronacher on Pi, the small agent Ondrej points at his office DGX Sparks.Ondrej interview
- Rauch's agentic workflow Ondrej with Vercel's CEO: the cloud-first view, with sandboxes in the cloud.Ondrej interview
- Ondrej's mini software factory Ondrej's own agent loops, and the subscription-value math this video pushes back on.Ondrej
- Theo on token-maxing Spending lots of cloud tokens on purpose. The opposite bet.Cost
- Agent cloaking A desktop app that masks private data before it reaches a cloud model: the hybrid idea as a product.Masking
FakeNerd's take
The ownership case is the strong part
Prices, limits, quiet model swaps and terms of service are all outside your control in the cloud. A model on your own machine removes those. You don't need his predictions to agree with that.
The numbers are claims
95% coverage, 72 tokens a second, beating vLLM, the Mac test: none were measured on screen, and the benchmarks were promised for later. Treat them as his estimates until they're published.
The demo was honest, which helps
The game took a 21-minute goal run, the focus app flagged its own unchecked preview, and the live task stopped at his sandbox. That's what local agents look like today: real, and not one-shot.
Don't buy yet. Start on what you have
He sells ODS and its enterprise layer, and hardware costs are rising. Try a small model and the hybrid habit first. Spend $7,000 only once you've hit a wall you can name.
Attribution
- David Ondrej's interview with Ahmad Osman, “Stop paying for AI, run everything locally (here's how)”, posted Oct 20, 2026. The ideas, stories, numbers and predictions are theirs, in our words. YouTube had no captions yet, so we transcribed the audio locally and fixed names against the screen. App names, model name, memory figures and the typed prompts are read off the screen.
- Sponsored by CodeRabbit, which isn't part of the setup. Ondrej also promotes his own product, Cloudroom, in the video.
- Ahmad Osman is founder and CEO of Osmantic, which makes ODS and sells ODS Enterprise and private-AI consulting. The demo laptop runs his company's software. ODS is Apache-2.0 except the bundled agent code, which has its own licence. More from him at ahmadosman.com. An earlier talk with Ondrej, on building a home AI server, is here; this page covers only the new one.
- His speed figures (72 and 50 tokens a second), the claim that their engine beats vLLM, the 95% figure, the RTX 5090 and 3090 prices and the CEO story are unverified; the benchmarks were promised for later. Prices, release dates, the Mac test and the quoted posts were checked on Oct 21, 2026; sources are in our research notes.
- The stack map, the three walkthroughs, the buying picker wording, every fill-in prompt, the mask starter, the setup order and the takes are FakeNerd's. Not sponsored by anyone on this page.