next level nerdingautoresearch: the lab that runs while you sleep
autoresearch: the lab that runs while you sleep
Andrej Karpathy gave a coding agent a small, real LLM training setup and told it to keep experimenting until he stopped it. The agent changes the code, trains for five minutes, checks one number, keeps the change or throws it away, and goes again. About a hundred experiments a night. Your job isn't the Python. It's the Markdown file that tells the agent how to be a researcher.
Start with what it is, or skip to the loop or setting it up.
Ten signals
The whole repo in ten boxes. Each links to the part of this page that explains it.
- 01
An agent runs experiments while you sleep
It changes the training code, trains for five minutes, checks one number, keeps the change or throws it away, and goes again. About a hundred experiments a night.
How - 02
The loop spread, not the repo
Within days people pointed the same loop at marketing copy, cold emails and Shopify's template code. Only what gets edited and how it's scored changed.
How - 03
You wake up to a full record
Every try gets a row in results.tsv, kept or not. The tip of the Git branch is the best code found so far.
How - 04
It found real gains
On full nanochat it tried roughly 700 changes over about two days and kept about 20. Together they cut Time to GPT-2 by about 11%.
How - 05
Three files, three owners
The agent edits train.py. You edit program.md. Nobody edits prepare.py, because it holds the scoring.
How - 06
One number decides: keep or reset
The change is committed before the run. If val_bpb goes down it stays. If not, Git resets it, so the branch can only get better.
How - 07
It never stops to ask
program.md tells the agent not to pause and ask whether to keep going. The rules are plain English in that one file, with no sandbox enforcing them.
How - 08
Fair by design
Every run gets the same five minutes and the same scoring, so a bigger model and a smaller one compete on equal terms.
How - 09
One NVIDIA GPU, and a box you can lose
Tested on an H100. The agent runs all night with permissions off, so use a rented GPU or a fresh machine. Forks exist for Mac, Windows and AMD.
How - 10
Your job is program.md
You don't write the Python. You improve the Markdown file that tells the agent how to be a researcher.
How
Words on this page
Fourteen terms, in plain words. Skip this if you already train models and use Git.
- Repo and fork
- A repo is a project's code and its history, kept in Git, usually on GitHub. A fork is someone's copy of a repo that they changed, like the versions adapted for Macs and AMD cards.
- LLM
- Large language model, the kind of AI behind ChatGPT. This repo trains a small one: about 50 million parameters in the example run.
- GPU and H100
- A GPU is the graphics chip that does the heavy math of training AI. The repo needs one NVIDIA GPU and was tested on an H100, a high-end NVIDIA card.
- Training run
- Feeding the model text so it gets better at predicting what comes next. Here every run gets exactly five minutes of training time, then a fixed test scores it.
- Hyperparameters
- The settings of a training run that a person picks instead of the model learning them: learning rate, batch size, model depth and so on. Many of the agent's kept changes are tweaks to these.
- Token and tokenizer
- A token is a small chunk of text, often part of a word. The tokenizer is what splits text into tokens; prepare.py trains one with 8,192 of them.
- Validation data
- Text kept aside that the model never trains on, used only for scoring. Here it is one fixed chunk of the dataset (a shard), about 21 million tokens, the same for every run.
- VRAM
- The GPU's own memory. The agent logs peak use for every run. The example run peaks at about 44 GB, and using more is allowed if the gain is real.
- Commit, branch, reset
- Git terms. A commit is a saved snapshot of the code with a short ID. A branch is a line of commits, here autoresearch/<tag>. A reset throws the latest commit away and goes back to the one before.
- Baseline
- The first run of every session, with train.py untouched. Its score is the number every later experiment has to beat on your machine.
- Ratchet
- The keep-or-reset rule. The branch only moves forward when the score improves, so whenever you stop, its tip is the best code found so far.
- Context window
- How much text an agent can hold in its working memory at once. That's why training output goes to run.log and the agent pulls only two lines out of it.
- Noise floor
- How much the score moves between identical runs by chance alone. A gain smaller than that, like the seed change at the end of the chart, may be luck.
- SSH and tmux
- SSH logs you into a remote machine, like a rented GPU, from your own terminal. tmux keeps the agent's session running there after you disconnect.
What it is
A small Git repo, not a product. You point your own coding agent at it, on a machine with a GPU, and the agent runs training experiments by itself until you stop it.
- 3files that matterOne the agent edits, one you edit, one nobody edits.
- 5 minper experimentA fixed training budget on the wall clock, whatever the agent changes.
- 1number to beatval_bpb, validation bits per byte. Lower is better.
- ~100experiments a nightAbout 12 an hour, for as long as you leave it running.
The idea, in his words
“Give an AI agent a small but real LLM training setup and let it experiment autonomously overnight. It modifies the code, trains for 5 minutes, checks if the result improved, keeps or discards, and repeats. You wake up in the morning to a log of experiments and (hopefully) a better model.”
The training code is a cut-down, single-GPU version of nanochat, his small ChatGPT-style model. The flip is who edits what. A researcher normally edits the Python. Here the agent edits the Python, and you edit program.md, the instructions the agent follows. He calls that file a “super lightweight ‘skill’” and the thing you're tuning the “research org code.”
What it isn't
- Not a hosted product or an app. It's a small Git repo you clone onto a machine with a GPU.
- Not an agent. You bring your own: Claude Code, Codex, whatever you use. The repo is what the agent works on.
- Not a general research assistant. It tunes one training script against one metric, and nothing else.
- Not a paper writer. The output is a Git branch and a table of results.
Released March 2026. About 97,000 GitHub stars and 13,500 forks as of Oct 1, 2026. MIT licensed, per the README.
How people are using it
The repo itself only trains one small AI model. What spread is the loop. Within days, people had copied it and pointed it at their own problem: try a change, score it, keep it or undo it, repeat. Six real ones, from the people who ran them. The numbers are theirs.
Three simple ones · the thing being improved is words
- Ole Lehmann · writes about AI workflows
Marketing copy that stops being vague
He has a saved set of instructions that makes Claude write landing-page copy. It was great about half the time. He let an agent rewrite those instructions, one small change per round.
The score: Five yes-or-no questions, like “Does the headline include a specific number or result?”
What happened: The copy went from passing 56% of the checks to 92% in four rounds. Three changes kept, one undone.
Mar 17, 2026His write-up
- Brandon Pizzacalla · non-technical software operator
Cold emails that get more replies
An agent writes a few versions of an outreach email, sends them, and watches which one gets answered. The winner becomes the one to beat. When nothing beats it, the agent stops by itself.
The score: Reply rate on real emails to real people.
What happened: He shared how the system works, not numbers. Rounds are slow here, because the score only arrives when people write back.
Apr 1, 2026His post
- Lotte Verheyden · Langfuse
A better how-to guide for an AI helper
Langfuse has a written guide its AI assistant follows to do one fiddly chore. They let an agent rewrite the guide and tried every version on six practice projects.
The score: A blend of three things: was the job done right, was all of it done, and how few steps it took.
What happened: 0.35 to 0.82 in 14 experiments. But the biggest jump came from deleting the step where the assistant asks permission first. Nobody answers in a test, so asking looked like failing. They kept the good changes and threw that one out.
Mar 24, 2026Langfuse's write-up
Three advanced ones · the thing being improved is software
- Tobi Lütke · CEO of Shopify
A smaller AI model that beat the bigger one
Before bed, he told his agent to read Karpathy's repo and build the same loop for a small model inside his own search tool. “I'm not a ML researcher of course.”
The score: A quality score for the model's answers, plus speed.
What happened: Eight hours and 37 experiments later, a model half the size scored 19% higher than the one it replaced.
Mar 8, 2026His post
- Tobi Lütke · CEO of Shopify
The code behind Shopify storefronts, 53% faster
Liquid is the roughly 20-year-old code that turns Shopify store templates into web pages. He pointed the loop at it: change something, run all 974 existing tests, time it, keep or undo.
The score: Time to process real store templates, with every existing test still passing.
What happened: About 120 experiments, 93 changes kept: 53% faster, and 61% fewer objects created in memory. His own caveat: “This is probably somewhat overfit.” Still waiting to be merged as of Oct 1, 2026.
Mar 12, 2026The pull request
- Ryan Li · Paradigm's prediction market challenge
First place in a trading-strategy contest
A hackathon to write the most profitable trading program for a simulated market. Instead of one agent in a loop, he ran many at once, each trying its own variations of the strategy.
The score: Average profit across many simulated runs.
What happened: 1,039 versions tried. His entry went from dead last, losing $15.83 a run, to first place at $42.32.
Apr 9, 2026His repo
Look at what changes between the simple ones and the advanced ones. Not the loop. Only what gets edited and how it's scored. Shopify packaged its version as a plugin, pi-autoresearch, and its engineers now report unit tests running 300 times faster and a build that got 65% faster, the kind of chore nobody ever gets around to.
What it does
It makes a training script better, one small kept change at a time, and leaves a full record of every attempt. This is the chart from the repo, redrawn: one overnight run.
Running best val_bpb · lower is better
83 experiments · 15 kept
- #00.9979baseline
- #20.9907halve total batch 524K→262K (more steps)
- #60.9897warmdown 0.5→0.7 (more cooldown helps)
- #80.9874add 5% warmup
- #140.9845depth 9, aspect_ratio 57 (same dim 512…)
- #230.9839x0_lambda init 0.1→0.05
- #280.9833unembedding LR 0.004→0.008
- #320.9821SSSSL window pattern (more sliding window)
- #380.9799short window 1/4 context instead of 1/2
- #390.9796short window 1/8 context (256 tokens)
- #430.9789embedding LR 0.6→0.8
- #640.9783RoPE base frequency 10000→50000
- #650.9780RoPE base frequency 50000→100000
- #670.9777RoPE base frequency 100000→200000
- #740.9773random seed 42→137
Fifteen dots, counting the baseline. The other 68 experiments didn't beat the best so far, or crashed, and were thrown away. Values are read off his chart and rounded; descriptions are the agent's own, from its log. Notice the shape: big steps early, small ones late, and the last “improvement” is a different random seed.
What one experiment prints
--- val_bpb: 0.997900 training_seconds: 300.1 total_seconds: 325.9 peak_vram_mb: 45060.2 mfu_percent: 39.80 total_tokens_M: 499.6 num_steps: 953 num_params_M: 50.3 depth: 8
The agent reads only the first line and the memory line. This example is from program.md: a 50-million-parameter model, 953 training steps in 300 seconds.
What you wake up to · results.tsv
commit val_bpb memory_gb status description a1b2c3d 0.997900 44.0 keep baseline b2c3d4e 0.993200 44.2 keep increase LR to 0.04 c3d4e5f 1.005000 44.0 discard switch to GeLU activation d4e5f6g 0.000000 0.0 crash double model width (OOM)
One row per experiment: the commit, the score, peak memory, what happened and what was tried. Also the example from program.md. Plus a Git branch whose tip is the best train.py found.
Does it find anything real? Round 1, on full nanochat
Karpathy left it tuning nanochat for about two days. It worked through roughly 700 changes and kept about 20. He re-tested them by hand: all of them stacked, and all carried over to a model twice as deep. Together they cut nanochat's “Time to GPT-2” from 2.02 hours to 1.80, about 11%, on a project he had already tuned by hand for a long time. His write-up, and the commit.
- ·His QK norm had no scaling multiplier, so attention was too diffuse. The agent found multipliers that sharpen it.
- ·The value embeddings wanted regularization and weren't getting any.
- ·The banded attention was too conservative. He'd forgotten to tune it.
- ·The AdamW betas were, in his words, “all messed up.”
- ·It tuned the weight decay schedule.
- ·It tuned the network initialization.
“It's not novel, ground-breaking ‘research’ (yet), but all the adjustments are ‘real’, I didn't find them manually previously, and they stack up and actually improved nanochat.”
Three files, three owners
“The repo is deliberately kept small and only really has three files that matter.” Who is allowed to edit each one is the whole design.
- You edit this
program.md
The instructions for one agent: how to set up a run, what it may and may not touch, how to log, the loop, and the order to never stop. This is where your work goes.
114 lines of MarkdownRead it
- The agent edits this
train.py
The whole experiment in one file: a GPT model, the optimizer (Muon + AdamW), the training loop and every setting. All of it is fair game for the agent.
630 lines of PythonRead it
- Nobody edits this
prepare.py
The referee. Downloads the data once, trains the tokenizer, and holds the fixed pieces: the 5-minute budget, the context length and evaluate_bpb, the function that scores every run.
389 lines of PythonRead it
And four things that show up once it's running
- results.tsvOne row per experiment, written by the agent. Never committed, so a reset can't erase it.
- run.logThe last run's full output. The agent greps two lines out of it instead of reading it all.
- analysis.ipynbA notebook for you. Turns results.tsv into the progress chart and a ranked list of wins.
- ~/.cache/autoresearch/Where prepare.py puts the data shards and the tokenizer. Outside the repo.
The loop
Seven steps, clockwise from the top, then round again. This is the “LOOP FOREVER” block from program.md. Click a step to see the command and the reason for it.
- and back to step 1, forever
Step 1 of 7
Where are we?
Check the branch
What happens
The agent looks at Git: which branch it's on and which commit is the current best. Every experiment starts from the last kept commit on autoresearch/<tag>.
Why it's done this way
After a hundred rounds and a few context compactions, Git is the one record of what the current best code actually is.
The line in program.md
“Look at the git state: the current branch/commit we're on”
Before the first round
The agent does this once per run, with you, and then stops asking questions. Each run gets its own branch, so last night's work and tonight's never mix.
- 1Agree on a run tag based on today's date, like mar5.
- 2Create the branch autoresearch/<tag> from master. It must not already exist.
- 3Read README.md, prepare.py and train.py for full context.
- 4Check that ~/.cache/autoresearch/ has the data and tokenizer. If not, tell you to run prepare.py.
- 5Create results.tsv with only the header row.
- 6Confirm with you, then go. The first run is always train.py untouched, to set the baseline.
The rules the agent plays by
All of this is plain English in one Markdown file. There is no sandbox enforcing it. The agent follows it because the file says so.
It can
- ✓Edit train.py. It is the only file the agent changes.
- ✓Change anything inside it: architecture, optimizer, hyperparameters, training loop, batch size, model size.
- ✓Use more VRAM if the gain is real. Memory is a soft limit: “it should not blow up dramatically.”
- ✓Rewind to an earlier commit if it's stuck, but “very very sparingly (if ever).”
It can't
- ✕Touch prepare.py. It holds the evaluation, the data loading, the tokenizer and the fixed constants.
- ✕Install packages or add dependencies. Only what's already in pyproject.toml.
- ✕Change the evaluation. “The evaluate_bpb function in prepare.py is the ground truth metric.”
- ✕Stop and ask whether to keep going.
The paragraph that makes it run all night
“Once the experiment loop has begun (after the initial setup), do NOT pause to ask the human if you should continue. Do NOT ask “should I keep going?” or “is this a good stopping point?”. The human might be asleep, or gone from a computer and expects you to continue working indefinitely until you are manually stopped.”
And when it gets stuck: “If you run out of ideas, think harder — read papers referenced in the code, re-read the in-scope files for new angles, try combining previous near-misses, try more radical architectural changes.”
The simplicity rule is the one people skip. “All else being equal, simpler is better.” A tiny gain that adds twenty lines of hacky code gets thrown away. The same result from less code is kept. That's what stops a hundred nights of this from turning train.py into a mess.
Why it's built this way
Four decisions make the loop trustworthy enough to leave alone. Each one removes a way for the agent to fool itself, or you.
One file to change
The agent only touches train.py. The search space is still huge, but every experiment is one small diff a person can read.
“This keeps the scope manageable and diffs reviewable.”
A fixed five minutes
Every run gets the same wall-clock budget, so a bigger model and a smaller one compete fairly: whichever learns more in five minutes wins. It also means the loop finds the best model for your machine. The cost is that your numbers can't be compared with anyone else's.
“This makes experiments directly comparable regardless of what the agent changes.”
Nothing else to set up
PyTorch and a few small packages. No distributed training, no config system, no experiment tracker.
“One GPU, one file, one metric.”
A score that survives big changes
val_bpb is how many bits the model needs, on average, to predict each byte of text it hasn't seen. Counting per byte, not per token, means the agent can change the vocabulary or the architecture and the yardstick stays the same. It's measured on one pinned shard of data, about 21 million tokens, the same every run.
“Lower is better, and vocab-size-independent so architectural changes are fairly compared.”
Set it up
Six commands, then one sentence to your agent. Steps marked FakeNerd are ours; the rest are straight from the README.
What you need
- One NVIDIA GPUTested on an H100. The example run in program.md peaks at about 44 GB of memory, so smaller cards need the settings turned down.
- Python 3.10+ and uvuv is the project manager that installs everything. PyTorch 2.9.1 comes pinned to the CUDA 12.8 build.
- A coding agentClaude Code, Codex, or another agent that can edit files and run shell commands on that machine.
- A box you can afford to loseThe agent runs with permissions off, all night. Use a rented GPU or a fresh machine, not your laptop.
- 01Get onto the GPU machine
SSH in and check the card is visible. No NVIDIA GPU? Skip down to the forks for Mac, Windows and AMD.
nvidia-smi
FakeNerd's addition - 02Clone the repo and install uv
uv is one line to install if you don't have it.
git clone https://github.com/karpathy/autoresearch.git cd autoresearch curl -LsSf https://astral.sh/uv/install.sh | sh
- 03Install dependencies
Creates the environment and pulls PyTorch and eight small packages.
uv sync
- 04Prepare the data, once
About two minutes. Downloads ten training shards and one validation shard of the ClimbMix dataset and trains an 8,192-token tokenizer. Everything lands in ~/.cache/autoresearch/.
uv run prepare.py
- 05Run one experiment by hand
About five minutes. If it ends with a summary block that has a val_bpb line, your setup works. That number is the baseline for this machine.
uv run train.py
- 06Start your agent in the repo, permissions off
The README says to “disable all permissions” so the agent never waits for a yes. A tmux session keeps it alive when your SSH connection drops.
tmux new -s research claude --dangerously-skip-permissions
FakeNerd's addition - 07Say the opening line
The agent reads program.md, proposes a run tag, makes the branch, checks the data, creates results.tsv and asks you to confirm. Say yes. It's the last question it will ask.
Hi have a look at program.md and let's kick off a new experiment! let's do the setup first.
- 08Leave. Come back in the morning
Detach from tmux with Ctrl-b then d, and close the laptop. To stop the loop, reattach and interrupt the agent. The branch tip is the best code it found.
tmux attach -t research
FakeNerd's addition
Steps 2 to 5, to paste
git clone https://github.com/karpathy/autoresearch.git cd autoresearch curl -LsSf https://astral.sh/uv/install.sh | sh uv sync uv run prepare.py uv run train.py
The opening line, from the README
Hi have a look at program.md and let's kick off a new experiment! let's do the setup first.
On Codex, the flag that turns approvals off is --dangerously-bypass-approvals-and-sandbox. Flag names change; check your agent's help. uv docs. New to neural networks? The README points to this beginner's guide.
In the morning
- 1Read the logEvery experiment, in order, with what was tried and whether it was kept.
column -t -s $'\t' results.tsv | less -S
- 2Read the kept commitsOne commit per improvement. Then the full difference from where you started.
git log --oneline master..HEAD git diff master -- train.py
- 3Draw the chartOpen analysis.ipynb. It reads results.tsv and saves progress.png. Jupyter isn't in the dependencies, so open it in an editor with notebook support or add a kernel yourself.
- 4Decide what to change in program.mdWhere did it waste rounds? What did it keep that looks like luck? Write the correction into program.md and run again tonight.
The notebook is analysis.ipynb. It prints the keep rate, lists every kept change, and ranks them by how much each one helped.
No H100?
The main repo needs one NVIDIA GPU, and Karpathy isn't sure he wants to widen that himself. For a MacBook or a smaller card he recommends a fork, and gives seven settings to turn down.
- macOSmiolini/autoresearch-macos The same repo adapted to run on a Mac.
- macOS · MLXtrevin-creator/autoresearch-mlx A port to Apple's MLX for Apple Silicon. No PyTorch.
- Windows · RTXjsegov/autoresearch-win-rtx For Windows machines with consumer NVIDIA cards.
- AMDandyluo7/autoresearch For AMD GPUs.
Seven settings to turn down
- 1Datasetprepare.pyUse text with less variety, like his TinyStories set, so a small model can still produce something reasonable.
- 2vocab_sizeprepare.pyDown from 8192 to 4096, 2048, 1024, or even a 256-value byte-level tokenizer.
- 3MAX_SEQ_LENprepare.pyLower it a lot, down to 256 on some machines. Raise DEVICE_BATCH_SIZE a little to compensate.
- 4EVAL_TOKENSprepare.pyDecrease it so the validation pass covers much less data.
- 5DEPTHtrain.pyThe main size knob. Default is 8; try 4. Many other values are derived from it.
- 6WINDOW_PATTERNtrain.pyUse just “L”. The default “SSSL” alternating pattern may be very slow on your hardware.
- 7TOTAL_BATCH_SIZEtrain.pyLower it a lot, keeping powers of 2. He suggests going as low as 2**14, about 16K.
The first four live in prepare.py, which is off limits to the agent. You change them once, by hand, before a run. His dataset suggestion: TinyStories. His other suggestion: paste the README section and the source into your coding agent and ask it to help.
Your job: program.md
“You are programming the program.md Markdown files that provide context to the AI agents and set up your autonomous research org.” The one in the repo is a bare-bones starting point, on purpose.
What's in the default file
- SetupSix steps the agent does with you before a run: tag, branch, read the files, check the data, start the log, confirm.
- ExperimentationWhat it can and can't touch, the goal in one line, the memory limit and the simplicity test.
- Output formatThe summary block train.py prints, and the grep that pulls the metric out.
- Logging resultsThe five columns of results.tsv and an example.
- The experiment loopNine numbered lines under LOOP FOREVER, plus what to do about timeouts and crashes.
- Never stopThe paragraph that keeps it going while you sleep.
Read the whole file. It's short.
Five edits worth trying
These are FakeNerd's, not Karpathy's. He says only that “it's obvious how one would iterate on it over time,” and that the next step is more agents.
- 1Give it a directionThe default goal is only “get the lowest val_bpb.” Add a paragraph on what tonight is about, say the optimizer or the attention pattern, so a hundred rounds aren't a hundred learning-rate nudges.
- 2Make it re-test small winsThe last kept change on his chart is a different random seed. A gain that size is luck. Run the baseline a few times, write the spread into program.md, and require anything smaller to win twice before it's kept.
- 3Ask for a notebook, not only a tableresults.tsv holds one short line per try. Have the agent also keep an untracked notes file: why it tried each thing and what it would try next. That's what a fresh session picks up from.
- 4One agent per GPUprogram.md already hints at branches like autoresearch/mar5-gpu0. The simplest way to go parallel is a separate clone and branch per GPU, each agent told which card is its own.
- 5Change one thing in program.md at a timeTreat your Markdown like the agent treats train.py. One edit, one night, then compare the keep rate and the final number with the night before.
Use the pattern on your own problem
Most people will never train a language model. The loop is still worth stealing, because it works on anything you can score quickly.
Karpathy, Mar 9, 2026
“Any metric you care about that is reasonably efficient to evaluate (or that has more efficient proxy metrics such as training a smaller network) can be autoresearched by an agent swarm. It's worth thinking about whether your problem falls into this bucket too.”
Seven ingredients
- 01One numberA single score where lower or higher is plainly better. No judgment calls.here: val_bpb
- 02A referee the agent can't touchThe scoring code is read-only, and the instructions say so twice.here: prepare.py
- 03One file to changeSmall scope, readable diffs, nowhere to hide a mess.here: train.py
- 04A fixed budget per tryEvery attempt costs the same, so they compare fairly and you know how many you get per night.here: 300 seconds
- 05Git as the ratchetCommit before the test. Keep on a win, reset on a loss.here: autoresearch/<tag>
- 06A log outside GitFailures are recorded even though their code is thrown away.here: results.tsv
- 07An order to keep goingWithout it, an agent stops after a few rounds to ask if you're happy.here: NEVER STOP
Where the same shape could fit
| Problem | The one file | The referee | The number |
|---|---|---|---|
| A prompt that classifies support tickets | prompt.md | A script that runs it over 200 labeled tickets | Accuracy, higher is better |
| A slow test suite | The test config and fixtures | The suite itself: every test must still pass | Wall-clock seconds, lower is better |
| A heavy web page | The page component | A build plus a fixed measuring script | Kilobytes shipped, lower is better |
| A slow database query | query.sql | A fixed dataset and a check that results match the original | Milliseconds at p95, lower is better |
FakeNerd's sketches, untested. The catch is always the referee. The agent will chase exactly the number you give it, so if the score can be gamed, it will be. Keep the scoring code where the agent can't edit it, and make it check the thing you actually care about.
FakeNerd's take
The repo is small on purpose
You can read program.md in five minutes, and you should before anything else. The lesson isn't the training code. It's the shape: a frozen referee, one editable file, one number, Git as a ratchet.
program.md is the product
Same idea as every skill page on this site. When the agent does something dumb, you don't correct it in chat. You fix the file, and every future night gets the fix.
Small wins are noise until proven
Fifteen kept changes took the chart from 0.998 to 0.977, about 2%. The first few did most of it. By the end it was keeping a seed change. Know your noise floor before you believe row ninety.
“Disable all permissions” means a box you can lose
An agent with no brakes, all night, on a machine with a shell. Give it a rented GPU with nothing else on it. You pay for GPU hours and agent tokens, so set a spending cap on both before bed.
Attribution
- Andrej Karpathy (@karpathy): karpathy/autoresearch, March 2026. The files, the loop, the rules, the design choices and the small-machine settings are his, from the README and
program.md. Quotes are word for word. Read at commit 228791f on Oct 1, 2026. - Results from his posts of Mar 5 and Mar 9, 2026. Round 1 ran on the full nanochat project, not on this cut-down repo.
- The chart is redrawn from progress.png. Only the baseline value is exact; the rest are read off the image and rounded.
- The six examples of use come from each person's own post, write-up or repo, linked on each card. Their numbers are self-reported; FakeNerd hasn't reproduced them. Most adapt the loop and don't run this repo as it is.
- The forks belong to their authors. FakeNerd hasn't run them.
- The setup steps marked FakeNerd, the five
program.mdedits, the seven ingredients and the table of other uses are FakeNerd's. Karpathy doesn't endorse them.