Could agents learn to work better from their own runs?

What if agents studied their own past runs to do future ones more efficiently?

This question fits inside the broader field of agent memory, which has become a major research area over the last few years. This surge is easy to understand. An agent that remembers your past conversations can give more personalised, effective answers, carry context across sessions, and generally behave like a person who knows you.

But nearly all of that work, as far as I can tell, is about user-interaction memory: as you talk to an agent, it extracts salient facts, preferences, and episodes from your conversations. I think there’s a second, less-explored opportunity: the agent learning from its own behaviour.

There’s a useful distinction here. Psychologists sometimes split memory into semantic (facts you know) and procedural (things you know how to do). User-interaction memory is mostly semantic: it remembers facts about you. But the knowledge a working agent needs is largely procedural: how to get around a particular kind of site, which tool to reach for, what to avoid. Today, we mostly supply that procedural memory by hand, as agent skills, with humans writing down the procedures and handing them over. Which raises the obvious question: what if the agent could learn its own procedures, by watching itself work?

Every time an agent runs, you get its trace: the steps it took, its reasoning, the tools it called, and what came back. Collect traces across many runs and you get a wider view: what the agent tried, whether it worked, and whether there are lessons worth carrying forward. Task-specific or general, the good ones make future runs cheaper: fewer tokens, less blind exploration, fewer dead ends.

Why browser agents

To test this, I picked a domain where exploration, often unnecessary exploration, is the norm: browser agents.

A browser agent is given a task and a toolset (Playwright MCP, Browser Use, Puppeteer etc). It calls tools like “open page”, “click”, “fill form”, and reads the page back as a screenshot, raw HTML, or an accessibility tree (roughly HTML stripped of styling, and usually a lot smaller). Because it has no built-in map of the sites it’s sent to, it leans on three things: good prompting (which tool for which job), skills (how to handle specific sites), and plain exploration. It visits pages and figures out the next move from what it sees.

Browser agents make a good testbed for run-based memory for three reasons:

• they make a lot of tool calls, so there’s plenty to learn from;
• many of the available tools are distractions. Playwright MCP exposes around 22 tools, of which only maybe 8 are useful on a regular basis;
• you can verify whether the task was actually done.

So I built a career-scout agent: you tell it what you’re after (“find 5 software engineer roles at Redis in London”) and off it goes.

The naive version

My first design was about as simple as it gets:

1. Store the traces. Run a bunch of tasks, save every trace.
2. Extract lessons. Have a second LLM, the lesson extractor, read each trace and write cards: one lesson per card, phrased as advice for future runs. Each card also carries a field for human review, because (spoiler) LLMs happily hallucinate lessons too.
3. Inject on future runs. When a new task comes in, retrieve the relevant cards (a keyword match on the company or domain) and drop them into the agent’s prompt so it starts smarter.

Simple, right?

Right?

Wrong!!

Here are three failure modes I experienced, and the lessons I learned solving them.

Failure 1: extracting the right lesson is hard

Two kinds of hallucination showed up. First, the agent itself would return a real-looking application link that it had never actually opened. One run handed back a plausible Stripe apply URL, right domain and all, that appeared nowhere in the pages it had visited. Maybe the listing was real, maybe not; the point is the agent couldn’t show where it had seen it, so there was no reason to trust it.

The lesson extractor overfit in its own way, producing lessons something like “the London filter is the third checkbox”: true once, on that one site at that moment, and useless or wrong the moment the page changed. And that is a deeper problem: you can’t reliably learn a general lesson from a single run. From one trace the extractor might note “using the site’s own search box worked well”, but it has no way of knowing whether that’s a quirk of this one site or a rule that transfers to other tasks.

What helped:

• A grounding check for hallucination. Before trusting any job the agent returns, we verify its application link was actually seen on a page the agent visited. Anything it couldn’t have seen, we treat as fabricated. A cheap, surprisingly effective liar-detector.
• The human-review field on every card, plus a recent, capable model as the extractor, which noticeably cut the over-specific noise.
• For generality: stop trying to squeeze universal truths out of one run. Only promote a lesson once it’s been corroborated across several distinct tasks.

Failure 2: knowing which lessons apply, and whether they actually helped

First, retrieval. If you just inject the highest-confidence cards (confidence being how many tasks backed them), a generic one like “scroll to the bottom of the page” rides along on every task and crowds out the specific cards that actually matter. The fix was to scope retrieval to the task: inject the cards actually about this company or this kind of site, and top up with general ones only if there’s room.

Second, and more subtly, the classic credit assignment problem: how do we know if a card actually contributed to a future run’s success? My naive approach rewarded a card whenever it was in the prompt and the run happened to succeed. But that is only correlation: the card might have been irrelevant, or the agent might have succeeded by ignoring it. So we added a check. Before giving a card credit, read the trace and confirm the agent actually did the thing the card advised.

There’s a subtler version of the same problem: even when you retrieve the right card, the agent can ignore it. Early on, my system prompt told the agent to always start by searching the web. So when memory handed it the exact page to go to, the prompt won and it searched anyway. The lesson was sitting in context, correct, and overridden. The fix was to loosen the prompt: state the goal and the tools, but stop dictating the steps, so a learned shortcut has room to take effect. How you inject memory matters as much as what you inject: a rigid prompt will override a perfectly good card.

Failure 3: the lessons that transfer aren’t the ones you’d expect

My instinct was that the valuable memory would be facts about a company: “Redis’s jobs live on this board, so go straight there and skip the marketing site.”

That’s useful, but two things undercut it. Company facts, by definition, don’t transfer to other companies. And once the model has a decent web search of its own, “finding the board” is cheap anyway, so shortcutting it saves less than you’d hope.

The lessons that actually transferred were about how to use the tools.

A lesson like “on a job board that renders its listings into the page, read them straight out of the DOM instead of clicking each one” is a fact about how the tool and the page technology behave, so it holds regardless of the company. It recurs, it corroborates across sites, and it’s the thing that actually made future runs cheaper. I’d expected the company facts to matter most. They didn’t. The tool tactics did.

Put another way, the agent was building its own procedural memory: the how-to knowledge we normally hand-write as skills, learned instead from watching itself work.

A couple of the cards that earned their place looked like this:

• (anti-pattern) When the site’s search box or filter ignores a normal “fill in the form” call, stop retrying it; set the field directly, fire its change event, then wait for the results to refresh.
• (company fact) Anthropic’s jobs live on its Greenhouse board at job-boards.greenhouse.io/anthropic; go straight there and skip the marketing site.

The anti-pattern can work on sites the agent had never seen. The company fact only ever helps with Anthropic.

The tool tactic that did the most work is worth seeing in full. Here it is exactly as the system stored it, only reformatted into labelled lines:

heuristic · confidence 0.95, from 8 supporting runs and 0 contradictions
when a careers page exposes a Greenhouse jobs board but the rendered page does not obviously list all openings,
prefer browser_evaluate over clicking into individual postings, and use it to enumerate all links whose href points to the board’s job URLs,
because Greenhouse boards often render the full listing in the DOM or expose stable job-detail links, so reading the rendered document can reveal postings without extra navigation or fragile clicks.

That confidence score is worth a note: the model never writes it. It is derived from evidence (eight runs backed this lesson, and nothing contradicted it) and it fades if the lesson stops proving useful. A card has to earn its way into the prompt.

How do you measure what works?

Proving that adding memory helped was harder than I expected. Here are some factors I hadn’t considered at the start.

• Noise. Runs are non-deterministic and vary enormously by task: a bespoke careers site can cost several times what a clean job board does, regardless of memory. Comparing raw averages would be misleading. The fix is to run each task both with and without memory and compare the two against each other (a paired test), so the task’s difficulty cancels out.
• Contamination. My first proper experiment cheated lol: the evaluation runs were leaking back into the memory the agent was being tested on. Eval and learning have to be kept strictly apart.
• Infrastructure issues. One run was wrecked when a third-party search API hit its monthly quota halfway through, and because the runs weren’t interleaved, the failures landed almost entirely on the no-memory group, faking a huge “memory wins” result. I later dropped that dependency entirely and used the model’s own web search.
• Blind spots in my own logging. When I switched to the model’s native web search, the searches stopped showing up in my traces, and I assumed the provider was hiding them. It wasn’t: the search calls came back under a different label in the response and my code was simply throwing them away. A reminder that “the tool didn’t do it” often means “I didn’t capture it”.
• A new capability changed the strategy. Giving the agent the model’s own web search did more than swap one search tool for another. The agent sometimes answered from search snippets and never opened the browser at all, which both broke the grounding check and changed the very behaviour I was trying to measure. I had to add a rule that roles must be read from a page the agent actually opened. A stronger tool can rewrite the agent’s plan out from under your experiment.
• Separate “it broke” from “it failed”. Crashes, timeouts, and quota errors have to come out of the efficiency numbers (they wreck the averages) but stay in the success count, or an agent can look “efficient” simply by giving up early. Keeping those two apart was the difference between honest numbers and flattering ones.

So, does it work?

With the pipeline cleaned up, I ran a proper paired comparison across three companies of increasing difficulty (a clean job board, a bespoke careers site, and a large enterprise portal), asking each for software engineering roles, with memory on versus off.

Company (difficulty)	Fewer steps	Fewer tokens	Less page read	Faster
easy (clean job board)	−29%	−39%	−20%	−39%
medium (bespoke site)	−10%	−3%	−44%	−39%
hard (enterprise portal)	−26%	−47%	−62%	−12%
average	−21%	−30%	−42%	−20%

Paired comparison, memory on vs off, three companies, five repeats each. The per-row numbers are noisy at this sample size (the medium row’s near-flat tokens but much lower time is a case in point, since wall-clock also answers to page round-trips and waiting). Read the average as the honest signal.

Memory made the agent more efficient on every company and every measure, at equal or better task-success: roughly a third fewer tokens and steps, and it read about 40% less of the page. In a smaller follow-up on the hardest site, the memoryless agent often gave up against the bot defences while the memory-equipped one finished. On that site, just finishing was the win. And the win came, as above, mostly from the tool tactics rather than the company facts.

There’s a cost, of course: building the memory isn’t free. Amortised out, it paid for itself after roughly ten reuses of a given kind of task. Below that, you’re better off without it.

How it works under the hood

For the curious, the whole pipeline in one place:

• Cards. Each lesson is a small structured record of one of three kinds: a company fact (where a firm’s jobs live), a heuristic (“prefer X because Y”), or an anti-pattern (“avoid Z”). Its confidence is derived from evidence, never written by the model, and it fades if the lesson stops being used.
• Extraction (offline). A cheap model reads the stored traces and proposes cards. The grounding check and a human-review flag gate them before any can be injected.
• Promotion. A card becomes eligible for injection only after it’s been corroborated across several distinct tasks, so one lucky run can’t graduate a lesson on its own. Two things are worth tracking separately: whether a lesson is reliable (does following it keep working) and whether it is general (does it hold across different tasks). A card can be rock-solid on one site and useless everywhere else.
• Retrieval. For a new task, the store returns the cards scoped to that company or site, topped up with a few of the strongest general ones.
• Credit. After a run, cards that were injected and that the trace shows the agent actually followed get their evidence updated, and confidence is re-derived from the totals.
• Dedup. The same lesson written two different ways barely overlaps word-for-word, so a keyword merge misses it and the store fills with three phrasings of one idea. Merging by meaning, with a model, rather than by shared words is what keeps the prompt clean.

Where I’d be careful

While this was fun to build and the results were encouraging, I’d be careful about claiming memory helps every kind of agent, for two reasons.

1. It’s a small experiment. Three companies with a few repeats each. In absolute terms it’s tiny: the memory was distilled from a couple of runs per company, the active card store never grew past 10 cards, and the extractor is a relatively cheap model. This is a proof of concept rather than a benchmark.
2. There’s a security edge here. The tactic that helped most, letting the agent run its own JavaScript inside the page to read it, means executing agent-authored code in the page’s context. Teaching the agent to prefer that, and then re-injecting that preference from memory, widens the attack surface: a malicious page, or a poisoned lesson, could turn a helpful habit into a liability. Worth designing against from the start.

Main lessons learned

1. Treat the agent’s own memory as untrusted input. It’s written by an LLM from web-page content, so it can be wrong or even adversarial. Validate before you trust; keep a human in the loop for anything that changes behaviour.
2. Memory should make the agent smarter without making it more powerful. A learned lesson can change which known-safe move the agent prefers; it should never hand it new capabilities (fresh code to run, new tools to reach for).
3. Learn the transferable thing rather than the brittle thing. “How this kind of page behaves” can be applied across different tasks; “the third checkbox on this exact page” cannot.
4. One run is an anecdote. Corroborate a “general” lesson across genuinely different tasks before you let it steer anything.
5. Decide what “better” means before you optimise it, and measure honestly. Pair your runs, keep evaluation away from learning, and assume the plumbing is lying to you until you’ve checked.
6. Benchmark where the agent fails rather than where it already wins. On easy sites a capable agent already succeeds, so memory can only shave a bit of cost; the interesting gains (it finishes at all) show up on the hard cases. Pick evaluation tasks with real headroom, or you’ll under-measure the thing you built.

Open Questions

A well-written skill and a good prompt might have bought a lot of what memory bought me here, without the extra cost or machinery. So the real question is no longer whether an agent can learn from its own runs (it can). It’s when learning is worth more than just writing down what you already know.

If I did it again, I’d keep the cheap, load-bearing parts and stay sceptical of the rest. The grounding check, the handful of tool tactics that actually transferred, human review, and paired measurement all earned their place. The company-fact cards barely travelled and were sometimes wrong, and the confidence scoring, decay, and dedup machinery mostly existed to clean up after a noisy extractor. Given how few general lessons ever graduated, a well-written skill plus a grounding check would get you most of the way at a fraction of the moving parts. The learning loop earns its complexity only once the set of lessons grows too large, too fast-changing, or too site-specific for a person to keep up by hand.

There are a few other questions worth considering:

• As models get better, can memory be trusted enough to act on unsupervised? Everything here kept a human near the loop. Taking that human out is where the security and bad-memory questions stop being hypothetical.
• When should a lesson expire? The web doesn’t stay still. A board gets redesigned, a company switches its hiring system, a tool tactic that worked last month stops working. I currently fade a card’s confidence when it goes unused, but that is a blunt instrument. How do we let learned lessons keep up as the world underneath them changes?

That last part, an agent relearning the tactics of a shifting web on its own, is a rather nice thing for it to be doing while you sleep, if we can make it safe.

Anyway, here’s the lesson I’d leave you with. Before you reach for any of this machinery, hold your own task up to the light and ask: is the how-to knowledge it needs stable enough to write down once, or changeable and site-specific enough that it is worth teaching the agent to learn it for itself?

By the way, are you currently doing any interesting work with agents learning from their own runs? I’d love to hear how it’s going for you.