Could agents learn to work better from their own runs?

What if agents studied their own past runs to do future ones more efficiently?

Agent memory has become a major research area over the last few years. This surge is easy to understand. An agent that remembers your past conversations can give more personalised, effective answers, carry context across sessions, and generally behave like a person who knows you. LLMs are stateless by default, so this continuity has to be architected into the agent harness.

But what do we usually mean when we talk about agent memory? Nearly all of that work, as far as I can tell, is about user-interaction memory: as you talk to an agent, it extracts salient facts, preferences, and episodes from your conversations.

That made me wonder about a second, less-explored opportunity: could the agent learn from its own behaviour?

There’s a useful distinction here. Psychologists sometimes divide memory into semantic (facts you know), episodic (events that happened), and procedural (things you know how to do). User-interaction memory is mostly semantic and episodic: it remembers facts about you and events from your conversations. But much of the knowledge a working agent needs is procedural: how to navigate a particular kind of site, which tool to reach for, and what to avoid.

Today, we mostly supply that procedural knowledge by hand, as agent skills, with humans writing down the procedures and handing them over. What would happen if the agent could learn its own procedures by watching itself work?

How can an agent watch itself work?

Every time an agent runs, you get its trace: the steps it took, its reasoning, the tools it called, and what came back. Collect traces across many runs and you get a wider view of what the agent tried, whether it worked, and whether there are lessons worth carrying forward. Those lessons can be task-specific or general, but the good ones should make future runs cheaper: fewer tokens, less blind exploration, fewer dead ends.

Why browser agents?

Once I had the question, I needed somewhere to test it. I picked a domain where exploration, often unnecessary exploration, is the norm: browser agents.

A browser agent is given a task and a toolset such as Playwright MCP, Browser Use, or Puppeteer. It calls tools like “open page”, “click”, and “fill form”, then reads the page back as a screenshot, raw HTML, or an accessibility tree (roughly HTML stripped of styling, and usually much smaller). Because it has no built-in map of the sites it’s sent to, it leans on three things: good prompting, skills for handling specific sites, and plain exploration. It visits pages and figures out the next move from what it sees.

Browser agents make a good testbed for run-based memory for three reasons:

• They make a lot of tool calls, so there’s plenty to learn from.
• Many of the available tools are distractions. Playwright MCP exposes around 22 tools, of which perhaps eight are useful regularly.
• You can verify whether the task was actually completed.

So I built a career-scout agent: you tell it what you’re after, perhaps “find five software engineer roles at Redis in London”, and off it goes.

The naive version

My first design was about as simple as it gets:

1. Store the traces. Run a set of tasks and save every trace.
2. Extract lessons. Have a second LLM, the lesson extractor, read each trace and write cards: one lesson per card, phrased as advice for future runs.
3. Inject lessons into future runs. When a new task comes in, retrieve the relevant cards and add them to the agent’s prompt so it starts smarter.

Simple, right?

Right?

Wrong!!

Three failure modes changed how I thought about the whole system.

Failure 1: Extracting the right lesson is hard. Really hard.

Let’s take a step back. What qualifies a lesson as the “right” one?

For me, the answer is simple: a lesson is right if it helps the agent perform future tasks better. The problem is that we do not know what those future tasks will look like at the time the lesson is extracted.

This led to two issues.

The first was specificity. My lesson extractor could produce lessons that were far too specific to transfer. A lesson like “the London filter is the third checkbox” might help once, on one site, at one moment. A small website change could make it useless.

The second was trust in the trace itself. The lesson extractor learns from whatever the agent recorded, so bad trace data can easily become bad memory. My agent sometimes returned real-looking application links that it had never opened. One run handed back a plausible Stripe application URL, right domain and all, that appeared nowhere in the pages it had visited. The listing may have been real. The agent could not show where it had seen it, so there was no reason to trust it or let another model learn from it.

These two issues forced me to separate reliability from generality. Reliability asks whether a lesson keeps working. Generality asks whether it works across different tasks. A lesson can be rock-solid on one site and useless everywhere else.

What helped:

• A grounding check for hallucination. Before trusting any job the agent returns, verify that its application link appeared on a page the agent visited. Anything absent from those pages is treated as fabricated: a cheap, surprisingly effective liar detector.
• Human review and a capable extractor. Every card carries a field for human review. Using a recent, capable model as the extractor also noticeably reduced the over-specific noise.
• Corroboration across distinct tasks. Stop trying to squeeze universal truths out of one run. In my experiment, a lesson qualified for promotion after more than five different tasks supported it.

Failure 2: Knowing which lessons apply and whether they helped

Extracting a useful lesson was only the first problem. When a new task arrived, which lessons should the agent see?

I started with retrieval. If you simply inject the highest-confidence cards, a generic one like “scroll to the bottom of the page” can ride along on every task and crowd out the specific cards that matter. So I scoped retrieval to the task: inject cards about the relevant company or kind of site, then top them up with a few strong general lessons if there is room.

That decided which cards entered the prompt. It still left me with the credit assignment problem. Suppose a new task asks for five software engineering jobs at Cloudflare in Glasgow and three cards are injected into the agent’s context. How many of those cards were useful? Did any hurt performance? Did the agent ignore one entirely?

My naive approach rewarded a card whenever it appeared in the prompt and the run succeeded. But that is only correlation: the card might have been irrelevant, or the agent might have succeeded by ignoring it. So I added another check. Before giving a card credit, another LLM reads the trace and confirms that the agent actually did what the card advised.

Then I ran into another wrinkle: what if the right card is retrieved and the agent ignores it? Early on, my system prompt told the agent always to start by searching the web. When memory handed it the exact page to visit, the prompt won and it searched anyway. The lesson was correct, present in the context, and overridden.

What changed? I loosened the prompt. It stated the goal and the available tools while giving the agent room to choose the steps. That gave a learned shortcut space to take effect. How you inject memory matters as much as what you inject. A rigid prompt can override a perfectly good lesson.

Failure 3: The lessons that transferred surprised me

What did I expect the agent to learn? Mostly facts about a company: “Redis’s jobs live on this board, so go straight there and skip the marketing site.”

That kind of fact is useful, but its reach is narrow. It only applies to the named company. Once the model has a decent web search of its own, finding the job board is relatively cheap anyway, so shortcutting it saves less than you might hope.

So what transferred? Lessons about how to use the tools.

A lesson like “on a job board that renders its listings into the page, read them straight from the DOM and avoid clicking each one” describes how the tool and the page technology behave, so it can hold regardless of the company. It recurs, it can be corroborated across sites, and it is the kind of lesson that actually made future runs cheaper.

The strongest gains came from tool tactics. That surprised me because I had expected company facts to matter most.

Once I saw that, the result made more sense. The agent was building its own procedural memory: the how-to knowledge we normally hand-write as skills, learned by watching itself work.

Two of the cards that earned their place looked like this:

• (anti-pattern) When the site’s search box or filter ignores a normal “fill in the form” call, stop retrying it; set the field directly, fire its change event, then wait for the results to refresh.
• (company fact) Anthropic’s jobs live on its Greenhouse board at job-boards.greenhouse.io/anthropic; go straight there and skip the marketing site.

The anti-pattern can work on sites the agent has never seen. The company fact only ever helps with Anthropic.

The tool tactic that did the most work is worth seeing in full. Here it is exactly as the system stored it, reformatted into labelled lines:

heuristic · confidence 0.95, from 8 supporting runs and 0 contradictions
when a careers page exposes a Greenhouse jobs board but the rendered page does not obviously list all openings,
prefer browser_evaluate over clicking into individual postings, and use it to enumerate all links whose href points to the board’s job URLs,
because Greenhouse boards often render the full listing in the DOM or expose stable job-detail links, so reading the rendered document can reveal postings without extra navigation or fragile clicks.

The model never writes that confidence score. It is derived from evidence: eight runs supported this lesson and none contradicted it. The score also fades if the lesson stops proving useful. A card has to earn its way into the prompt.

How do you measure what works?

I thought measurement would be straightforward: run the agent with memory, run it again without memory, and compare the numbers. In practice, proving that memory helped became one of the hardest parts of the experiment. How do you make a fair comparison when every task and every run can behave differently?

Several factors I had not considered at the start ended up mattering:

• Noise. Runs are non-deterministic, and difficulty varies enormously by task. A bespoke careers site can cost several times what a clean job board does, regardless of memory. Comparing raw averages would therefore be misleading. The fix is to run each task both with and without memory and compare the pair, so the task’s difficulty largely cancels out.
• Contamination. My first proper experiment accidentally cheated: evaluation runs were leaking back into the memory the agent was being tested on. Evaluation and learning have to be kept strictly separate.
• Infrastructure failures. One experiment was wrecked when a third-party search API hit its monthly quota halfway through. I had run each group in sequence, so the failures landed almost entirely in the no-memory group and created a huge but false “memory wins” result. I later dropped that dependency and used the model’s own web search.
• Blind spots in my logging. When I switched to the model’s native web search, the searches disappeared from my traces, and I assumed the provider was hiding them. The calls were coming back under a different label, and my code was throwing them away. A reminder to check the logging before blaming the tool.
• A new capability changed the strategy. Giving the agent native web search did more than swap one search tool for another. It sometimes answered from search snippets without opening the browser, which broke the grounding check and changed the behaviour I was trying to measure. I had to require that roles be read from a page the agent actually opened. A stronger tool can rewrite the agent’s plan from under your experiment.
• Track crashes separately from task failures. Crashes, timeouts, and quota errors have to come out of the efficiency numbers because they distort the averages, but they must remain in the success count. Otherwise, an agent can appear efficient simply by giving up early.

So, does it work?

Once I had cleaned up the pipeline, I could return to the question I started with.

I ran a paired comparison across three companies of increasing difficulty: a clean job board, a bespoke careers site, and a large enterprise portal. I asked each for software engineering roles with memory on versus off.

Company (difficulty)	Fewer steps	Fewer tokens	Less page read	Faster
easy (clean job board)	−29%	−39%	−20%	−39%
medium (bespoke site)	−10%	−3%	−44%	−39%
hard (enterprise portal)	−26%	−47%	−62%	−12%
average	−21%	−30%	−42%	−20%

Paired comparison, memory on versus off, three companies, five repeats each. The per-row numbers are noisy at this sample size. The medium row’s near-flat token count but much lower time is a case in point, since wall-clock time also depends on page round-trips and waiting. Read the average as the honest signal.

Memory made the agent more efficient on every company and every measure, at equal or better task success: around 30% fewer tokens, 21% fewer steps, 42% less page content read, and 20% less time.

In a smaller follow-up on the hardest site, the memoryless agent often gave up against the bot defences while the memory-equipped one finished. On that site, finishing at all was the win. Tool tactics produced most of the advantage; company facts contributed less.

Building the memory also has a cost. Amortised out, it paid for itself after roughly ten reuses of a given kind of task. Below that, you are probably better off without it.

How far should I generalise from these results? Carefully. There are two caveats.

First, this was a small experiment: three companies with a few repeats each. In absolute terms, it was tiny. The memory was distilled from a couple of runs per company, the active card store never grew beyond ten cards, and the extractor was a relatively cheap model. Treat this as a proof of concept.

Second, what happens when the most useful lesson also widens the attack surface? The tactic that helped most was letting the agent run its own JavaScript inside the page to read it. That means executing agent-authored code in the page’s context. Teaching the agent to prefer that tactic, then reinjecting the preference from memory, creates another route for a malicious page or a poisoned lesson to influence future behaviour. Safety has to be part of the memory design from the start.

Main lessons learned

Looking back, what would I carry into another experiment? Six things:

1. Treat the agent’s memory as untrusted input. It is written by an LLM from web-page content, so it can be wrong or even adversarial. Validate it before trusting it, and keep a human in the loop for anything that changes behaviour.
2. Keep memory within the agent’s existing permissions. A learned lesson can change which known-safe move the agent prefers. New capabilities, such as fresh code to run or new tools to reach for, need their own security controls.
3. Learn durable tactics. Lessons about how a kind of page behaves can apply across tasks. Exact details such as “the third checkbox on this page” become brittle quickly.
4. One run is an anecdote. Corroborate a general lesson across genuinely different tasks before letting it steer anything.
5. Decide what “better” means before optimising it, and measure honestly. Pair the runs, keep evaluation separate from learning, and assume the plumbing is lying until you have checked it.
6. Benchmark tasks with genuine headroom. Easy sites offer incremental savings because a capable agent already succeeds on them. Harder tasks reveal the interesting gains, including whether the agent finishes at all.

Open questions

A well-written skill and a good prompt might have bought much of what memory bought me here at a lower cost. This experiment answered my opening question: agents can learn from their own runs. It left a harder one: when does learning outperform writing down what you already know?

If I did this again, which parts would stay? The cheap, load-bearing ones: the grounding check, the handful of tool tactics that transferred, human review, and paired measurement. The company-fact cards barely travelled and were sometimes wrong. Confidence scoring, decay, and deduplication mostly existed to clean up after a noisy extractor.

Given how few general lessons ever graduated, a well-written skill plus a grounding check would get you most of the way at a fraction of the complexity. So when does the learning loop earn its machinery? When the lessons become too numerous, fast-changing, or site-specific for a person to maintain by hand.

A few questions remain:

• How does credit assignment scale? My approach asks another LLM to inspect the trace and determine whether the agent followed each injected card. That is manageable with ten active cards, but potentially expensive and noisy with hundreds or thousands.
• Can memory be trusted without human supervision? Everything here kept a human near the loop. Removing that person is where the security and bad-memory questions stop being hypothetical.
• When should a lesson expire? The web keeps changing. A board gets redesigned, a company switches hiring systems, or a tool tactic that worked last month stops working. Fading a card’s confidence when it goes unused is a blunt instrument. How should learned lessons keep up as the world underneath them changes?

That last possibility, an agent safely relearning the tactics of a shifting web on its own, is a rather nice thing for it to be doing while you sleep.

Before reaching for any of this, I think it’s important to hold your own task up to the light and ask: is the how-to knowledge it needs stable enough to write down once, or changeable and site-specific enough that it is worth teaching the agent to learn for itself?

Thanks for reading. Are you doing any interesting work with agents learning from their own runs? I’d love to hear how it’s going.