288 papers behind the product

paper-a-mem-agentic-memory.md

ActionGuard: Tool Call Authorization under Poisoned Skills

Addresses the skill supply-chain threat: third-party skills carry task-specific instructions, scripts, and tool procedures — and an attacker who inserts malicious instructions into an otherwise benign skill can turn a benign user request into data exfiltration, file deletion, or unauthorized execution.

ActiveSaddler: Automated Curriculum Learning for Agent Harness Optimization

Optimizes the scenarios harness optimization learns from, not just the harness — the training curriculum co-evolves with the artifact it trains.

Adaptive Auto-Harness: Sustained Self-Improvement for Agentic System Deployment on Open-Ended Task Streams

Evolves a tree of harnesses under a non-stationary task stream, with solve-time routing and human-steering hooks — the comparator OO-Harness cites for specialized-harness families.

Automated Design of Agentic Systems

Names the field Automated Design of Agentic Systems (ADAS): treat inventing prompts, tool use, workflows, and combinations as a search problem rather than hand craft.

Automated Design of Agentic Systems: A Survey of Algorithms for Searching, Optimizing, and Evolving LLM Agents, Workflows, and Prompts

Is the best single map tying ADAS to harness/scaffold terminology in this catalog.

AEGIS: No Tool Call Left Unchecked — A Pre-Execution Firewall and Audit Layer for AI Agents

Fills the missing control point in agent stacks: model-generated tool calls are handed to the execution layer with nothing framework-agnostic in between.

AFlow: Automating Agentic Workflow Generation

Treats an agent as a code-represented workflow: LLM-invoking nodes connected by code edges (control flow, conditions, dependencies).

Agent-as-a-Router: Agentic Model Routing for Coding Tasks

Argues the bottleneck in model routing isn't the routing algorithm — it's information deficit: routers make a static one-off classification from the task text, when what they lack is execution-grounded knowledge of which model is good at what.

Agent-Controlled Forgetting

Lets the acting model choose what to forget: replace stale tool results with short stubs, keep originals in a recoverable archive — 50% token savings in noisy trajectories, with honest workload-dependence caveats.

Where LLM Agents Fail and How They can Learn From Failures

Introduces three linked contributions: AgentErrorTaxonomy, a five-dimension classification of LLM agent _runtime trajectory_ failures; AgentErrorBench, a human-annotated dataset of failed agent runs across ALFWorld, GAIA, and WebShop; and AgentDebug, a framework that traces a failure back to its root-cause step and generates targeted in-context corrective feedback.

MAGMA: A Multi-Graph based Agentic Memory Architecture for AI Agents

Argues that flat, semantic-similarity-only memory stores "struggle to reason about why, since they lack explicit representations of causal structure," and proposes splitting memory into four orthogonal relation graphs over a shared set of event-nodes, retrieved via a policy-guided traversal rather than top-k similarity search alone.

paper-agent-optimization-vero.md

An Exploratory Study of Agent Plans for Agentic AI Coding Tools in Open-Source Software

empirically catalogs the plan file — the task-oriented counterpart to the instruction file — as it exists in the wild, not as designed.

Agent Retrieval Bench: Evaluating Repository Context Retrieval for Coding Agents

Isolates the stage before the patch: finding the files the task actually needs.

From Agent Traces to Trust: A Survey of Evidence Tracing and Execution Provenance in LLM Agents

Unifies process-level accountability: final-answer accuracy can't explain how an output was produced, what evidence supported it, or where failure originated.

AgentAssay: Token-Efficient Regression Testing for Non-Deterministic Agent Workflows

Formalizes a question existing eval tools skip: did the agent get worse after a prompt, tool, model, or orchestration change?

AgentCgroup: Understanding and Controlling OS Resources of AI Agents

Characterizes OS-level resource dynamics of sandboxed coding agents and proposes an intent-driven eBPF / cgroup controller aligned with tool-call boundaries.

AgentDoG: A Diagnostic Guardrail Framework for AI Agent Safety and Security

Argues that a safe/unsafe label is the wrong output for an agent trajectory.

The Attack and Defense Landscape of Agentic AI: A Comprehensive Survey

Is billed as the first systematic survey of AI agent security, reviewing 128 papers across 51 attack methods and 60 defense mechanisms.

Characterizing Faults in Agentic AI: A Taxonomy of Types, Symptoms, and Root Causes

Is an empirical study of 13,602 issues/PRs across 40 popular Python agentic-AI repositories, from which 385 faults were sampled (stratified, 95% confidence / 5% margin) and coded via grounded theory + Apriori association-rule mining into a taxonomy of 34 fault types across four architectural dimensions, then validated by 145 practitioners (mean relevance 3.97/5, Cronbach's alpha 0.91).

The Agentic Scientist

Is the third generation of Sakana's AI Scientist line: an end-to-end, human-steerable orchestration scaffold that decomposes computational research into task drafting, parallel sandboxed investigation, retrospective evidence generation, and grounded manuscript writing.

AgentLTL: A Trace-Verification Framework for Measuring, Enforcing, and Training Procedural Compliance in Tool-Using LLM Agents

Applies a First-Order Linear Temporal Logic dialect to agent traces so procedure is part of correctness, not only final answers or LLM judges.

AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases

Shows that unverified entries in a memory store or RAG KB can hijack agent behavior at very low poison rates.

AgentRx: Diagnosing AI Agent Failures from Execution Trajectories

Targets a gap outcome-only metrics leave open: where a long-horizon agent trajectory became unrecoverable, with auditable evidence.

Evaluating AGENTS.md: Are Repository-Level Context Files Helpful for Coding Agents?

) is the first rigorous study of whether always-loaded repository context files actually help coding agents.

On the Impact of AGENTS.md Files on the Efficiency of AI Coding Agents

Is the counterpoint to the ETH study: context files don't fix correctness, but they make agents cheaper and faster.

AgentSpawn: Adaptive Multi-Agent Collaboration Through Dynamic Spawning for Long-Horizon Code Generation

replaces static multi-agent workflows with runtime spawning: when analysis reveals unanticipated complexity, the main agent spawns a sub-agent that inherits a sliced memory, not the full context.

AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agents

Proposes a lightweight DSL for runtime constraints on LLM agents.

AgentSquare: Automatic LLM Agent Search in Modular Design Space

AgentSwift: Efficient LLM Agent Design via Value-guided Hierarchical Search

Enlarges modular ADAS search: joint search over workflow structure and functional components (memory, planning, tool use), not workflow-only optimization.

AgentSys: Secure and Dynamic LLM Agents Through Explicit Hierarchical Memory Management

Defends against indirect prompt injection by OS-style hierarchical memory isolation.

AgentTrace: A Structured Logging Framework for Agent System Observability

argues agent nondeterminism defeats static auditing — so instrument at runtime, capturing structured logs across three surfaces: operational, cognitive, and contextual — as a foundation for security, accountability, and trust calibration.

AgentTrust: Runtime Safety Evaluation and Interception for AI Agent Tool Use

Sits between an agent and its tools, producing a structured verdict — allow, warn, block, review — before every action executes.

paper-ahe-observability.md

AI Code Sandboxes: A Comparative Security Study, Part 1 — Engine-Level Properties

reads six engine-level measurements together across five AI-sandbox products — arguing no single axis supports a ranking, and the cross-axis reading is the load-bearing analysis.

paper-ai-harness-engineering-runtime-substrate.md

Dive into Claude Code: The Design Space of Today's and Future AI Agent Systems

Is a source-code analysis of Claude Code v2.1.88 (TypeScript), supplemented by official Anthropic documentation, with a comparison against OpenClaw (an independent multi-channel gateway).

AIP: A Graph Representation for Learning and Governing Agent Skills

Proposes the Agent Instruction Protocol (AIP): an extension of the Agent Skills open standard that models each skill as a directed execution graph — typed step nodes (deterministic scripts or judgment prose), explicit input/output edges, and a schema-validated YAML spec — rather than free-form `SKILL.md` prose the agent re-derives every session.

Aletheia: Permission-Minimality Testing for Coding-Agent Rules

asks whether the authority a repository instruction file requests is actually needed — and answers by executing the task under one-permission-removed sandboxes until a dispensability witness emerges.

APDMem: Agent-Controlled Progressive Disclosure for Query-Adaptive Long-Term Memory

Reframes memory retrieval as a query-adaptive disclosure problem: not which documents match, but which conversations matter *and how much detail to expose*.

ARC: Active and Reflection-driven Context Management for Long-Horizon Information Seeking Agents

Treats context as a dynamic revisable internal reasoning state, not an append-only transcript or passive summary.

ASAP: Agent Scaffold Adaptation for Pretraining Evaluation

Evaluates base models' agentic capability by adapting existing agent scaffolds to the completion interface — no chat fine-tuning required — and exposes the trajectories for behavioral analysis.

AutoCompact: Learning When to Compact Context in Long-Horizon Coding Agents

Argues that as a task progresses, earlier exploration becomes *stale* — so context management is more than avoiding overflow.

AutoHarness: improving LLM agents by automatically synthesizing a code harness

Shows a smaller model can synthesize a deterministic code harness that prevents illegal environment actions, then outperform larger models at decision time.

Budget-Aware Agentic Routing via Boundary-Guided Training

Reframes model routing for agents: invoking the frontier model at every step is economically unsustainable, and routing a *trajectory* is a sequential, path-dependent problem — early mistakes compound, feedback arrives at episode end, and deployments demand strict per-task spending limits.

BenchTrace: A Benchmark for Testing Reflection Ability and Controlled Evolution in LLM Agents

Evaluates reflection quality and whether failure experience becomes avoidance behavior, not only end-task scores.

Beyond the GUI Paradigm: Do Mobile Agents Need the Phone Screen?

Asks whether mobile-agent harnesses need screen-based perception at all when a device CLI is available.

paper-beyond-outlining.md

Beyond Prompts: Measuring and Optimizing LLM Tool-Agent Harnesses

Scopes harness optimization to guarded edits at the tool boundary and, critically, makes the *reliability* of the chosen harness a first-class reported quantity.

paper-beyond-the-ide.md

Do Biological Structural Guarantees Earn Their Complexity?

Is not primarily a harness paper.

BM25 Wins at Scale: A Scaling Study of Retrieval-Augmented Generation Paradigms

Runs the cleanest scale-sweep of retrieval paradigms: same questions, same reader model, corpus grown along 28 strictly nested tiers from 1.7M to 601M tokens.

Bootstrapping Coding Agents: The Specification Is the Program

Reproduces the classical compiler bootstrap in the coding-agent domain: starting from a 926-word specification and a first implementation produced by an existing agent (Claude Code), a newly generated agent re-implements the same specification correctly from scratch.

Blocking at the Boundary: Auditing Long-Horizon Agents against Staged Prompt Injection

names and defeats staged prompt injection: an attack that exploits task-specific context, propagates across causally connected stages, and alters a consequential action while the workflow continues — confirmed against Claude Code and…

The Art of Building Verifiers for Computer Use Agents

Focuses on the verifier component of a computer-use agent (CUA) harness: without reliable verification, neither evaluation nor training signal can be trusted.

ByteRover: Agent-Native Memory Through LLM-Curated Hierarchical Context

Argues that a memory pipeline which chunks, embeds, and extracts entities does not know what the agent meant to store.

Context-as-a-Service: Surfacing Cross-File Dependency Chains for LLM-Generated Developer Documentation

Defines the cross-file documentation problem: documentation can be locally plausible and globally wrong, because confirming a claim requires following a non-obvious dependency chain — API comment → registry component, method name → state-management layer, tutorial step → framework lifecycle rule.

Authority Is Not a String: A Capability-Scoped Harness for Prompt-Injection-Resistant Coding Agents

Argues that a coding agent's tools carry ambient authority: naming a path or a command is enough to act on it.

Harness Engineering as Categorical Architecture

Argues harness design lacked a formal theory of composition and comparison, and shows the ArchAgents Architecture triple (G, Know, Φ) supplies one.

CheckerBench: Can Long-Horizon Agents Synthesize Static-Analysis Checkers?

tests a capability paniolo cares about directly: can an agent write a working static-analysis checker — interpret a defect spec, inspect a repo, implement analyzer logic, and refine through compile/analysis feedback — end to…

ClawVM: Harness-Managed Virtual Memory for Stateful Tool-Using LLM Agents

Applies operating-system virtual memory semantics to agent context management.

When Parallelism Pays Off: Cohesion-Aware Task Partitioning for Multi-Agent Coding

Formalizes multi-agent coding orchestration as a graph partitioning problem governed by the classic distributed-systems communication-to-computation trade-off: decomposition shortens critical-path computation, but every cross-agent dependency costs context transfer.

Cognitive Architectures for Language Agents

Is a taxonomy, not a build recipe.

Code as Agent Harness

Is a large multi-lab survey treating code itself as the harness substrate for coding agents, not only as agent output.

From Industry Claims to Empirical Reality: An Empirical Study of Code Review Agents in Pull Requests

Tests industry claims that code review agents can handle 80% of PRs unsupervised — against actual merge outcomes in AIDev's 19,450 agent-authored PRs.

Bigger Isn't Always Better: A Comparative Evaluation of LLMs for Automated Code Review

Evaluates five LLMs (Sonnet 4.6, Haiku 4.5, GPT-5.4 mini, Minimax M2.7, GLM-5 Turbo) on 150 code-review samples — 100 synthetic mutation-injected bugs + 50 real bug-fix PRs from eight OSS repos.

Codebase-Memory: Tree-Sitter-Based Knowledge Graphs for LLM Code Exploration via MCP

replaces repeated file-reading and grep-searching with a persistent Tree-Sitter knowledge graph served over MCP — structure-aware exploration without stuffing the context.

CodeGrep: An RL-Trained Retrieval Agent for LLM Coding Agents

Trains a 14B agent to do the *find-the-file* stage so the coding agent doesn't spend its budget on grep/glob/view loops.

CODENS: Transforming Code Changes into Living, Accessible, and Queryable Documentation

Turns pull requests into living documentation for production codebases — design knowledge is scattered across source files and PRs, and CODENS captures it incrementally at merge time.

CODESKILL: Learning Self-Evolving Skills for Coding Agents

treats skill-bank maintenance — what to extract, abstract, keep, merge, drop — not as a fixed heuristic but as a learnable policy trained with RL. Ingest context: awesome-harness-engineering-papers.

CodeWiki: Automated Repository-Level Documentation at Scale

Is the first open-source framework for holistic repository-level documentation across seven programming languages — developers spend ~58% of their time understanding codebases, and function-level doc tools can't capture architecture or cross-module interaction.

Codified Context: Infrastructure for AI Agents in a Complex Codebase

Is an observational study of a three-tier memory architecture built organically while developing a 108,000-line C# distributed simulation, evaluated across 283 sessions over 70 days.

Scaling Inference-Time Compute with Teams of Coding Agents

formulates multi-agent design as a compute-allocation problem: starting from a solo agent, where should extra inference compute go to reach the reward-cost Pareto frontier?

Small Agents with Semantic Search: Efficient Multilingual Code Localization

delegates file localization to compact, tool-specialized agents under 2B parameters — on-device search that cuts the big agent's token, latency, and cost bill.

Do Context Files Help Coding Agents? A Two-Agent Ablation Study on Real Repositories

Is the reconciliation study between paper-agents-md-context-files (no help, +20% cost) and paper-agents-md-efficiency (−28.6% runtime).

Context Rot in AI-Assisted Software Development: Repurposing Documentation Consistency for AI Configuration Artifacts

Names the exact problem paniolo's stale lane exists to solve: context rot — the gradual divergence between what AI configuration files (CLAUDE.md, AGENTS.md, GEMINI.md, .cursorrules, copilot-instructions.md) claim about a codebase and what actually holds.

Cortex: Measurement-Gated Evolution and Pruning of Coding-Agent Skills and Rules

Treats the agent's harness — CLAUDE.md/AGENTS.md, skills, file-scoped rules — as the editable surface and asks the question paniolo-evolve exists to answer: how do you know a harness text change actually helps, when agent success is stochastic?

CR-Bench: Evaluating the Real-World Utility of AI Code Review Agents

Builds the eval infrastructure code-review agents lacked: coarse success metrics can't capture behavior where false positives are costly.

paper-dean.md

DeepRefine: Agent-Compiled Knowledge Refinement via Reinforcement Learning

Is directly about the next product layer above llm-wiki-as-agent-memory: not how to compile a wiki once, but how to keep a pre-built agent knowledge base useful as users ask real questions.

DeepResearch Bench II: Diagnosing Deep Research Agents via Rubrics from Expert Reports

Fixes two failure modes in deep-research eval: benchmarks that don't test analysis/writing, and evaluation criteria that are coarse or LLM-defined — biased relative to human experts and hard to verify.

DEER: A Comprehensive and Reliable Benchmark for Deep-Research Expert Reports

Tackles expert-level report evaluation: report quality is multifaceted, LLM judges miss domain-expertise errors, and deep research's dependence on retrieved evidence makes report-wide claim verification necessary.

DeltaSelect: Affordable A/B Testing for Coding Agents

Is the evaluation-economics paper this batch needed: benchmarks are built for broad comparison, not for the frequent baseline-vs-candidate decisions harness iteration actually requires.

Doc2Harness: Benchmarking LLMs on Long-Horizon Harness Engineering of Coding Agents

Evaluates harness engineering as the task: build a complete, runnable harness from a spec, then freeze and execute it under a scripted mock model.

DocsChisel: Adaptive Tool Documentation Optimization Framework for LLM Agents

treats tool documentation not as a fixed input but as a tunable per-agent artifact — the fields that help one model/domain/paradigm hurt another.

DocSync: Agentic Documentation Maintenance via Critic-Guided Reflexion

Targets the repair half of documentation debt: static tools verify *presence* of docs but are blind to semantic consistency, while bare LLMs hallucinate without structural grounding.

paper-dyknow.md

When Sub-Agents Work in Parallel: The Promises and Pitfalls of Dynamic Concurrency in Long-Horizon Coding Tasks

is the first systematic study of native dynamic concurrency — a main agent deciding mid-execution whether/when/how to spawn sub-agents — across Codex, Claude Code, and Kimi Code.

Dynamic Agent Skills: A Lifecycle Survey and Taxonomy of Evolving Skill Libraries

synthesizes 124 papers (2023–2026) into the claim that skill libraries are lifecycle-managed, verified, evolving artifact stores — not static prompt bags.

The EPIC Framework for Spec-Driven Development

Asks what *good* spec-driven-development artifacts actually contain.

paper-erase.md

ESAA-Conversational: An Event-Sourced Memory Layer for Continuity, Handoff, and Curation Across Heterogeneous LLM Coding Agents

attacks conversational state drift: each agent keeps a private vendor log, so goals/decisions/tasks established with one agent aren't reliably available when another takes over.

Agent Harness Engineering: A Survey

; snapshotted 2026-07-24) is the practice-grounded seven-layer taxonomy for the agent harness as an independent system layer.

EVOHARNESSBENCH: Can Your Agents Keep Pace with an Evolving Harness?

Puts the non-stationarity inside the harness itself — the benchmark asks whether agents retain competence as their tools, skills, and specialist agents change underneath them.

EvolveNet: Collaborative Harness Evolution for Agent Self-Improvement

Moves experience extraction to the data: a shared harness is broadcast to data-local deployments, each evolves it on its own workload, and only the resulting program adaptations are composed back — raw experience never leaves its site.

ExecCritic: Learn to Test, Test to Improve for Coding Agents

Formalizes the coupled-validation problem in repository repair: when one trajectory writes both the patch and the test, their errors agree — a wrong patch passes a wrong test and looks validated.

Execution-Grounded Security Testing for Coding Agents in Software Engineering Pipelines

embeds target unsafe operations inside routine software-engineering workloads — unit testing, regression testing, crash reproduction, validation — and finds agents carry them out when the risk is hidden inside plausible tasks.

The Balkanization of Execution-Security Research for AI Coding Agents: Isolation, Access Control, and Time-of-Check-to-Time-of-Use Vulnerabilities

systematizes 39 papers (2023–2026) into 17 categories and finds the execution-security literature is structurally fragmented — mechanism papers rarely cite across categories, and the gaps live between the papers, not inside them.

ExecWeave: Cross-Layer Execution Provenance for Observable Agentic Software

solves the attribution gap between application traces (logical operations) and OS telemetry (machine activity) — relating the two is a reconstruction problem where a plausible-but-wrong association blames the wrong action.

Cost-Effective Repository Exploration for Agentic Issue Localization

asks whether the repo-exploration stage can be delegated to cheaper models — and answers yes, but the right operating point depends on how the downstream stage consumes the handoff. Ingest context: awesome-harness-engineering-papers.

Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineering

Is the strongest single survey for wiki-splitting work: it treats harness engineering as one of four externalization paradigms alongside memory, skills, and protocols.

paper-fact-duration.md

Facts as First-Class Objects: Knowledge Objects for Persistent LLM Memory

Is the quantitative case for moving durable facts out of the context window into addressable external storage.

FailForge: Distilling Procedural Competence from Persistent Failures into Code Agents

mines the samples rejection-sampling fine-tuning throws away: failed rollouts are the hardest, most informative training signal, and standard RFT discards them. Ingest context: awesome-harness-engineering-papers.

Failure as a Process: An Anatomy of CLI Coding Agent Trajectories

Is the first large-scale empirical study treating failure as a *temporal process* — onset, evolution, recovery — rather than a final outcome.

FastContext: Training Efficient Repository Explorer for Coding Agents

proposed a dedicated exploration subagent — specialized 4B–30B models that handle repo search and return concise file paths/line ranges so the solver's context stays clean.

paper-freshllms.md

Self-Evolving Agent Harnesses via Gated Semantic Quality-Diversity

Treats automatic harness improvement as a credit-assignment problem: proposing edits is easy; knowing which edits truly helped is hard when self-generated feedback is noisy.

Governed Shared Memory for Multi-Agent LLM Systems

Formalizes the fleet-memory problem and ships a production multi-tenant memory service plus an evaluation harness for governance leakage.

Scoped Verification for Reliable Long-Horizon Agentic Context Evolution under Distribution Shift

Keeps the evolving instruction as a typed semantic graph and verifies each update inside its local typed neighborhood — not against the whole corpus — with a large win over flat-text maintenance.

paper-graphagents.md

Decoupling Memory from Context: Structured Memory for Token-Efficient Test-Time Continual Learning

Does two things: gives memory systems a formal frame — a memory update is an optimization step over the model's context — and shows the flat-file design most systems use has the wrong scaling law.

paper-graphrag.md

Is Grep All You Need? How Agent Harnesses Reshape Agentic Search

Is the strongest evidence that the harness, not the retriever, is the dominant variable in agentic search.

Grep is All You Need: Zero-Preprocessing Knowledge Retrieval for LLM Agents

) is a production systems report: no embedding model, no chunking strategy, no vector database — just grep over raw sources plus nightly LLM-compiled concept/FAQ files.

GrepRAG: An Empirical Study and Optimization of Grep-Like Retrieval for Code Completion

asks the unfashionable question: how far can index-free lexical retrieval — ripgrep — go before you need the expensive machinery? Ingest context: awesome-harness-engineering-papers.

Grimlock: Guarding High-Agency Systems with eBPF and Attested Channels

Is an Agent Guard design: move identity, authorization, provenance, and delegation into the sandbox substrate so high-agency orchestration code stays unchanged.

Do Agent Rules Shape or Distort? Guardrails Beat Guidance in Coding Agents

Runs the first large-scale controlled study of agent rule files: 679 rule files scraped from GitHub (25,532 rules: CLAUDE.md, .cursorrules, skills, plugin manifests, persona definitions), 5,000+ Claude Code / Opus 4.6 runs on SWE-bench Verified.

Handoff Debt: The Rediscovery Cost When Coding Agents Take Over Interrupted Tasks

measures what benchmarks ignore: real work is interrupted, reassigned, and resumed from partial state — and an opaque predecessor imposes a rediscovery tax on whoever takes over. Ingest context: awesome-harness-engineering-papers.

HARBOR: A Harness Framework for Agentic Robot Reinforcement Learning

Extends harness-engineering framing to robotics / manipulation RL.

Harmony Harness: Multi-Objective Self-Evolution of Agent Harnesses

Evolves harnesses along a Pareto frontier instead of a single objective, with factorized operators that separate transferable design from benchmark-specific adaptation — the same Google group as paper-oo-harness.

Harness-1: Reinforcement Learning for Search Agents with State-Externalizing Harnesses

Trains a 20B search / retrieval subagent with RL inside a stateful search harness that externalizes bookkeeping.

Harness Engineering: Anatomy, Architecture, and Evolution of Coding Agents

Is the largest source-code audit of production coding harnesses to date — the empirical floor for what the field actually builds.

Architectural Design Decisions in AI Agent Harnesses

Is a protocol-guided empirical study of 70 public agent-system projects (corpus frozen 23 March 2026).

What makes a harness a harness: necessary and sufficient conditions for an agent harness

Is a definitional paper that answers a question paniolo-scan implicitly faces every day: given a repo artifact, is it harness wiring or something else (a generator, guardrail, framework, eval scaffold, SDK, IDE plugin, orchestrator)?

An Empirical Study of Harness Design for Coding Agents

Harness-IF: Evaluating Instruction Following Across Instruction Surfaces in Coding Agents

asks the uncomfortable question: when a coding agent obeys a rule, was it compliance — or was it going to do that anyway? Ingest context: awesome-harness-engineering-papers.

Natural-Language Agent Harnesses

Asks whether harness design-pattern logic can live outside controller code as a portable, executable artifact.

Where Does Harness-Optimization Value Live? Slot-Level Attribution for Evolved LLM-Agent Harnesses

Decomposes the harness into four separately-evolvable slots and uses leave-one-in/out attribution as a microscope.

Towards Direct Evaluation of Harness Optimizers via Priority Ranking

Argues that evaluating harness optimizers only by downstream agent score gains is indirect: intermediate optimizer actions can be wrong even when end scores look good.

Harness-R1: Learning to Edit Executable Runtime Harnesses from Agent Failure Trajectories

Makes harness editing a *learned* capability: a 9B "harness engineer" model converts failure trajectories into validated executable patches to the runtime — context construction, tool mediation, action validation, recovery — trained by online RL on realized task success.

Self-Evolving Agent Harness

Gives the harness-evolution line its theory paper: separate update target from proposal and admission mechanism, then bound generalization and retention under continual PAC-learning.

HarnessSecurity-Bench: Do Security Mechanisms Really Protect Coding Agent Harnesses?

Is an empirical study of the security settings coding-agent harnesses actually ship, and of what those settings do to attack success and to finishing the user's task.

Harness Updating Is Not Harness Benefit: Disentangling Evolution Capabilities in Self-Evolving LLM Agents

Separates two capabilities that end-to-end evolve scores conflate.

Harness2: Recursive Agent Harnessing for an Open World

solves the open-world problem for harness self-improvement: novel tasks have no ground-truth verifier to guide harness edits — so guide them with a contrastive probe instead, at two levels of recursion.

HarnessDev: Can LLMs Create and Evolve Their Own Agent Harness?

Shifts the unit of evaluation from task outputs to runnable infrastructure: can an LLM author a working harness, and then improve it?

Harnessing Agent Skills: Architectural Patterns and a Reference Architecture for Skill-Mediated LLM Agents

Treats skill harnessing as the architectural layer that turns skill artefacts into run-scoped skill-in-use.

HarnessX: A Composable, Adaptive, and Evolvable Agent Harness Foundry

Is a foundry for typed, composable harness primitives plus a trace-driven evolution engine.

Harness Engineering for Software Engineering via Modular Executable Dev-Primitives

Pairs each repository artifact with a resident LLM so components become active participants: natural-language reasoning about their own implementation, inter-component communication, localized self-modification.

HiL-Bench

Reframes agent evaluation around judgment rather than raw capability: frontier coding agents solve well-specified tasks but collapse when specs are incomplete, ambiguous, or contradictory.

Use and Disuse: Intent-Structured Experience Consolidation for Memory and Learning in LLM Agents

Builds memory and learning into one mechanism: an agent that remembers *by deliberately maintaining its own context*, not by writing to a separate memory bank.

HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models

Is a retrieval framework inspired by hippocampal indexing theory.

Holistic Evaluation and Failure Diagnosis of AI Agents

Attacks the eval gap: outcome metrics report success/failure without explaining *why*, and process-level approaches can't connect failure types to locations in long traces.

Ignored, Not Refused: What Low Prompt-Injection Success Rates Hide in Small Tool-Using Agents

shows ASR is a dishonest security metric: most "defeated" attacks aren't refused — the agent silently ignores them, and a low ASR can hide a paralyzed agent.

INFUSE: A Unified Surface Benchmark for Indirect Prompt Injection in Agentic Systems

evaluates IPI across the whole lifecycle of long-horizon agent workflows — not isolated malicious strings, but attacks injected through realistic interaction surfaces mid-execution.

Many-Tier Instruction Hierarchy in LLM Agents

breaks the instruction-hierarchy assumption that a few rigid privilege levels suffice — real agentic settings have many more instruction sources than the system>user binary can express. Ingest context: awesome-harness-engineering-papers.

Where Instruction Hierarchy Breaks: Diagnosing and Repairing Failures in Reasoning Language Models

localizes instruction-hierarchy failures into three stages — the model may fail to identify the relevant instructions, fail to resolve the conflict, or resolve it correctly in reasoning yet still produce a violating response. Ingest…

Formal Architecture Descriptors as Navigation Primitives for AI Coding Agents

tests whether declaring a project's architecture — module boundaries, symbol signatures, constraints, data flows — cuts the tool calls agents burn on undirected exploration.

The Interplay of Harness Design and Post-Training in LLM Agents

Studies how scaffold choices interact with RL post-training instead of treating them as independent levers.

paper-kb-encyclopedia-sync.md

paper-kb-stability.md

paper-kb-sync-topic-graph.md

λₐ: A Typed Lambda Calculus for LLM Agent Composition

Gives agent frameworks a typed calculus with oracle calls, bounded fixpoints (ReAct loop), probabilistic choice, and mutable environments — cited from paper-categorical-architecture’s bibliography.

Long-Horizon-Terminal-Bench 2.0: Automating Diverse Task Construction and Measuring Sustained Agent Progress

argues binary pass/fail discards the signal long-horizon evals most need: whether the agent is making sustained progress, not just whether the final state passes. Ingest context: awesome-harness-engineering-papers.

LinuxArena: A Control Setting for AI Agents in Live Production Software Environments

builds the largest AI-control testbed: agents work real software tasks inside live, multi-service production environments while a trusted monitor watches for sabotage they attempt on the side. Ingest context:…

Retrieval as Reasoning: Self-Evolving Agent-Native Retrieval via LLM-Wiki

Is the measured version of the Karpathy compile-once pattern.

Observability and Fault Injection for LLM-Based Multi-Agent Systems in Software Engineering

Presents llmmas-otel, a lightweight framework-agnostic tool combining OpenTelemetry distributed tracing with fault injection for LLM multi-agent systems in SE workflows.

Beyond Semantic Organization: Memory as Execution State Management for Long-Horizon Agents

Introduces Mage (Memory as Agent-Guided Exploration): treat agent memory as an execution-state manager, not a similarity-ranked archive.

Model Context Protocol (MCP) Tool Descriptions Are Smelly!

Examines the natural-language descriptions agents rely on to select tools and pass arguments — a critical and largely unlinted surface.

From Docs to Descriptions: Smell-Aware Evaluation of MCP Server Descriptions

Builds the quality standard the MCP ecosystem lacks: server selection and onboarding rely on loosely-constrained free-text descriptions that misrepresent or omit key semantics — driving trial-and-error integration and worse agent behavior.

Which Metadata Matters for LLM Tool Selection in MCP? Evidence from Controlled Factor-Level Interventions

Asks which part of a tool's metadata actually drives selection.

Don't believe everything you read: Understanding and Measuring MCP Behavior under Misleading Tool Descriptions

Studies the gap MCP doesn't enforce: documented tool behavior vs actual code execution, on servers running with broad system privileges.

MCPTox: A Benchmark for Tool Poisoning Attack on Real-World MCP Servers

Is a large-scale benchmark for Tool Poisoning Attack (TPA) on live MCP servers — malicious instructions in tool metadata, with the poisoned tool never executed.

MCRI: A Four-Dimensional Framework for Analyzing and Evaluating Agent Skills

builds an information-gain + behavioral-constraint framework to score skills before running them — because execution-based evaluation is too expensive to apply to the whole library.

Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory

Is the closest thing in this ingest to a production systems paper.

MemArchitect: A Policy Driven Memory Governance Layer

Frames the Governance Gap in persistent agents: standard RAG and most memory architectures treat storage as passive and append-only, so zombie memories (stale or contradictory facts) pollute the context window with no harness-level way to evict, reconcile, or forget them.

MemDerive: A Diagnostic Benchmark for Agent Memory over Evolving Data

exposes what final-answer memory benchmarks hide: agent task state is usually derived — no record states it, it must be computed by rules over an evolving history — and a wrong answer can't be traced to the missed intermediate state.

MemGPT: Towards LLMs as Operating Systems

Treats the LLM context window like OS virtual memory.

MemoHarness: Agent Harnesses That Learn from Experience

Optimizes the harness control layer from execution experience instead of shipping one global configuration for every case.

paper-memory-as-metabolism.md

paper-memory-autonomous-agents-survey.md

MemoryTax: The Bitter Lesson of Agent Memory

Prices designed memory against raw-history search: write-time extraction only pays off for weak models, a store's ceiling is set by the model that wrote it, and an agent searching the raw record tops the cost–accuracy frontier.

Mem++: Non-Destructive Memory for Long-Term Organizational LLM Agents

Identifies the failure mode of write-time distillation: compressing each document into facts/notes/graph edges at ingest fixes what can be answered before any question is asked.

MemTrace: State-Consistent Memory for Long-Horizon Coding Agents

names the coupled failure of long-horizon memory: context strain + repository drift can silently invalidate earlier execution evidence — and solves it with provenance-anchored traces that check validity before reuse.

Agent Harness for Large Language Model Agents: A Survey

At `83bb85f`; snapshotted 2026-07-24) formalizes the agent harness as a six-component architectural object and reviews 110+ sources across 23 systems plus production reports (Stripe, OpenAI, Cursor, METR, …).

Meta-Harness: End-to-End Optimization of Model Harnesses

Is the research paper 3rd-party-repo-meta-harness-superagentic-metaharness|`superagentic-metaharness` is an unofficial implementation of.

MOSS: Self-Evolution through Source-Level Rewriting in Autonomous Agent Systems

Argues that most self-evolving agents only mutate text-mutable artifacts (skills, prompts, memory schemas, workflow graphs) and leave the code harness (routing, hooks, invariants, dispatch) unreachable.

MS-Eval: Benchmarking Sub-Agent Delegation and Orchestration in MCP-Intensive Working Scenarios

measures the Main-Agent Delegation / Sub-Agent Execution (MAD-SAE) paradigm — orchestration quality as its own evaluand, not just end-task success.

Multi-Agent Memory from a Computer Architecture Perspective: Visions and Challenges Ahead

Is a position paper that borrows computer-architecture framing (cache coherence, memory hierarchies) for multi-agent memory.

NOMOS: Compiling Written Policies into Statically Verified Tool-Call Gates for LLM Agents

compiles a natural-language policy into a deterministic tool-call gate — and shows that naive compilation produces inoperable rules (a rule blocks the tool that satisfies its own precondition, or reads arguments its tool doesn't have),…

NVIDIA-labs OO Agents: Native Python Object-Oriented Agents

Collapses agent development into one Python object: methods are actions, fields are state, docstrings are prompts, type annotations are contracts.

Verifying Coordination in Parallel Coding Agents: NP-Bench and a Scheduling Planner

names the failure single-agent evals can't see: each parallel agent passes its own tests while the merged result is broken — and reframes coordination as up-front scheduling, not reactive conflict detection.

OctoBench: Benchmarking Scaffold-Aware Instruction Following in Repository-Grounded Agentic Coding

Measures whether coding agents follow *scaffold-specified* instructions — the heterogeneous, persistent constraints that AGENTS.md-style files, system prompts, and repo policies impose.

Learning to Control LLM Agent Harnesses with Offline Reinforcement Learning

Treats the execution harness as a learnable control layer, not only a static artifact optimized before run time.

One Skill Too Many: How Co-Installed Skills Conflict in Coding Agents

Is the first empirical study of what happens when two installed skills do the same job.

Object-Oriented Harness: Evolving Agent Harnesses through Object-Oriented Design

Evolves a hierarchy of executable harness classes — subclasses that inherit, override, or extend a base harness — then routes each task to the right class at test time.

Before the Tool Call: Deterministic Pre-Action Authorization for Autonomous AI Agents

Frames the gap it fills as: "AI agents today have passwords but no permission slips" — agents commonly hold credentials (identity) but no standardized, machine-checkable statement of _what they're allowed to do with a given tool call before they do it_.

Building AI Coding Agents for the Terminal: Scaffolding, Harness, Context Engineering, and Lessons Learned

Is the first open technical report focused on a production terminal-native interactive coding agent — not a benchmark harness (SWE-Agent) or browser UI (OpenHands).

OpenPort Protocol: A Security Governance Specification for AI Agent Tool Access

Specifies OPP: a governance-first server-side gateway for exposing application tools to agents — model/runtime-neutral, bindable to existing ecosystems including MCP.

paper-oracles-that-cannot-fail.md

Organizational Memory for Agentic Business Process Execution

Is one of the few papers in this ingest about shared, governed knowledge across many agents, not a single agent's personal memory.

Package Hallucination Attacks on Coding Agents through Prompt Injection in Rule Files

injects adversarial prompts into community-shared rule files (AGENTS.md, .cursorrules) so the coding agent swaps a legitimate dependency for an attacker-controlled package — slopsquatting delivered by the config layer.

paper-page-refresh-policies.md

paper-paste.md

PhoneHarness: Harnessing Phone-Use Agents through Mixed GUI, CLI, and Tool Actions

Extends harness engineering to phone-use agents that must choose among GUI, CLI, and structured tool actions.

Runtime Governance for AI Agents: Policies on Paths

Argues the execution path — not isolated actions — is the central object for runtime governance.

Harness Engineering for Language Agents: The Harness Layer as Control, Agency, and Runtime

Is an academic framing of the harness layer as a distinct, reportable object in language-agent research.

Probe-and-Resume: When and How to Evaluate Agent Execution

finds MCTS-style test-time search fails in terminal environments because critics can't see enough state at intermediate nodes — and fixes it by only evaluating where the agent claims it's done, giving the critic a fork it can probe…

Is Progressive Disclosure All You Need for Long-Context Agents?

Is the first controlled study of the pattern behind Agent Skills: give the agent a short description and a path, let it fetch depth on demand.

ProPerAgent: Toward Memory-Driven Proactive Assistance

reframes memory's role for proactive agents: memory doesn't just supply answers — it decides whether and when to interrupt at all. Ingest context: awesome-harness-engineering-papers.

Event-Centric Memory with Query-Aware Graph Augmentation for Long-Term Conversational Agents

Splits the difference between the two memory paradigms: flat storage is cheap but leaves event relations implicit; persistent graphs expose relations but cost construction and accumulate irrelevant edges.

RAISED: Self-Distillation for Robustness to Prompt Injection in LLM Agents

diagnoses why training-time IPI defenses degrade the model — they shift the output distribution even on benign inputs, and specifically make the agent refuse steps that a legitimate tool output asks for — then fixes it with…

Does My README File Need To Be Updated? Exploring LLM-Based README Maintenance

Frames documentation maintenance as a surgical update task inside the pull-request workflow: for each PR, decide whether a README update is needed, locate the precise paragraphs, and justify via the triggering event — a bot recommends, the developer decides.

Persistent Context Graphs for Efficient Memory Compaction in LLM Agents

Observes that every forward pass already computes attention over the history — and then throws it away.

Recoverability as a System Primitive for Long-Horizon AI Agents

argues a saved state is not necessarily a suitable place to resume — and makes reuse an explicit, evidence-bound decision: select a supported starting point and a permitted recovery action, or withhold automatic continuation.

ReCreate: Reasoning and Creating Domain Agents Driven by Experience

Critiques prior search-based ADAS methods for treating agent generation as a black box steered mainly by coarse scalar scores, without reasoning over interaction experience when updating the scaffold.

Recursive Agent Harnesses

Names and measures the pattern between recursive language models (RLMs — recursion over bare model calls) and production agents spawning subagents: the Recursive Agent Harness (RAH), where the recursive unit is a *full agent harness* — filesystem tools, code execution, planning — not a contextless model call.

In Prospect and Retrospect: Reflective Memory Management for Long-term Personalized Dialogue Agents

Is about revising stored memory over time, not only appending it.

ReFold: Training-Free Reversible Inter-Turn Context Folding for Long-Horizon Agents

Is a rendering layer, not a compressor: it preserves the append-only interaction history while compressing only what the model sees.

Towards a Science of AI Agent Reliability

Argues that compressing agent behavior into a single success-rate metric hides the operational flaws that make agents fail in practice: whether they behave the same way twice, whether they withstand small perturbations, whether their confidence tracks their accuracy, and how badly they fail when they do fail.

REM: What to Store, When to Recall, What to Forget in Experience Memory for Coding Agents

argues experience-reuse's bottleneck isn't retrieval — it's lifecycle management: what to store, when to recall, what to forget, decided from inside the agent's think-act loop.

Remory: Learning Residual Memory for Context Compaction

Targets the information a textual summary *can't* carry.

RepoDoc: A Knowledge Graph-Based Framework to Automatic Documentation Generation and Incremental Updates

Puts a repository knowledge graph (RepoKG) at the center of the whole documentation lifecycle — unlike approaches that treat code as flat text chunks or file structure.

Resume Means Resume: A Machine-Checked Conformance Contract for Checkpoint, Interrupt, and Resume Semantics in Workflow Persistence Layers

states what "resume" must mean — then measures five deployed workflow frameworks and finds each violates even the fragments of the contract they claim.

Rethinking the Evaluation of Harness Evolution for Agents

Is the primary pushback / rigor paper for automatic harness evolution.

Robust and Efficient Tool Orchestration via Layered Execution Structures with Reflective Correction

— RETO — argues tool failures often come from how tools are organized, not single-call errors.

Retrieval-Oriented Code Representations in Agentic Bug Localization

isolates the variable everyone else conflates: what text you retrieve over matters as much as how you retrieve — the representation is a first-class design choice.

Evolving Agents in the Dark: Retrospective Harness Optimization via Self-Preference

Introduces Retrospective Harness Optimization (RHO): improve the full agent harness — skills, tools, workflows, prompts — using only past unlabeled trajectories, with no ground-truth validation set.

RippleCP: Measuring Counterfactual Checkpoint Advantage in Coding Agents

prices checkpoint placement: existing checkpoint systems decide what state is recovery-relevant and how to snapshot it, but none decides which of the safe boundaries they expose are actually worth materializing.

Harness Self-Evolution for Long-Horizon Terminal Agents

targets the reliability gap: an agent that solves a task sometimes but not consistently — and closes it by evolving the harness, not the model.

SafeHarness: Lifecycle-Integrated Security Architecture for LLM-based Agent Deployment

Argues prior harness engineering optimized performance while under-weighting security, then embeds defenses inside the harness lifecycle.

Returning to Development States: When Full Sandbox Snapshots Help Coding Agents

tests whether branching from full environment snapshots beats retry-from-scratch — and finds the gains live in the state a git diff can't capture.

Sandlock: Confining AI Agent Code with Unprivileged Linux Primitives

builds a lightweight Linux sandbox for agent-generated code organized around a clean split: static, input-independent policy compiles into kernel-enforced rules; a narrow supervisor handles only runtime-dependent decisions. Ingest…

AgentLens: Revealing The Lucky Pass Problem in SWE-Agent Evaluation

Argues that binary pass/fail is insufficient for SWE-agent evaluation and trajectory curation: agents that reach correct patches through regression loops, blind retries, or missing verification are scored the same as principled solutions.

Harness-Bench: Measuring Harness Effects across Models in Realistic Agent Workflows

Introduces a diagnostic benchmark where the primary measured unit is the model–harness configuration, not the base model alone.

StaminaBench: Stress-Testing Coding Agents over 100 Interaction Turns

Measures stamina: how many consecutive user change-request turns a coding agent sustains before failing — not single-shot task solve rate.

Scanning the Harness: An Empirical Study of Supply-Chain Defects in AI Coding-Agent Configurations

treats the agent harness itself — instruction files, skills, commands, hooks, MCP declarations, subagent definitions — as a dependency layer with no lockfile, no install-time check, and no permission vocabulary, and measures what that…

Scrouting: Cost-Aware Routing of Coding Agents by Scouting the Repository First

Fixes routing's information deficit: existing routers pick a model from the *issue text alone*.

SEA-Eval: A Benchmark for Evaluating Self-Evolving Agents Beyond Episodic Assessment

Formalizes the Self-Evolving Agent (SEA) and the Evolutionary Flywheel, then introduces SEA-Eval as a benchmark beyond episodic success rate.

SEAGym: An Evaluation Environment for Self-Evolving LLM Agents

Builds an evaluation environment (not just a scoreboard) for measuring agent harness updates across train / validation / test / replay / cost records.

SearchWiki: Learning to Build and Navigate Knowledge Wikis for Active Information Seeking

Compiles a corpus into a typed wiki and then trains a small agent to walk it.

Self-Harness: Harnesses That Improve Themselves

Is a same-model harness self-improvement loop: the agent under evaluation mines its own failure traces, proposes minimal harness edits, and promotes only edits that pass regression tests — without human engineers or a stronger external proposer.

Self-Evolving Coding Agents via Harness-Policy Optimization

argues harness and policy are mutually reinforcing — a harness tuned for one checkpoint doesn't target the updated policy's failures, and a fixed harness limits what trajectories the policy can learn from — so optimize them alternately.

Toward Training Superintelligent Software Agents through Self-Play SWE-RL

Proposes Self-play SWE-RL (SSR): train a software agent via RL that iteratively injects and repairs bugs in sandboxed repos without human-labeled issues or tests.

SemaClaw: A Step Towards General-Purpose Personal AI Agents through Harness Engineering

Responds to OpenClaw adoption at personal-agent scale by open-sourcing a multi-agent framework where the harness layer — not model choice — carries controllability, auditability, and extensibility.

Distributing Security Controls Through Harness Engineering

Asks whether off-the-shelf controls can be packaged into one harness and installed for a team, instead of configured agent by agent.

SIA: Self Improving AI with Harness & Weight Updates

Bridges two previously disjoint self-improvement lines.

Skill Blocks: How Should an Agent Load Its Skill?

Is the caching-correct comparison of skill-loading strategies: Full (inject everything), Skill Block, Reference (compact catalog + on-demand files), Hybrid (short stubs fetched on demand).

Agent Skill Evolution: How Revisions Affect Coding Agents

Measures what a SKILL.md *revision* does — the first large-scale study of skills as evolving files rather than fixed artifacts.

Skill-Use: Can LLMs Actually Use Skills in Agentic Harnesses?

Asks whether agents can find and follow skills at all under progressive disclosure — and finds reliable skill use is still out of reach.

SkillJuror: Measuring How Agent Skill Organization Changes Runtime Behavior

) holds skill *content* fixed and varies only *organization* — flat file vs progressive disclosure — and measures what changes.

SkillReducer: Optimizing LLM Agent Skills for Token Efficiency

An Empirical Study of Agent Skills' Downstream Utility

Asks the question every skill marketplace quietly avoids: does supplying a *relevant* skill actually help?

How Agent Skills Fail under Long Contexts: A White-Box Study in Code Auditing

Holds a skill-based task fixed and varies only the surrounding context, asking: when a loaded skill's requirements stop being honored mid-trajectory, where does the failure first become visible?

SkillScriptBench

Benchmarks what prior skill evaluations conflate: revising an *executable* skill package means repairing scripts, repairing documentation, and preserving what already worked — three different capabilities.

SkillTester: Benchmarking Utility and Security of Agent Skills

Defines a comparative quality-assurance harness for third-party agent skills at selection and enablement time.

Socratic-SWE: Self-Evolving Coding Agents via Trace-Derived Agent Skills

closes the self-evolution loop: the agent's own solving traces distill into skills that summarize its recurring failures — and those skills drive generation of targeted repair tasks. Ingest context: awesome-harness-engineering-papers.

SpecFirst: Behavioral Specification Elicitation as a First-Class Step in Agent-Based Program Synthesis from Scratch

Targets from-scratch program construction: given only natural-language documentation and an execute-only binary as behavioral oracle, frontier models solve fewer than 1% of ProgramBench instances when documentation reading, behavioral exploration, and synthesis are conflated into one pass.

SpecGuardBench: Can Coding Agents Construct Reliable Validation Harnesses from Natural Language Specifications?

Asks whether agents can build the *checker*, not just pass the check.

SpecPath: Testing Coding Agents Across Contract-Equivalent Specification Histories

Identifies specification-path sensitivity: requirement histories that are equivalent in their final meaning lead the same agent system to produce behaviorally different programs.

SquidAgent: Parallelize Wisely, Coordinate Efficiently

explains why parallel multi-agent systems often run slower than a single agent — two hidden costs serial execution avoids — and derives a decision criterion from it.

StateHarness: Maintaining Evolving Project State for Long-Horizon Coding Agents

Keeps a structured, evolving record of project state — current problem, hypotheses, findings, decisions, repo relationships, unfinished work — so local edits stay connected to their later consequences.

paper-storm.md

paper-streaming-knowledge-compilation.md

SwarmAgentic: Towards Fully Automated Agentic System Generation via Swarm Intelligence

Targets full autonomy in agentic-system generation: build systems from scratch (no hand-coded seed scaffold), and jointly optimize agent functionality and collaboration as interdependent parts via language-driven exploration.

SwarmResearch: Orchestrating Coding Agents for Open-Ended Discovery

Diagnoses why long-running coding agents underperform on open-ended optimization: they converge on one high-level approach, then grind low-level edits.

SWE-Bench Pro V2: A Cleaner, Harder-to-Game Coding Benchmark

treats benchmark rot as inevitable: as agents improve they reach task defects earlier versions never touched and interact with the eval environment in ways the original harness didn't anticipate — so the benchmark itself needs a repair…

SWE-Replay: Efficient Test-Time Scaling for Software Engineering Agents

replaces sample-from-scratch test-time scaling with trajectory recycling: archive prior trials and branch new attempts at critical intermediate steps rather than re-running the whole trajectory.

Opinion: Coding-Agent Benchmarks Should Match Their Users' Task Flows

measures what real sessions look like and finds issue-derived benchmarks are structurally unrepresentative: real users mix task types and switch between them constantly. Ingest context: awesome-harness-engineering-papers.

Arbiter: Detecting Interference in LLM Agent System Prompts — A Cross-Vendor Analysis of Architectural Failure Modes

treats production system prompts (245–1,490 lines) as software artifacts with no type checker, no linter, no tests — and runs the equivalent of a compiler pass over them.

From Model Scaling to System Scaling: Scaling the Harness in Agentic AI

Argues the field scaled model size extensively but under-studied the harness as a controllable, reproducible object of study in its own right.

Task-to-Model Optimization for Enterprise LLM Coding Assistants

Is a data-driven framework for production model routing: enterprise coding assistants burn inference spend, and naive token-cost minimization *fails to reduce end-to-end cost* once retries, escalations, and developer wait time are priced in.

Beyond Retrieval: Evidence Access and Event Aggregation in Agent Memory

splits memory access into two jobs — source-preserving evidence access (keep the original trajectory, build a reading view at question time) plus executable aggregation (a deterministic event ledger for questions that require combining…

tap: A File-Based Protocol for Heterogeneous LLM Agent Collaboration

lets agents from different vendors collaborate on one codebase with no shared runtime, no central server, no common API — the file system is the message bus. Ingest context: awesome-harness-engineering-papers.

paper-test-smell-detection-tools-mapping-study.md

TestJack: Agentic Coding Benchmarks Auditing via Evaluator Evolution

Audits benchmark-passing trials rather than trusting fixed test suites.

TestPrism: Rethinking Test Evaluation Beyond a Single Reference

Attacks the convention that a generated test is good if it fails on the buggy state and passes on *the* reference solution.

paper-text-obsoleteness.md

Towards Lifelong Dialogue Agents via Timeline-based Memory Management

Argues that outdated memories are a feature.

Learning to Evolve: A Self-Improving Framework for Multi-Agent Systems via Textual Parameter Graph Optimization

Introduces Textual Parameter Graph Optimization (TPGO) for multi-agent systems (MAS).

Training a Knowledge Base: Supervised Structure Learning for Agent-Curated Document Stores

Treats the store as the model.

TrajAudit: Automated Failure Diagnosis for Agentic Coding Systems

Targets repository-level coding trajectories specifically: they're long, noisy (redundant program structure, verbose code context), and land squarely on LLMs' long-context weakness.

TrajDebug: Tracing Error Lifecycle to Identify Critical Failures in Long-Horizon Agent Trajectories

Targets critical error detection: locating the earliest error step actually *responsible* for the final failure — which is neither the first local error nor the one temporally closest to the end.

Maximising Transparency in Context Compaction Improves Retention of Facts and Commitments for Lightweight Coding Agents

Studies compaction for *lightweight* models (local, high-volume workloads) where the usual model-written summary is weakest: a summary selects and rewords, and the agent cannot see what was lost.

VACE: Validation-Gated Alternating Co-Evolution of Agent Models and Harnesses

Alternates agentic RL with trajectory-driven harness refinement — and lets a candidate harness guide further training only if it beats the incumbent on validation.

VERSE: Verified Self-Evolving Optimizer for Agent Harnesses

Evolves the optimizer itself: the agent that edits another agent's harness gets better at diagnosing failures, developing edits, and testing their effects — under execution-based verification.

KARMA: Leveraging Multi-Agent LLMs for Automated Knowledge Graph Enrichment

Is a close research analog for `wiki-io` because it treats knowledge-base maintenance as a graph enrichment workflow, not a one-shot document summarization pipeline.

paper-wiki-llava.md

paper-wikibigedit.md

WikiLoop: Jointly Learning to Build and Navigate Agent-Native Wikis with Downstream Feedback

Starts from the split paper-llm-wiki-retrieval-as-reasoning leaves in place: the wiki is built, then a separate agent reads it.

paper-wikimonitor-onto.md

WikiSkill: Compiling Agent Experience into Persistent Knowledge for Skill Evolution

Puts a wiki between raw traces and the skills an agent is allowed to keep.

WildClawBench: Evaluating Long-Horizon Agents in Real Runtimes

Benchmarks agents inside actual deployed CLI harnesses — OpenClaw, Claude Code, Codex, Hermes — on real tools, not mock services.

Wink: Recovering from Misbehaviors in Coding Agents

Is a production system: a lightweight, asynchronous self-intervention layer that observes agent trajectories and nudges the agent back to a productive path — rather than restarting or waiting for failure.

Diagnosing Live Within-Policy Instruction Conflicts in LLM Agents with Witnessed Resolution Profiles

finds that individually reasonable standing rules collide inside a single prompt policy — and measures how badly models resolve the pressure.

WiseSpec: Requirements-Driven Agents for Code Generation

Argues that existing work improves coding agents through better tools, skills, and workflows while overlooking the quality of the *task requirements themselves* — which are frequently incomplete, ambiguous, or missing critical context.

Zoro: Active Rules for Reliable Vibe Coding

Argues rules files are *passive*: nothing shows when they're used, followed, or need improvement.