Layering
One shared source of truth (AGENTS.md, canonical rules) wired into each tool through thin adapters — instead of six diverging instruction files.
Free to use — Diagnostic command
The AI
technical debt
scanner.
Every repeated agent mistake traces back to a gap in your repo's guidance,
enforcement, or validation surfaces. paniolo scan
finds those gaps — across Copilot, Cursor, Codex, Antigravity, Claude Code, and
Gemini — and scores them. It reads your repo; it never writes a byte.
# From any repo root — no install, no config, no writes
npx @paniolo/cli scan
28.0%
Of agentic-AI faults are schema mismatches at the LLM boundary
Fault taxonomy, arXiv:2603.06847
74.6% → 0%
Attacker success when a deterministic policy gate replaces a permissive one
Open Agent Passport CTF, arXiv:2603.20953
53–82 pp
Capability drop on under-specified tasks without an escalation protocol
HiL-Bench, arXiv:2604.09408
Three Repos, One Story
Diagnose
paniolo-ai/paniolo-scan — the paniolo scan CLI. Scores and findings, no file writes.
Remediate (optional)
paniolo-ai/scan — thin agent adapters (skill, slash command, workflow) that run the scan and fix findings on request.
Set up & Evolve
paniolo init and evolve — adopt or connect a shared harness and keep its integrations current.Six dimensions of
harness quality
Not prompt-file style. The scan evaluates whether your whole repo — across every tool your team uses — is engineered for coding agents: shared guidance, thin adapters, discoverable skills, enforcement gates, and guardrails.
Sharing
How much guidance lives once vs. duplicated per harness, in lines and percent. Duplicated guidance drifts, and drift wastes always-loaded context.
Discoverability
Can an agent actually find your skills, agents, and rules? Indexes, routing tables, and search scripts are scored, not assumed.
Harness wiring
Per-tool config surfaces — VS Code skill locations, Codex hooks, Claude settings — checked for presence and validity.
Maintainability
Guidance stays bounded, linked, indexed, and free of duplicate trees — so it keeps working as the repo grows.
Guardrails
Hooks, permissions, CI gates, and security hygiene: the deterministic enforcement that keeps an agent honest when prose fails.
Research in.
Rules out.
Each feature below started as a finding in the published literature and shipped as a deterministic check. The cadence is research → rule → release.
Scoring
Continuous scores, honest scopes
Six scoring curves replace pass/fail gates: 2,500 words against a 2,000-word budget scores 0.8, not "FAIL." Dimensions that don't run report "status": "not_run" with a null score — never zero — and scoreScope records whether a score covers core dimensions or core plus extended, so scores stay comparable across repos.
Security hygiene
Nine deterministic guardrail checks
Blanket auto-approve permissions, hook scripts that call non-local hosts, pull_request_target workflows, unpinned Actions, missing secret scanning, loop-prone Stop hooks, broken hook wiring, and unignored .env files — the findings that matter most are the ones easiest to miss.
LLM boundary
Scans the probabilistic/deterministic seam
The top recorded root cause of agentic-AI faults — 28.0% of a 385-fault coded sample — is LLM/tool output feeding typed code without validation. Three rules check schema validation, model-interface pinning, and tool contract tests in your source, not just your prompt files.
28.0% / 11.6% — Agentic-AI fault taxonomy, arXiv:2603.06847
Security posture
Declared defenses, proportionate to capability
Six conservative checks — escalation protocol, resource caps, tool allowlists, untrusted-input boundaries, memory-write provenance, high-impact-action confirmation — grounded in a USENIX Security 2026 survey's risk taxonomy. They verify the protocol is declared, not that an agent exercises judgment.
Risk grid — Attack/defense survey, arXiv:2603.11088
Calibration
Published thresholds, not vibes
Every default has a documented derivation, anchored where possible to production-validated research numbers, and locked by weak/normal/mature calibration fixtures in CI. Reference values measure deviation; they are not pass/fail gates.
Anchors — Codified Context, arXiv:2602.20478 · How we calibrateFrom an actual scan,
not a mockup
Excerpt from a real @paniolo/[email protected] run against Paniolo's own harness repository. Seven harnesses detected, score 100, with the dimensions that did not run reported as not run.
{
"version": "0.5.54",
"totalScore": 100.0,
"scoreScope": "core",
"harnesses": ["copilot", "cursor", "codex", "antigravity", "claude", "gemini", "devin"],
"metaHarness": {
"dimensions": {
"discoverability": { "status": "run", "score": 100, "grade": "excellent", "passedRules": 12, "applicableRules": 12 },
"guardrails": { "status": "run", "score": 99, "grade": "excellent", "passedRules": 44, "applicableRules": 45 },
"harnessWiring": { "status": "run", "score": 100, "grade": "excellent", "passedRules": 3, "applicableRules": 3 },
"layering": { "status": "run", "score": 100, "grade": "excellent", "passedRules": 7, "applicableRules": 7 },
"maintainability": { "status": "run", "score": 100, "grade": "excellent", "passedRules": 27, "applicableRules": 27 },
"sharing": { "status": "run", "score": 99, "grade": "excellent", "passedRules": 15, "applicableRules": 16 },
"session": { "status": "not_run", "score": null, ... }, // not run ≠ zero
"deep": { "status": "not_run", "score": null, ... }
}
},
"summary": { "error": 0, "warn": 0, "info": 1 }
}And one finding, as the terminal prints it (from a small test repository with an unpinned action):
[warn] actions-sha-pinned
.github/workflows/ci.yml uses actions pinned to mutable refs: actions/checkout@v4.
Location: .github/workflows/ci.yml
Hint: Pin each action to a full 40-character commit SHA (e.g. actions/checkout@<sha>) to prevent tag-hijack supply-chain attacks.Pick your
first scan
Solo developer
See what your agents are missing
One command from the repo root scores your guidance across every coding agent it detects and lists each gap with a file, a reason, and a fix hint.
npx @paniolo/cli scan .
Team lead / platform
Stop guidance drift in CI
Cross-harness sharing analysis and context budgets show where six tools' instructions duplicated and diverged. JSON output and exit codes drop into any pipeline.
npx @paniolo/cli scan --format json --fail-on error
Security / AppSec
Audit the agent attack surface
Hooks that call external hosts, blanket auto-approve, secret leaks, privilege-escalating workflows — deterministic checks grounded in the published risk taxonomies.
npx @paniolo/cli scan --fail-on warn
Staff+ / architect
Interrogate the methodology
Every threshold has a published derivation, and fixtures lock the bands in CI. Read how the scores are made before you trust them.
How we calibrate →Before you ask
Does the scan modify my repo?
No.
scan is diagnostic only: no autofix and no file writes. Other Paniolo commands, such as init, do write files; scan never does, which is what makes it safe to run on any repo.Does the scan call an AI model?
No. Every check is deterministic, runs locally, and gives the same result for an unchanged tree. Nothing is sent to a model. The default scan makes no network requests; the one network check, dead external links, runs only with
--external-links.Who decided these thresholds?
Each one has a documented derivation — research anchors where the literature provides them, reference-repo calibration where it doesn't — and a fixture suite that fails CI if a weak repo shape starts scoring like a mature one. Full story on the calibration page.
How is this different from AgentLint or agnix?
They're good tools, and we say so. The short version: paniolo-scan measures cross-harness sharing and always-loaded context budget (which no peer reports), scans application code at the LLM boundary, and stays strictly diagnostic. The longer, fair version is on the comparison page.
Skill, slash command, or CLI?
CLI for CI and headless use. The portable skill (
npx skills add paniolo-ai/scan --all) for any coding agent. The Claude Code plugin if you specifically want the native /paniolo-scan command. They're separate installs — the docs have a decision tree.Goodwill service
paniolo scan and the remediation prompts are free to use. Each finding says what to change and why, so you can fix it yourself or hand it to your coding agent.
Sixty seconds to your first score.
npx @paniolo/cli scan — diagnostic only, safe anywhere