Skip to content
Ironheights

Limitations

No scanner gives complete protection, and we would rather you hear that from us. Here is what Ironheights 0.3.0 does not do.

Quick answerLast updated

What can Ironheights not detect?

Ironheights cannot see novel or obfuscated attacks its rules do not describe, a compromised host, or social engineering outside the files. Its optional guard plugin is not a sandbox. It reports skipped files and directories as incomplete (exit code 3), not clean. No findings means no rule matched, not that a skill is harmless. Read the skill too.

What it cannot detect

  • Novel attacks, and attacks obfuscated in a way the current rules do not describe.
  • Runtime-only behavior that appears after a script is executed. The scanner reads files; it does not watch processes or network traffic. The optional guard plugin watches a short list of OpenClaw tool calls from inside the agent. It is not a sandbox, a compromised agent can switch it off, and a quiet log is not proof of safety.
  • A host that is already compromised, including a baseline an attacker can rewrite. Anyone who can write your home directory can edit the baseline file, unless you sign the baseline with a key kept somewhere they cannot reach, and even a signed baseline does not help against an attacker who also has the key.
  • Social engineering that never lands in a file the scanner reads.
  • Files larger than 1 MiB. The CLI skips any file over the size limit (limits.maxFileBytes, 1,048,576 bytes by default) without reading it, and it does not enter .git or node_modules. The report names each skipped file and directory and the verdict is incomplete with exit code 3, so a padded file no longer passes as clean, but its content is still not scanned. dist/ is scanned. Pass --allow-skipped only if you accept that, or raise limits.maxFileBytes in your config if your skills contain large files. The browser scanner shows the same incomplete result when it skips a file for size. Use ignoreDirs with a written reason to acknowledge .git or node_modules so they no longer make a scan incomplete.

What a verdict means

no-findings means the rules did not match. It does not mean the skill is safe. incomplete means a file or directory was skipped and was not checked at all. The A to F grade is only a summary of the risk points the rules added up; it is not a safety rating, and a scan that skipped anything is graded incomplete. review and block can also be false positives; tune them with ruleOverrides and allowDomains.

Scope today

  • scan --all includes OpenClaw bundled skills, custodian skills, and the directories behind ~/.openclaw/plugin-skills symlinks. Bundled skills do not declare their hosts yet, so a stock install still reports their API hosts.
  • The advisory OpenClaw skill runs inside the agent, so a hostile skill can try to bypass it.
  • The guard is not a sandbox. The guard plugin runs inside the OpenClaw process and sees only the tool name and parameters OpenClaw passes in. A compromised skill that can edit your OpenClaw config, the policy file or the plugin can switch it off. Monitor mode, the default, only logs. A quiet log is not proof that nothing happened.
  • A signed baseline only helps while the key stays private. Anyone who can write your home directory and also has the key can sign a new baseline. Keep a copy of the key off the machine if you can.
  • Advisory feed support is in the CLI, but the feed is not published yet, so advisories update has nothing to download and scans report nothing from it. A skill missing from a feed is not evidence that it is harmless.
  • The browser scanner runs the content rules from the 0.1.5 engine. MCP, advisory, config audit, grade and guard checks need the CLI.
  • Windows support is new in 0.3.0. It is tested in CI on Node.js 20.0.0 and 24, which is not the same as years of use.

Benchmark

The benchmark corpus in the repository is synthetic: harmless text that matches rules. A public comparison against other scanners on known malicious samples is in progress, and we will publish the method and the misses along with the results. See the benchmark documentation.