Verification Requirements
Without Foundry installed, every finding is labeled needs_review. The PoC is still generated and saved, but it is not compiled or executed. You must install Foundry to get executable verification.
SIREEN's coverage is narrow and deliberately so. There are three distinct levels, and mixing them up is easy:
- Detected by heuristics and fully proven with a generated PoC: reentrancy, missing access control, denial of service (unbounded loop), and business-logic / accounting inconsistency.
- PoC templates exist but no heuristic currently raises these hypotheses: arithmetic overflow/underflow, flash loans, oracle manipulation. They can only be reached through an AI-assisted hypothesis, and that path has not been evaluated.
- Hypotheses may be raised but are then explicitly skipped: delegatecall, signature replay, weak randomness. No PoC template exists, so SIREEN skips the test rather than reporting a fake pass.
Governance attacks, MEV, and sandwich attacks are not supported at all — SIREEN does not look for them. Verified against the v0.1.1 source on 2026-10-15.
AI-Assisted Mode
The AI pipeline is wired end-to-end and functional, but we have not systematically measured whether AI-generated hypotheses produce better, worse, or different findings than heuristic-only mode. Use it experimentally.
When AI mode is enabled, contract source is sent to OpenRouter's API. Do not use AI mode on sensitive or proprietary code.
Findings and Accuracy
Business-logic findings (e.g., "accounting inconsistency") are particularly noisy. Any function that updates multiple state variables may trigger this heuristic. Always review needs_review findings manually.
Severity labels (CRITICAL, HIGH, MEDIUM, LOW) are estimates based on the attack vector type. They are not CVSS scores and should not be used for formal risk assessment.
A contract that returns clean_with_coverage has no findings within SIREEN's current scope and heuristics. It does not mean the contract is secure, formally correct, or free of out-of-scope vulnerability classes.
Platform and Environment
The extension builds successfully (125 KB, 11 files) and its code has been inspected for engine parity with the CLI. However, it has not been installed or activated in a real VS Code environment. Runtime behavior is unknown.
SIREEN is tested on Linux and macOS. WSL should work. Native Windows compatibility is unknown.
Summary Table
| Capability | Status | Notes |
|---|---|---|
| Reentrancy PoC generation | Working | Verified with Foundry |
| Access control PoC generation | Working | Verified with Foundry |
| DoS PoC generation | Working | Verified with Foundry |
| Business-logic / accounting PoC | Working, high false-positive rate | Manual triage required |
| Arithmetic / flash loan / oracle PoC | Template only | No heuristic raises these vectors; reachable only via AI mode |
| Delegatecall / signature replay / weak randomness | Skipped | Hypothesis may be raised, then test is skipped — never a fake pass |
| Forge execution | Working | Requires local Foundry install |
| Heuristic analysis | Working | Fully offline |
| AI-assisted mode | Wired, unevaluated | Requires OpenRouter key |
| VS Code extension | Built, untested | Manual .vsix install only |
| Governance / voting exploits | Not supported | SIREEN does not look for these |
| Vyper/Move/Cairo | Not supported | Solidity only |