AgentRE-Bench

All projects

Apps & Tools

A deterministic benchmark for AI agents reverse-engineering Windows PE binaries

byAgentRE-Bench

PlayableOpen source

AgentRE-Bench media is blocked

Allow external media to connect to the provider and play this content.

Description

A research framework evaluating how frontier AI agents perform static analysis on paired stripped and unstripped Windows binaries. The benchmark uses deterministic scoring across 120 evaluation episodes to track model performance in identifying behavior, recovering configurations, and tool planning without relying on LLM judges.

Descriptions, tags, and model credits may be AI-generated or inferred from public sources and can be incomplete or wrong. Learn more.

Details

Why this is interesting: It establishes a rigorous, deterministic methodology for testing AI agent capabilities on complex binary analysis tasks.
Attributed to: @agentrebench
Built with: Opus 5 Kimi K3 GPT-5.6 Sol Opus 4.8 GPT Image 2 Opus 4.7 Opus 4.6 Opus 4.5 GPT-5 Codex GPT-5
Links: agentre-bench.ai
View tweet
View GitHub repo
Tags: #benchmark #reverse-engineering #security #static-analysis

Discovered: X
First posted: X · Aug 4, 2026
Listed: Added 45d ago
Build: Strong
Models: Strong

AgentRE-Bench

Description

Timeline

AgentRE-Bench

Description

Timeline