SEMPRE automatically reconstructs protocol structures with high accuracy
The claim
The authors claim SEMPRE achieves >0.4 accuracy on structure recovery and 0.72 on semantic inference for protocol reverse engineering
Published as: “SEMPRE: Semantic-Aware Protocol Reverse Engineering via Consistency-Driven Joint Optimization”
So what
The approach can dramatically improve automated analysis of undocumented or malware protocols, enabling more reliable generation of IDS signatures and vulnerability discovery.
The five-minute version
SEMPRE achieves an average F1 of 0.76 for field detection and 0.72 for semantic inference, far surpassing prior tools (no other tool exceeds 0.4 on detection and BinPRE scores 0.14 on inference). The authors evaluate on eight public protocols, three undocumented protocols, and a Mirai botnet protocol, comparing against six PRE systems. The evidence is strong in terms of reported F1 scores, but the paper does not provide detailed message counts or statistical significance testing. The authors acknowledge that convergence of their iterative process is only conditionally guaranteed and depends on reliable high-confidence LLM predictions.
What they found 2
- SEMPRE attains the highest average field-detection F1 of 0.76, with no other tool exceeding 0.4
“In field detection, SEMPRE attains the highest average F1 score of 0.76, demonstrating a 90%+ improvement with no other tools exceeding 0.4.”
- SEMPRE achieves semantic-inference F1 of 0.72, far above BinPRE's 0.14
“In semantic inference, SEMPRE achieves an average F1 score of 0.72, substantially outperforming the 0.14 of BinPRE, the PRE tool.”
Stated limits 2
- we do not claim unconditional convergence for arbitrary executions.
- In practice, an LLM may generate an inconsistent hypothesis or incorrectly mark a prediction as high-confidence, violating the behavioral assumptions above.
What’s new here 4
- Joint iterative optimization of field detection and semantic inference using an EM-like loop
- Recursive LLM-driven detection that explicitly handles nested protocol payloads
- Integration of fine-grained dynamic taint and loop analysis with Tree-of-Thought prompting
- Elimination of hand-crafted heuristics in favor of LLM reasoning grounded in execution evidence
Artifacts & methods 6
- SEMPREsystem
- dynamic taint analysisprimitive
- loop detectionprimitive
- Tree-of-Thought promptingprimitive
- large language model (GPT-4/ GPT-o3)primitive
- evaluation dataset (public + undocumented + malware protocols)dataset