Pen-DHRL yields higher rewards and captures more hosts than baselines in dynamic network pen testing
The claim
Pen-DHRL outperforms existing RL baselines across multiple dynamic network scenarios
Published as: “DYNAS: A Dynamic Network Penetration Testing Framework Using Deep Hierarchical Reinforcement Learning”
So what
Hierarchical RL with IDS-aware subgoals can substantially improve attack planning performance under realistic defensive dynamics, suggesting a path toward more robust automated penetration testing.
The five-minute version
Pen-DHRL achieves up to 64% higher reward per step when transferring from defense-disabled training to defended evaluation, and captures up to 30% more sensitive hosts than flat baselines. The paper introduces DYNAS, a simulator that adds stateful IDS responses, scan false-positive/negative noise, temporary service churn, and duration-based action timeouts to NASimEmu, and evaluates hierarchical versus flat RL agents across four scenario variants and multiple training regimes. Results show Pen-DHRL stays positive under full defensive dynamics, while flat baselines produce negative returns without curriculum; with curriculum some flat baselines become competitive and even exceed Pen-DHRL on certain metrics (e.g., GNN higher reward on VarA). Evidence comes from simulated experiments on networks up to 83 hosts and 12 subnets; no real-world deployment is reported.
What they found 2
- Pen-DHRL obtains substantially higher reward per step and host capture than flat baselines when evaluated with IDS enabled after training without IDS.
“Pen-DHRL obtains approximately 64% and 24% higher reward per step on Bridge and Dynamic, respectively, and approximately 26% and 30% more captured hosts on VarA and VarB than the next-best baseline.”
- With all defensive dynamics active from the start of training, Pen-DHRL remains positive across all scenarios while flat baselines produce negative mean returns.
“With all defensive dynamics active from the beginning of training, PenDHRL achieves positive mean episode returns in all four scenarios, whereas the flat baselines produce negative mean returns in all 12 architecture-scenario combinations”
What undercuts it the paper’s own results
- In the IDS-disabled-training / IDS-enabled-evaluation transfer, the GNN baseline achieves higher reward per step (0.38) and captures more hosts (62.45) than Pen-DHRL (0.14 reward, 46.42 capture) on the VarA scenario.
Stated limits 1
- Future work will extend DYNAS across additional topology families and integrate active defender policies and host-level firewall rules.
What’s new here 4
- Adds a configurable defense-dynamics layer (stateful IDS, scan noise, service churn, action timeouts) to a penetration-testing simulator.
- Implements a curriculum learning schedule that stages defensive mechanisms during training.
- Introduces Pen-DHRL, a recurrent manager-worker hierarchical policy that conditions subgoal selection on IDS context.
- Provides a template-based generator for large, parameterized network scenarios while preserving a fixed action vocabulary.
Artifacts & methods 5
- DYNASsystem
- Pen-DHRL
- NASimEmudataset
- Curriculum scheduling
- Hierarchical reinforcement learning