Basic Block Agent

Concept

The Basic Block Agent is the low-level micro-step policy in HiFuzz's two-level hierarchical reinforcement-learning architecture. Within the global context fixed by the Program Agent, it decides the instruction-category mix and termination mode of each basic block, is trained with Proximal Policy Optimization (PPO), and receives both an extrinsic coverage-based reward and an intrinsic semantic novelty reward derived from a frozen Semantic-Aware Basic Block Encoder.

First seen 7/10/2026

Last seen 8/19/2026

Evidence 8 chunks

Wiki v2

WIKI

Overview

The Basic Block Agent is the low-level micro-step policy in HiFuzz's two-level hierarchical reinforcement-learning architecture. While the high-level Program Agent selects the global structure of a test program, the Basic Block Agent decides the instructions inside each basic block (BB) at micro steps within that global context.[C1]

HiFuzz separates these roles because program-level choices such as memory layout, BB count, and control flow operate at a different abstraction level and timescale from instruction-level decisions such as operand selection, dependency management, and BB termination.[C1] The decomposition follows the hierarchical reinforcement-learning framework of Sutton, Precup, and Singh, in which a high-level policy selects among temporally extended options whose execution is carried out by lower-level policies.[C8]

READ FULL ARTICLE →

NEIGHBORHOOD

No graph connections found for this entity yet. It may appear in future ingestion runs.

explore full graph →

RELATIONSHIPS

8 connections

HiFuzz ← uses 100% 3e

HiFuzz uses a Basic Block Agent as the low-level agent in its hierarchical RL architecture.

Proximal Policy Optimization uses → 100% 3e

The Basic Block Agent uses Proximal Policy Optimization to learn its policy.

Dual-Advantage Actor Loss uses → 90% 2e

The Basic Block Agent employs a dual-advantage actor loss combining intrinsic and extrinsic advantages.

Multi-Head Actor-Critic uses → 100% 2e

The Basic Block Agent uses a multi-head Actor-Critic architecture with separate value heads for intrinsic and extrinsic returns.

HiFuzz part of → 100% 2e

The Basic Block Agent is a component of the HiFuzz dual-agent architecture.

Intrinsic Novelty Reward uses → 100% 2e

The Basic Block Agent receives intrinsic novelty rewards from the BB encoder.

Dual-Advantage Actor Loss ← part of 90% 1e

The dual-advantage actor loss is a design contribution within the Basic Block Agent's training objective.

Multi-Head Actor-Critic ← part of 100% 1e

The multi-head Actor-Critic is the architecture used by the Basic Block Agent.

LINKED ENTITIES

3 links

HiFuzz USES Extracted graph relationship

Proximal Policy Optimization USES Extracted graph relationship

Hierarchical Reinforcement Learning PART_OF The Basic Block Agent is the low-level policy in HiFuzz's two-level HRL design, justified by the options/semi-MDP framework of Sutton, Precup, and Singh.

CITATIONS

12 sources

12 citations — click to expand

[1] The Basic Block Agent is the low-level micro-step policy in HiFuzz's two-level hierarchical reinforcement-learning architecture, deciding instructions inside each basic block while the Program Agent selects global structure. HiFuzz: Hierarchical Reinforcement Learning for Semantic-Aware and Adaptive CPU Fuzzing

[2] Given the global context set by the Program Agent, the Basic Block Agent determines the mix of instruction categories within a specific basic block and decides when a basic block should terminate. HiFuzz: Hierarchical Reinforcement Learning for Semantic-Aware and Adaptive CPU Fuzzing

[3] For each basic block, HiFuzz represents the local generation problem as a BB Level Config (target instruction-category mix plus termination mode), and a constrained generator instantiates concrete instructions while enforcing operand, privilege, address, and jump-target constraints. HiFuzz: Hierarchical Reinforcement Learning for Semantic-Aware and Adaptive CPU Fuzzing

[4] The Basic Block Agent outputs compact category-level distribution parameters and a BB-termination signal rather than every concrete instruction, avoiding an impractically large action space. HiFuzz: Hierarchical Reinforcement Learning for Semantic-Aware and Adaptive CPU Fuzzing

[5] HiFuzz trains the Basic Block Agent with Proximal Policy Optimization using the clipped-surrogate objective L^CLIP, providing stable micro-step updates. HiFuzz: Hierarchical Reinforcement Learning for Semantic-Aware and Adaptive CPU Fuzzing

[6] The PPO implementation uses a multi-head Actor-Critic with separate intrinsic and extrinsic value heads, with coefficients α_int=1, α_ext=2 and extrinsic-baseline diminishing factor γ=0.75. HiFuzz: Hierarchical Reinforcement Learning for Semantic-Aware and Adaptive CPU Fuzzing

[7] The intrinsic reward R_int = min_k (1 − cos(E(bb), C_k)) is computed as cluster-distance novelty over frozen BB-Encoder embeddings. HiFuzz: Hierarchical Reinforcement Learning for Semantic-Aware and Adaptive CPU Fuzzing

[8] The extrinsic reward R_ext = (Δcov − baseline)/baseline with baseline = mean_recent_rewards * γ provides a coverage-derived learning signal normalized against a dynamic baseline. HiFuzz: Hierarchical Reinforcement Learning for Semantic-Aware and Adaptive CPU Fuzzing

[9] The Basic Block Agent uses a dual-advantage actor loss L_actor = α_int A_int + α_ext A_ext combined with per-head critic MSE losses and an entropy bonus in the total objective. HiFuzz: Hierarchical Reinforcement Learning for Semantic-Aware and Adaptive CPU Fuzzing

[10] The Semantic-Aware Basic Block Encoder consists of a structured RISC-V tokenizer, a Bi-LSTM backbone, and a two-stage training pipeline (Masked Language Modeling pre-training followed by supervised BB-Sim fine-tuning), and is DUT-agnostic across Rocket, BOOM, and CVA6. HiFuzz: Hierarchical Reinforcement Learning for Semantic-Aware and Adaptive CPU Fuzzing

[11] Proximal Policy Optimization was introduced by Schulman, Wolski, Dhariwal, Radford, and Klimov. Proximal Policy Optimization Algorithms

[12] The two-level Program Agent / Basic Block Agent decomposition is justified by the options framework and semi-MDP formalization of hierarchical reinforcement learning. Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning