GenHuzz: An Efficient Generative Hardware Fuzzer Lichao Wu, Mohamadreza Rostami, and Huimin Li, Technical University of Darmstadt; Jeyavijayan Rajendran, Texas A&M University; Ahmad-Reza Sadeghi, Technical University of Darmstadt https://www.usenix.org/conference/usenixsecurity25/presentation/wu-lichao

This paper is included in the Proceedings of the 34th USENIX Security Symposium. August 13–15, 2025 • Seattle, WA, USA 978-1-939133-52-6

    Open access to the Proceedings of the

34th USENIX Security Symposium is sponsored by USENIX.

ARTIFACT EVALUATED 73 AVAILABLE

GenHuzz: An Efficient Generative Hardware Fuzzer

        Lichao Wu                      Mohamadreza Rostami

Technical University of Darmstadt Technical University of Darmstadt Huimin Li Jeyavijayan Rajendran Ahmad-Reza Sadeghi Technical University of Darmstadt Texas A&M University Technical University of Darmstadt

    Abstract can potentially be exploited through sophisticated cross-layer

Hardware security is crucial for ensuring trustworthy comput- attacks via, e.g., specific weaknesses in hardware implementa- ing systems. However, the growing complexity of hardware tions [3,4] or inherent functional features [5–7]. The Common designs has introduced new vulnerabilities that are challeng- Weakness Enumeration (CWE) identifies numerous vulnera- ing and expensive to address after fabrication. Hardware fuzz bilities affecting hardware and software [8]. Addressing these testing, particularly whitebox fuzzing, is promising for scal- issues after fabrication (post-silicon) can be prohibitively ex- able and adaptable hardware vulnerability detection. Despite pensive [9]. Therefore, it is crucial to identify and address its potential, existing hardware fuzzers face significant chal- these vulnerabilities before fabrication (i.e., pre-silicon) to lenges, including the complexity of input semantics, limited ensure the security of hardware systems and avoid the sub- feedback utilization, and the need for extensive test cases. stantial costs associated with post-silicon patches. To address these limitations, we propose GenHuzz, a novel To reduce human intervention in security assessment, re- white-box hardware fuzzing framework that reframes fuzzing searchers have developed methods such as formal verifi- as an optimization problem by optimizing the fuzzing policy cation [10–14], runtime detection (a.k.a. dynamic verifica- to generate more subtle and effective test cases for vulnera- tion) [15, 16], information flow tracking [17–19], and hard- bility and bug detection. GenHuzz utilizes a language model- ware fuzzing [20, 21]. Among them, hardware fuzzing has based fuzzer to intelligently generate RISC-V assembly in- emerged as a promising approach due to its scalability and structions, which are then dynamically optimized through a adaptability to various designs. Inspired by the success of soft- Hardware-Guided Reinforcement Learning framework incor- ware fuzzing [22], researchers have extended its principles porating real-time feedback from the hardware. GenHuzz is to hardware fuzzing, particularity in the white-box model, to uniquely capable of understanding and exploiting complex in- identify vulnerabilities in hardware. Hardware fuzzing shows terdependencies between instructions, enabling the discovery considerable promise by automating vulnerability and bug of deeper bugs and vulnerabilities. Our evaluation of three detection processes, particularly in testing across multiple RISC-V cores demonstrates that GenHuzz achieves signifi- processor cores [20, 21, 23–31]. cantly higher hardware coverage with fewer test cases than four state-of-the-art fuzzers. GenHuzz detects all known bugs Hardware fuzzing challenges. Despite the advantages men- reported in existing studies with fewer test cases. Furthermore, tioned above, hardware fuzzing faces several challenges. it uncovers 10 new vulnerabilities, 5 of which are the most Firstly, the hardware fuzzer frequently overlooks the complex- severe hardware vulnerabilities ever detected by a hardware ity of input semantics. Indeed, a fuzzer should understand the fuzzer targeting the same cores, with CVSS v3 severity scores interdependencies between instructions to trigger vulnerabili- exceeding 7.3 out of 10. ties and bugs that hide more profoundly in the DUT (Device Under Test). Unfortunately, existing approaches solely focus on basic instruction mutations [20, 24, 32]. This oversight 1 Introduction means that vulnerabilities requiring a combination of multi- ple instructions often remain undetected. Secondly, existing Hardware is the cornerstone of trust in secure computing sys- fuzzers have low feedback utilization from DUT. Although tems. Despite enforcing specific hardware countermeasures conventional hardware fuzzing methods have a feedback loop guided by established security standards [1, 2], the increasing from the DUT to the fuzzer, this feedback typically serves complexity of hardware designs has led to functional bugs only to discard ineffective test cases rather than to inform the and security-critical vulnerabilities that are uncovered, which generation of better ones. This limited feedback causes hard-

USENIX Association              34th USENIX Security Symposium 1787

ware coverage to plateau quickly, leaving some vulnerabilities adapt GenHuzz. The results outperform state-of-the-art unexplored if they are associated with unreachable coverage fuzzing frameworks, reaching high hardware coverage points. Thirdly, the portability problem affects the general- with 1% of test cases compared with state-of-the-art. ity of hardware fuzzing. Although existing works explore RISC-V ISA, they typically need to customize the fuzzer for 4. GenHuzz successfully identified 10 previously unre- a specific target implementation, limiting their applicability ported bugs and vulnerabilities in the tested cores (Rock- across the broader spectrum of hardware security assessments. etChip [33], Boom [34], and CVA6 [35]). Among them, Lastly, hardware fuzzing requires a large number of test cases GenHuzz uncovered five new security vulnerabilities to detect specific vulnerabilities, leading to efficiency issues. with high severity scores exceeding 7.3 (out of 10) ac- Considering the substantial overhead involved in the hardware cording to the Common Vulnerability Scoring System simulation and execution of the test cases and the time con- Version 3.1 (CVSS V3) [36]. In addition, we also tested straints typically imposed during security evaluations, the vul- older versions of the benchmark cores and successfully nerability detection capability of hardware fuzzing becomes triggered all previously reported bugs and vulnerabili- limited. We discuss the shortcomings of existing white-box ties in previous work [20, 24, 25, 27, 28]. The discovered hardware fuzzing in Section 9. vulnerabilities and bugs are detailed in Section 6. Our goals and contributions: This work introduces a novel The remainder of this paper is structured as follows. We pro- coverage-based white-box hardware fuzzing framework, Gen- vide the necessary background information in Section 2. In Huzz, which effectively addresses the challenges mentioned Section 3, we describe the design of GenHuzz in detail; the above. Unlike existing fuzzers, GenHuzz employs a novel implementation is introduced in Section 4. Section 5 evalu- approach by framing hardware fuzzing as an optimization ates the hardware coverage of GenHuzz and benchmark with problem. This is done by optimizing the fuzzing policy for state-of-the-art fuzzers. Section 6 details the vulnerabilities test case generation using real-time hardware feedback. We and bugs detected by the GenHuzz. Section 7 assesses the per- have developed a custom language model-based fuzzer to formance of GenHuzz and the influence of key components. understand and generate RISC-V assembly code. The fuzzing Section 8 provides a discussion on the proposed method. Sec- policy is dynamically adjusted through reinforcement learn- tion 9 elaborates on related works. Finally, we summarize this ing guided by feedback from the hardware, a method we term work in Section 10. Hardware-Guided Reinforcement Learning (HGRL). The inte- 2 gration of HGRL allows GenHuzz to go significantly beyond Preliminaries random instruction generation; it comprehends the semantics of the assembly language and discovers subtle but crucial 2.1 Fuzzing inter-relation among the several instructions that may inherit Fuzzing has gained significant traction due to its low de- a vulnerability, an insight that other fuzzers cannot provide. ployment costs and expedited verification process for testing Consequently, the augmented test case with complex data and complex designs. Fuzzing primarily involves generating and control flows significantly advances hardware coverage and mutating random test cases, monitoring the Device Under vulnerability detection. Our main contributions are: Test (DUT), and analyzing for vulnerabilities or vulnerabili- 1. We introduce a novel language model-based white-box ties. Traditional fuzzing begins with generating random test fuzzer, GenHuzz, capable of accurately generating RISC- cases or input stimuli. To efficiently cover the DUT’s state V assembly. Unlike conventional fuzzers that rely on space, most fuzzers use mutation algorithms to create new mutations and instruction rules to create new test cases, test cases, differentiating it from dynamic verification. The GenHuzz comprehends the semantics of the instructions, input stimuli are then fed into the DUT, where its status is enabling greater vulnerability detection capability to the monitored, and any crashes during the fuzzing period are specifics of DUTs. recorded. The monitored output is analyzed for vulnerabili- ties, and the DUT executes test inputs until a crash is recorded. 2. We propose a novel learning scheme, Hardware-Guided Software fuzzers analyze these crashes to detect vulnerabili- Reinforcement Learning (HGRL), which fine-tunes the ties; hardware fuzzing, on the other hand, verifies the expected fuzzing policy based on real-time hardware coverage outcome from the DUT against assertions or the output from feedback. This approach guides the fuzzer in producing the Golden Reference Model (GRM), which generates the high-coverage test cases while avoiding the pitfalls of expected responses. local optima. Fuzzing techniques can be broadly classified into three types depending on the available information regarding the 3. We demonstrate the flexibility of GenHuzz across RISC- DUT: black-box fuzzing, grey-box fuzzing, and white-box V processors. We experimentally validate GenHuzz fuzzing [21, 37]. For instance, white-box fuzzers have com- on three different RISC-V cores without the need to prehensive knowledge of the DUT and peripheral information

1788 34th USENIX Security Symposium USENIX Association

about data flow, control flow, data format, protocols, and high- 2.3 Language Model level architecture. Coverage-based white-box fuzzing aims to achieve maximum code coverage through feedback en- Language models are pivotal in natural language processing gines. Various coverage metrics exist in hardware, including (NLP), functioning as systems that can understand, generate, finite state machine (FSM), line, condition, and MUX toggle and manipulate human language. These models are trained coverage. During fuzzing, input seeds are stored and passed on vast amounts of text data to predict the likelihood of a to the mutation engine, which performs mutation operations sequence of words, enabling them to perform various tasks to generate multiple input seeds. Then, coverage reports are such as translation, summarization, and question-answering. extracted based on the input provided to the DUT and fed Among the most advanced language models are those based back to the mutation engine. The mutation engine discards on transformer architectures [40], notably the Generative uninteresting seeds from the input pool and further mutates Pre-trained Transformer (GPT) [41], and Bidirectional En- the interesting input seeds to generate a new set. The design coder Representations from Transformers (BERT) [42]. For is simulated on inputs with these seeds, and any potential instance, GPT models, such as GPT-3 and GPT-4, are de- crashes are saved for vulnerability analysis. signed using a unidirectional approach. The model predicts the next word in a sentence based on the preceding words. This is achieved through a stack of transformer decoder layers 2.2 Reinforcement Learning comprising multi-head self-attention mechanisms and feed- forward neural networks. The GPT models are pre-trained Reinforcement learning (RL) [38] is a machine learning (ML) on a diverse corpus of text and can be fine-tuned for specific technique that trains software to make decisions to achieve tasks with relatively small amounts of task-specific data. the most optimal results. It mimics the trial-and-error learning process that humans use to achieve their goals. Actions that 3 work towards the objective are reinforced, while actions that GenHuzz detract from the objective are ignored. RL differs from other forms of machine learning, such as supervised and unsuper- GenHuzz employs a teacher-student framework, where the vised learning [39], in that the agent cannot access labeled Design Under Test (DUT) acts as the teacher, guiding the data or explicit rules. fuzzer to generate more effective test cases that explore a broader state space of the DUT. Aligned with existing works, this paper focuses on white-box fuzzing of the open-source cores with the standard RISC-V ISA, as they publicly provide RTL and detailed hardware coverages, in contrast to closed- source processors from, such as ARM and Intel. 5 a A high-level overview of this method is depicted in Fig. 2, which comprises three main stages: 1) fuzzer initialization, T1 2) hardware-guided reinforcement learning (HGRL), and 3) vulnerability detection. A detailed explanation of each stage Figure 1: A demonstration of a generic RL environment. is provided in subsequent sections. In the first stage, we ini- tialize a fuzzer based on a language model (top-left graph) to grasp the fundamental rules of assembly instructions. This In general, RL consists of five basic blocks: 1) agent: the allows it to generate highly accurate and adaptable random learner and the decision maker; 2) environment: the physical assembly instructions. The initialized fuzzer actively inter- world in which the agent operates; 3) state (s): the current acts with the DUT in the second stage through the HGRL situation of the agent; 4) reward (r): feedback from the en- framework. As these interactions increase, the fuzzer learns vironment; 5) policy (π): the method to map agent’s state to the fuzzing policy by understanding the semantics of the test actions. A graphical representation can be found in Fig. 1. An cases and the entanglement between instructions, generating agent makes observations from the environment called states. test cases that potentially cover more hardware states. The In time step t, the agent receives state st from the environment feedback from the DUT is driven by a reward system designed and acts by following a specific policy π or transition prob- to accurately reflect the hardware’s behavior and encourage ability Pr by taking action at . The action is rewarded with the fuzzer to generate test cases that may uncover previously rt by applying a reward function f to the state-action pair unexplored vulnerabilities. In the final stage, execution logs (st , at ); the state is updated to st+1. When the agent reaches a from the DUT and the Golden Reference Model (GRM) are predetermined terminal state, the environment sends a termi- compared to detect bugs. GRM serves as the ground truth for nate signal to the agent. From there on out, a new sequence correct execution. It executes RISC-V instructions according of states, actions, and rewards begins. to the ISA specification, ensuring its output can be used as a

USENIX Association    34th USENIX Security Symposium 1789

                               re                                    I. Bug!&             The tokens are subsequently encoded into numerical repre-

| Initialization || Reinforcement Learning | | Detection | sentations suitable for model input. We define a vocabulary V | Nettoken || Prediction i||| | Policy Reward i| | ! and a mapping function to the word embedding f : V → Rd, Assignment | | T= ! representing each encoded token, which are high-dimensional ! - & a |1 where d is the dimension of the embedding space uniquely || t i 4 t | vectors optimized to capture semantic similarities between Encoding | | &Decoding ip | | Tokenize& || Detokenize |} | Difference | words. The encoded dataset E is thus represented as: i I | ! |Homi nal | E = f (T₁) f (TSEP) f (T₂) f (TSEP) . . . f (TSEP) f (TN). (2) | t 12 TestCase | | t | The encoded data is then used to initialize the fuzzer. The I | Constructi R pvr HIRE training pipeline is detailed in Appendix A. The training ob- jective of the fuzzer is to predict the next token in the sequence \ IBN Ed 1 ar based on the preceding tokens so that the fuzzer can generate AON rg new instructions based on previous instructions. Formally, Figure 2: An Overview of the GenHuzz. let x = (x1, x2, . . . , xM) be the sequence of embedded tokens, where M is the total number of tokens in the dataset E. The reference to compare with DUT outputs 1. fuzzer is trained to predict the next token xt+1 given the se- quence (x1, x2, . . . , xt ). The model parameters θ are optimized by minimizing the cross-entropy loss L : 3.1 Fuzzer Initialization L M−1 We conceptualize a test case composed of multiple instruc- (θ) = − ∑ log Pθ(xt+1 | x1, x2, . . . , xt ), (3) tions as a coherent sentence communicated to the Device t=1 Under Test (DUT). The proposed fuzzer leverages a language where the probability Pθ is parameterized by the trainable model for its exceptional performance in natural language parameter θ of the fuzzer. Finally, we utilize gradient descent processing tasks; the implementation details are provided in to minimize the loss function: Section 4. Fundamentally, an effective fuzzer should compre- θ(k+1) = θ(k) − η∇θL (θ(k)), (4) hend both intra-instruction semantics (i.e., how to generate valid individual instructions) and inter-instruction semantics where η is the learning rate and k denotes the iteration step. (i.e., how to strategically combine multiple instructions to This process iterates until the number of generated tokens uncover more complex bugs and vulnerabilities). GenHuzz is equals a preset maximum token number. We use this value to initialized with an understanding of intra-instruction seman- constrain the number of instructions per test case. tics. The inter-instruction semantics are learned dynamically through interaction with the hardware, detailed in Section 3.3. 3.2 State Transition and Reward Function The initialization of the fuzzer begins with data generation. We generate a set ofI N assembly instructions, denoted as As mentioned in Section 3.1, the fuzzer takes action by pre- = I1, I2, . . . , IN. Assembly language strengthens the seman- dicting the next token based on previous tokens. The reward tic relationships between instructions, allowing for more should be assigned to each action (token) for the reinforce- meaningful combinations that can reveal subtle hardware vul- ment learning tasks as the action changes the state. How- nerabilities. The generated instructions are concatenated into ever, assigning a reward when an action is an element of an a single sequence, D, using a specific separator, SEP: instruction, e.g., an operand, is more challenging than the D = I₁ SEP I₂ SEP I₃ SEP . . . IN. (1) common reinforcement learning tasks in which each action immediately gets a reward. There are three reasons: 1) only The SEP operator flags each instruction’s beginning and end, a complete instruction can be rewarded; thus, the reward for thus helping the fuzzer better understand the intra-instruction an action will be delayed; 2) not all instructions will be exe- semantics. Next, the combined instructions dataset is tok- cuted. For instance, a branch instruction could skip multiple enized and encoded to prepare it for training the model. instructions. The unexecuted instructions do not contribute to Each instruction Ii is tokenized into a sequence of tokens the hardware coverage, but they are essential for generating Ti = (ti,1,ti,2, . . . , ti,ki), where ki is the number of tokens in the upcoming instructions; 3) even a syntactically correct in- instruction Ii. The tokenized dataset is then represented as struction could cause exceptions when executing on the DUT. T = T₁ TSEP T₂ TSEP . . . TSEP TN. We cannot simply assign zero or negative rewards to these 1Since the RISC-V GRM we use [43] does not support speculative or instructions, as they also contribute to the fuzzing process. out-of-order processing, GenHuzz can only detect static bugs, aligned with To assign proper rewards to each action, we first identify literature [20, 24, 25, 27, 28]. possible instruction status when executing on the DUT, thus

1790 34th USENIX Security Symposium USENIX Association

valid & executed valid & failed Algorithm 1 State Transition and Identification Ii t 1nst6 Require: env, f uzzer, GRM Inst1 Inst7 1: obs ← reset(env) 2: while not done do | 3: action ← f uzzer(obs) 4: next_obs, done ← step(env, action) valid & unexecuted invalid & unexecuted 5: obs ← next_obs Figure 3: Possible instruction status in a test case. 6: instructions ← sperate(obs) 7: for inst in instructions do 8: valid ← verify(inst) 9: if valid then defining the state transition for a generated test case. Using a 10: valid_instructions ← append(inst) test case shown in Fig. 3 as an example, it consists of seven 11: executed_instruction ← GRM(valid_instructions) instructions. The DUT throws an exception in the fifth instruc- tion (inst5), highlighted in red, and the following instructions signed when the instruction is valid and executed. As men- (inst6 and inst7) are unexecuted. In this case, there are four tioned, r instruction statuses in total2: 1) Valid & Executed: an instruc- valid is essential for our framework as it ensures the tion is valid and is executed by DUT; 2) Valid & Unexecuted: fuzzer generates valid instructions during fine-tuning. The an instruction is valid but unexecuted by DUT due to, for involvement of the hardware_coverage is a natural process instance, branch instructions or failed instructions before the as we want the fuzzer to generate test cases with higher cov- current instruction; 3) Valid & Failed: an instruction is valid erage, thus leading to a higher probability of vulnerability and is executed by DUT, but DUT throws an expectation, and detection. The realization of the hardware_coverage calcu- the test case execution is stopped; 4) Invalid & Unexecuted: lation is presented in Section 4.1, relies on the access to the an instruction is invalid (violates the ISA specification) and, RTL code. Finally, we involve a bonus reward rbonus, assigned naturally, not executed by DUT. if the generated test case leads to the highest hardware cover- Due to the complexity of the instruction status, we separate age compared to all previously tested coverage. rbonus further the state transition into three steps, detailed in Algorithm 1. pushes the fuzzer to generate high-quality test cases. Our ex- First, as shown in lines L2 to L5, the instruction token is periment in Section 7 shows that without rvalid and rbonus, the generated without any restrictions until the done condition is fuzzer works unstable and quickly loses the dynamic instruc- fulfilled, realized by a token number threshold. As mentioned, tion generation capability after a few learning iterations. we use this threshold to control the number of instructions that In this work, a test case that contains N instructions is sepa- form a test case. Unlike other hardware fuzzers that terminate rated into N sub-test cases, with each sub-test case concatenat- the test case generation process once an illegal instruction is ing a new instruction in sequence. This implementation leads identified, these instructions are considered part of our test to an accurate R evaluation and ensures that the upcoming case. Although these instructions are excluded from the hard- instructions do not overwrite the architectural state of a trig- ware execution, they become a learning experience that gives gered bug. Consequently, we detect bugs with significantly feedback to the fuzzer so that the fuzzer is enforced to gen- fewer test cases than existing works, presented in Section 6. erate valid instruction while involved based on the hardware feedback. Next, the post-processing begins with separating 3.3 Hardware-Guided Reinforcement Learn- the generated token array into individual instructions with ing SEP. Each instruction is checked for its validity in L8. Fi- nally, valid instructions are concatenated and simulated by The fuzzer initialized in Section 3.1 can be employed for the golden reference model (GRM), which returns the index fuzzing tasks by generating random assembly instructions. of executed instructions. However, its lack of interaction with the Device Under Test The instruction status forms the basis of the reward assign- (DUT) limits its ability to detect specific hardware vulnera- ment, shown in Eq. (5): bilities requiring multiple instruction combinations. To ad- dress this limitation, we bridge the fuzzer-DUT interaction R = rvalid + α ∗ hardware_coverage + rbonus, (5) using Proximal Policy Optimization (PPO), a model-free re- inforcement learning (RL) algorithm. Applying PPO directly where rvalid denoted the reward for the valid and invalid in- to hardware fuzzing poses significant challenges. Firstly, con- structions, α represents the weights of the hardware coverage. ventional RL agents typically manage simple states, such as The last two terms on the right of the equation are only as- sequences of binary decisions, and are not equipped to handle 2One may wonder about the existence of the invalid & executed instruc- complex scenarios involving assembly instructions. Further- tions. This instruction status does not exist in our framework, as the generated more, as discussed in Section 2.2, RL methods, in general, assembly will first be compiled into the binary and then sent to the hardware. are designed to optimize a policy that enables an agent to An invalid instruction will fail the compilation process. complete a task efficiently. This objective does not align with

USENIX Association 34th USENIX Security Symposium 1791

the goals of hardware fuzzing: an effective fuzzing policy that through rewards rt . The current fuzzing policy π is evaluated generates a test case with high hardware coverage or identifies by computing the advantage estimates ˆ At : a hardware vulnerability becomes obsolete after execution. Reapplying the same policy would be redundant, as it has Aˆt = rt + γVφ(st+1) −Vφ(st ), (7) already been tested and evaluated. where V To overcome these challenges, we introduce the Hardware- φ(st ) represents the value estimate of the state st out- Guided Reinforcement Learning (HGRL) framework, illus- putted by the scorer parameterized by φ, and γ is the discount trated in Fig.4, which dynamically adapts the fuzzer during factor that determines how future rewards are weighted rel- the fuzzing process. The HGRL framework consists of four ative to immediate rewards. Eq. 7 estimates the relative ad- fundamental components: a fuzzer that generates assembly vantage of taking a particular action in the given state st , com- instructions (described in Section 3.1), a scorer that evaluates pared to the expected baseline Vφ(st ). In this work, instead of the actions of the fuzzer by estimating the value of being directly using the DUT as a scorer or building a scorer from in a specific state or taking a particular action (detailed in scratch, we reuse the initialized fuzzer with one additional Section 3.2), a reset module that balances the exploration and output layer for the value prediction. Indeed, a scorer who can exploitation of the fuzzer, and a DUT that provides hardware evaluate the assembly input should understand its semantics. coverage feedback to refine the fuzzer and scorer iteratively. By reusing the fuzzer structure and weight, we avoid learn- ing a value function from scratch. The scorer is trained to estimate the expected value from a given state st . The scorer, [reᵀᵒᵍᵃ represented by the value function Vφ(st ), is updated by mini- mizing the mean squared error between the predicted values —— value function accurately reflects the expected return from D Scorer"ET and the observed returns, shown Eq. (8). This ensures that the JE,² any given state. Lscorer(φ) = Et [(Vφ(st ) − (rt + γVφ(st+1)))2]. (8) = pes Aˆt is used to update the policy followed by the fuzzer. Con- cretely, the fuzzer parameters θ are updated by maximizing Figure 4: Hardware-Guided Reinforcement Learning. the clipped surrogate objective Lfuzzer(θ). Formally, let the state at time step t be st , the generated in- Lfuzzer(θ) =Et [min( ππθ(at |st ) Aˆ , struction elements such as opcode and operand (a.k.a. action) θold(at |st ) t be at , and the coverage feedback (a.k.a. reward) provided by clip( πθ(at |st ) , 1 − ε, 1 + ε)Aˆ )], (9) the DUT be rt (see Eq. (5)). The HGRL objective is to maxi- πθold(at |st ) t mize the expected cumulative reward over time following the where πθ and πθ represent the current policy and the previ- old Bellman Equation [44]: ous policy before the fuzzer update, respectively. For simplic- T ity, we use πθ and π interchangeably throughout this paper. J(θ) = Eθ[∑ γt rt ], 0 ≤ γ ≤ 1, (6) The hyperparameter ε controls the clipping range to ensure t=0 the stability of policy updates. The clipping mechanism re- where γ is the discount factor prioritizing immediate rewards stricts the deviation of the new policy from the old one by over future rewards; T is the time horizon that defines the total taking the minimum of the clipped and unclipped objective number of time steps considered for the cumulative reward. values, thereby maintaining stability during fuzzer updates. Eq. (5) allows the fuzzer to "look forward": its goal is not only According to Eq. (9), actions with a higher advantage esti- maximizing the reward with the current generated instructions, mate, ˆAt , are more likely to be selected in future iterations. but the current generated instruction should also help boost the Although this approach is standard to guide the policy toward reward for the upcoming instructions. This characteristic is actions that yield higher rewards, it presents significant limi- beneficial for archiving coverage that requires a combination tations when applied to hardware fuzzing. First, the fuzzing of multiple instructions. Based on Eq. (5), the fuzzer will process tends to converge prematurely, leading to saturation in evolve to generate more valid and executed test cases that hardware coverage. This phenomenon arises from the action maximize hareward_coverage and try to get the rbonus. generation process, where at are sampled from a probability HGRL involves multiple iterations of fuzzer/scorer-DUT in- distribution generated by the fuzzer’s policy πθ(at |st ) based teraction. Based on the initial input, the fuzzer first generates on the current state st . The probability distribution over ac- assembly instructions to form a test case in each iteration. The tions is defined as: DUT executes the test case and provides coverage feedback πθ(at |st ) = softmax(Fθ(st )), (10)

1792 34th USENIX Security Symposium USENIX Association

where Fθ(·) outputs a vector of logits representing the like- 3.4 Bug and Vulnerability Detection lihood of each action. Indeed, certain opcodes and operands We employ differential testing, a commonly used approach in could be sampled more frequently. Consequently, those with lower sampling probabilities, denoted as at′, are less likely to the hardware fuzzing framework, to define a crash triggered by receive rewards. This creates a negative feedback loop where generated test cases. The detailed implementation is presented these instructions eventually disappear from the action space: in Section 4.1. This technique involves running the same test case on both the RTL (Register-Transfer Level) model of the

    t erence model, followed by a comparison of their execution
    tlim P(a′) ≈ 0. (11) DUT and the Instruction Set Architecture (ISA) golden ref-
    →∞

As a result, the fuzzer tends to repeatedly generate specific traces [20, 21, 25, 27]. However, discrepancies between these instructions, limiting the diversity of test cases. This behavior two models frequently lead to numerous mismatches [21], is similar to issues observed in training large language models many of which are duplicates. Consequently, after several (LLMs) with LLM-generated content, where skewed sam- hours of fuzzing, many of these mismatches are identified as pling leads to performance degradation [45]. Although the false positives or duplicate mismatches, reducing the fuzzing contexts differ, the underlying problem of reduced diversity process’s effectiveness. To address these two challenges, we is common, reducing effectiveness in both scenarios. have developed two heuristic algorithms, which we will de- The second limitation stems from the fuzzing policy π it- scribe in the following two sections. The differential testing self. RL inherently seeks to optimize the test case generation process is fully automated within our framework. Specifically, process by creating a stable policy π∗ that maximizes the it is capable of 1) executing the same test cases on the GRM, expected reward (Eq. 6). However, this stability can lead to 2) collecting and parsing the execution traces for both the core suboptimal exploration. Once the policy π∗ reaches a point system and the GRM, 3) identifying mismatches between the where certain hardware coverage or specific vulnerabilities execution traces of the core system and the GRM, and 4) are triggered, it becomes biased toward actions that have pre- applying developed heuristics to the detected mismatches to viously resulted in high rewards. This results in the policy minimize repetitive and unrelated cases. While manual verifi- repeatedly generating similar test cases, reducing the likeli- cation is necessary to determine if a vulnerability triggers a hood of exploring new instruction combinations that might detected mismatch, our framework offers a significant over- reveal additional vulnerabilities. Finally, π converges to a head reduction compared to advanced techniques [20, 24, 25] narrow region of the action space Aoptimal, where: that do not incorporate these heuristics. π∗(at ∈ Aoptimal|st ) ≈ 1. (12) 3.4.1 False Positives Our analysis of the most recent hardware fuzzing literature When looking at the entropy of the action distribution identified two primary causes that can lead to False Positives H(πθ(at |st )) = − ∑aₜ πθ(at |st ) log πθ(at |st ), which measures when comparing the ISA model and RTL traces: 1) Differ- the diversity of chosen actions. π∗ decreases entropy, repeat- ences in the device tree: the device tree describes the hardware edly favoring specific opcodes and operands. components and their configurations in a system [46]. Differ- To address these issues, we introduce a reset module that ences in the device tree between the RTL and ISA reference dynamically balances exploration and exploitation by moni- models can lead to discrepancies in execution traces. These toring entropy and hardware coverage. We track the maximum discrepancies occur because the RTL model might interpret hardware coverage Cmax achieved over time. If coverage stops or access hardware resources differently from the ISA model to improve, it suggests that the framework is trapped in a local due to mismatched hardware configurations, leading to mis- optimum and failing to explore new, potentially rewarding matches that are falsely interpreted as bugs or vulnerabilities. test cases. Then, the reset module is activated to restore the 2) Differences in boot ROM code: the boot ROM code initial- parameters θ and φ of both the fuzzer and the scorer to their izes the hardware and loads the test case, operating system, or initial states: firmware. Variations in the boot ROM code between the RTL θ ← θinit; φ ← φinit. (13) and ISA reference models can result in different system states or initialization sequences. These differences can produce The reset module forces the fuzzer to explore the action space mismatched traces during the comparison process, which are anew rather than relying on previously successful actions. Si- falsely flagged as a bug or vulnerability. For instance, if the multaneously, resetting the scorer allows it to focus on newly RTL model’s boot ROM code changes the privilege mode discovered vulnerabilities rather than existing ones. This dy- to user mode before resuming the execution of the test case, namic balance between exploration and exploitation ensures but the ISA reference model does not change the privilege that the fuzzer is not trapped in local optima, leading to more mode and remains in supervisor mode, the resulting execu- comprehensive hardware coverage and the discovery of previ- tion traces will differ. This difference in privilege mode can ously undetected bugs and vulnerabilities. lead to discrepancies in handling instructions and address

USENIX Association    34th USENIX Security Symposium 1793

access, causing mismatches incorrectly identified as bugs or epochs with a learning rate of 1e-5, which takes around one vulnerabilities by the differential testing process. To address hour with a single NVIDIA A6000 GPU. Once the fuzzer is these two primary causes, during the fuzzing, we fixed the trained, the same model is copied to build a scorer with one device tree and boot ROM code for both RTL and ISA refer- additional layer as the output layer. Using a shared structure ence models to ensure consistency in the device tree and boot for fuzzer and scorer could lead to a worse performance due to ROM between the RTL models [33–35] and Spike [43]. the interference between objectives [47]. Since the scorer still needs the training effort to estimate the action value reliably, 3.4.2 Duplicate Mismatches we use a larger learning rate for the scorer (5e-5) than the actor (1e-5) to balance the training effort. Different fuzzers triggered the same bug or vulnerability mul- tiple times, leading to a mismatch report each time. We pro- vided a signature extraction algorithm to avoid duplicate mis- 4.1 Fuzzing Process match reports in our mismatch detector mechanism. This The HGRL framework bridges the gap between the fuzzer/s- algorithm generates a unique signature for each mismatch corer and DUT. The goal of the fuzzer is to generate actions based on the opcode, register values, and exception cause. that form a better test case; the scorer evaluates the generated We did not include the register’s name to keep the signature action, including its intermediate reward and (discounted) register independent since some bugs or vulnerabilities have future rewards. The HGRL framework starts with the state the same root cause, but they trigger with register sequences. transition and reward assignment, detailed in Section 3.2. Using these signatures, our tool reports only one instance of Concretely, in Eq. (5), r each unique mismatch, reducing the manual work developers valid is assigned with -0.7 and 0.1 require. We generate a unique signature for each mismatch for the invalid and valid instruction tokens, respectively. The containing the instructions, source and destination register hardware_coverage metric is computed by parsing and an- values, and the exception cause value. These details are ex- alyzing various hardware behavior coverage metrics from tracted by analyzing execution traces. Memory addresses and Register-Transfer Level (RTL) code. These metrics include values are skipped, as this information is redundantly captured finite state machine (FSM) coverage, condition coverage, and within the registers during execution. line coverage. The coverage data is generated during the sim- ulation using Synopsys VCS [48], allowing us to assess how thoroughly the hardware design has been exercised. FSM 4 Implementation coverage measures how well the states and transitions of fi- nite state machines within the design are tested. Condition GenHuzz leverages a GPT-based language model as both a coverage assesses the evaluation of all possible logical condi- fuzzer and scorer. The original GPT-2 model, with 1.5 billion tions, while line coverage determines the extent to which the parameters and a 50 257-word vocabulary trained on eight lines of code in the RTL have been executed during simula- million web pages, uses Byte-Pair Encoding (BPE) tokeniza- tion. Together, these metrics provide a comprehensive view of tion. However, BPE splits opcodes into subwords, increasing the effectiveness of the test cases in validating the hardware the risk of invalid instructions; the already small RISC-V design. When an instruction is valid and executed, the instruc- vocabulary, around 500 unique opcodes and operands, min- tion token gets an additional reward from the hardware cov- imizes the need for vocabulary size compression with BPE. erage and a reward bonus Rbonus. The corresponding weight Consequently, we adopt a word-level encoding strategy that actor α equals 0.2; Rbonus is assigned with 0.4. Intuitively, assigns each opcode, operand, and immediate value a unique we want the last two factors of the reward function, namely token. This approach preserves the validity of opcodes and α ∗ hardware_coverage and Rbonus, to have a balanced contri- operands, requires less aggressive vocabulary reduction, and bution to the reward to control the exploration and exploitation simplifies training by shortening the tokenized length of each of the model. Furthermore, we normalize the reward for the instruction. Since assembly instructions are far simpler than straightforward reasoning: when the fuzzer performs rather human languages like English, we further reduce the com- badly, it receives much worse than good rewards. Normaliza- plexity by employing a model roughly ten times smaller than tion makes the gradient steeper for (puts more weight on) the GPT-2 with only approximately 12 million parameters. Our good rewards and shallower for (puts less weight on) the bad customized model incorporates hardware feedback while sig- rewards. Besides, making their distribution consistent helps nificantly reducing computational overhead. stabilize training. We prepare five million randomly generated instructions as The GRM and DUT execute the generated test case; register the training dataset, containing the basic RISC-V 32 and 64- values have been randomized to increase the possibilities of bit ISA and their extensions, including integer multiplication covering edge cases. The output is twofold: the hardware and division instructions, single-precision floating-point in- coverage is feedback to the fuzzer and scorer for the train- structions, atomic instructions, compressed instructions, and ing. The execution traces are fed to a mismatch detector for machine-level instructions. We train the model with 50 000 vulnerability detection. To implement the vulnerability de-

1794 34th USENIX Security Symposium    USENIX Association

tection component of our framework, we utilized Synopsys coverage metric of GenHuzz continues to increase with an VCS and the Spike simulator [43] to perform simulations increasing number of test cases (e.g., Fig. 5b), whereas the for both RTL and the Spike, which serves as the reference coverage for Cascade plateaus after a relatively small num- model for RISC-V ISA simulations. Synopsys VCS generates ber of test cases. Indeed, it highlights the critical role of the hardware simulation traces that detail the values of modified HGRL scheme that integrates reinforcement learning and a registers and memory transactions for each executed instruc- language model, enabling intelligent instruction combinations tion. In contrast, the Spike simulator provides reference traces, to target specific coverage points. Moreover, when the fuzzing which indicate the expected values of registers and memory policy converges to a local optimum, GenHuzz’s reset mod- locations after each instruction is executed when running a ule activates to encourage exploration of previously untested RISC-V binary file (e.g., an ELF file) according to the RISC- hardware states, further extending its effectiveness. V ISA. Therefore, any discrepancies between the values of Subsequently, we benchmark the performance of all consid- registers or memory transactions, such as differences in regis- ered fuzzers on RocketChip using condition coverage metrics, ter values, memory addresses, or memory data, could indicate with Cascade included for completeness. As illustrated in a bug or vulnerability. Fig. 6, GenHuzz significantly outperforms all other fuzzers. We developed an automated mismatch detection tool designed The coverage for DifuzzRTL, TheHuzz and ChatFuzz quickly to parse execution trace files from various RISC-V cores, in- saturates, indicating limited exploration capacity. In contrast, cluding RocketChip [33], Boom [34], and CVA6 [35], with GenHuzz maintains its superior coverage even with a substan- Spike [43] serving as the RISC-V ISA reference. By ana- tial increase in test cases (up to 100K). Remarkably, GenHuzz lyzing these execution traces, the tool extracts state changes achieves comparable coverage to these fuzzers using only for each test case at the granularity of individual instructions. 1% of the test cases, demonstrating its ability to efficiently Specifically, for each instruction in a fuzzing test case, the explore diverse hardware states. This efficiency directly en- resulting CPU state changes are identified and recorded. To hances bug and vulnerability detection, highlighting GenHuzz ensure reusability and ease of integration across different as a powerful tool for hardware security evaluation. cores, our tool employs a unified data structure. This structure organizes each test case as an ordered list of instruction trace 6 objects, where each object encapsulates the state changes of Detected Vulnerabilities and Bugs architecturally visible CPU components resulting from the GenHuzz identifies ten new vulnerabilities and bugs in the execution of the corresponding instruction. This unified ap- tested cores that have not been previously reported. Addi- proach simplifies the process of comparing execution traces tionally, to assess the effectiveness of GenHuzz, we tested across cores. The workflow of the tool begins by parsing the it on older versions of the benchmark cores to determine if execution traces and storing them in the unified data struc- it could detect bugs that were already reported in previous ture. Subsequently, it compares the state changes caused by studies [20, 24, 25, 27, 28]. GenHuzz successfully triggered all each instruction in the core’s execution trace against the cor- of the previously reported bugs and vulnerabilities with only responding trace from Spike, which serves as the GRM. Any 50 000 test cases, indicating its strong capability in vulnera- deviations from the ISA [49, 50], as defined by Spike [43], bility and bug detections. are identified and reported as mismatches. Table 1 shows all the newly discovered vulnerabilities and the most significant known vulnerabilities detected by Gen- 5 Coverage Evaluation Huzz. Five of the 10 newly discovered vulnerabilities have a severity score greater than 7.3 out of 10, according to the As a coverage-based white-box fuzzing method, GenHuzz CVSS V3 [36]. This indicates that these vulnerabilities pose aims to maximize the exploration of hardware states, thereby serious security risks to the benchmark cores. Furthermore, increasing the likelihood of detecting bugs and vulnerabili- triggering these vulnerabilities requires carefully crafting and ties. In this section, we evaluate GenHuzz compared to four combining multiple instructions, often more than two and state-of-the-art fuzzers: Cascade [27], DifuzzRTL [28], The- sometimes up to six instructions. It is highly unlikely (or prac- Huzz [20], and ChatFuzz [25]. Given that Cascade is the most tically impossible) for previous fuzzers, which rely on random recent fuzzers, we perform a comprehensive coverage analysis instruction generation [20, 24, 27, 28], to generate such com- using condition, line, and FSM metrics across three RISC- plex instruction sequences. This is because two challenging V cores: RocketChip [33], Boom [34], and CVA6 [35]. For requirements must be met: 1) the generated instructions must the other fuzzers, due to space constraints, our benchmarking have both data and control flow entanglement, e.g., subsequent focuses on RocketChip with condition coverage metrics. instructions must use data loaded by one instruction, and 2) As depicted in Fig. 5, except for the FSM metric on Rock- the instructions must be semantically intertwined, e.g., config- etChip (the bottom figure on Fig. 5a) where GenHuzz and Cas- uring Physical Memory Protection (PMP) settings and PMP cade perform identically, GenHuzz consistently outperforms address CSR registers within the same test case. However, by Cascade across all cores and evaluation metrics. Notably, the learning the inter-instruction semantics of assembly language

USENIX Association    34th USENIX Security Symposium 1795

© Foss . | Soss 2

Sors 3

                                          goss                                           z

   oi                              oo                    70000 30000            30000         20600 30000     40300 50000
   Number of test cases                                 Number of test cases                 Number of test cases
   (a) RocketChip                                        (b) Boom                             (c) CVA6

   Figure 5: Coverage Benchmark between GenHuzz and Cascade

0.82 [FE Listing 1: Code snippets for reading minimun PMP granular- 0.80 ity on CVA6. 0.78 1 csrw pmpcfg0 , x0 2 addi a3 ,x0 , -1 5076 | 3 csrw pmpaddr0 , a3 4 csrr a3 , pmpaddr0 // a3=0x003FFFFFFFFFFFFF 5

                                         Cascade

070 DifuzzRTL four-byte granularity for PMP [50]. However, if the PMP TheHuzz 0.681 ChatFuzz | address matching mode is set to 0x2 (indicating Naturally 0.6617 This work Aligned four-byte regions NA4), the core automatically resets 0 10000 20000 30000 40000 50000 the PMP configuration register to 0x0, effectively deactivating Number of test cases the memory protection mechanism within a security-critical section, thereby enabling the attacker to perform unauthorized Figure 6: Coverage benchmark on the RocketChip. read, write, and execute operations on the memory. GenHuzz generates a test case that attempts to configure PMP using the unsupported NA4 mode. The test case first configures through hardware feedback, GenHuzz generates test cases the PMP address and then sets the PMP configuration regis- with integrated data and control flow, utilizing informative ter. However, the vulnerability detection module identified feedback to the model. that the PMP configuration was not maintained as intended The following section provides detailed explanations and dis- because setting the NA4 mode in the CVA6 core results in cusses potential exploitation scenarios for three of the most the core resetting the PMP configuration to 0x0, leaving the complex (which require a minimum of four instructions to memory region unprotected. This discrepancy does not occur trigger the vulnerability successfully) and critical vulnera- in the Spike RISC-V ISA simulator, where the same sequence bilities detected by GenHuzz. Due to the page limit, the de- of instructions creates a protected region and throws a Load scriptions of other detected novel bugs and vulnerabilities are Access Fault exception in case of access attempts. The provided in Appendix B. sequence of instructions demonstrating this vulnerability is PMP Configuration Leading to Bypass of Memory Pro- shown in Listing 2. tection (V1). In the RISC-V architecture, PMP is designed The implementation of the PMP in CVA6 demonstrates two to provide fine-grained memory protection for both user and critical flaws. First, there is no support for the NA4 mode, and privileged modes. PMP allows for configuring access per- any attempt to configure it results in the complete disabling missions and address-matching modes for various memory of memory protection. Second, this behavior introduces a regions to enforce security policies and prevent unauthorized significant security risk as it allows the bypassing of PMP- access. In CVA6, the granularity of the PMP is determined based protection entirely. Consequently, code running on the by setting the address range to four bytes, as evidenced by CVA6 core (whether in user, kernel, or firmware) may operate isting 1, which is implemented based on the standard defined without the intended memory protections, leading to potential by RISC-V ISA specification [50]. information leakage and privilege escalation vulnerabilities, Here, register a3 is set to 0x003FFFFFFFFFFFFF, indicating i.e., CWE-1281 and CWE-1220, in multi-threaded systems. The that the least significant bit is set to 0, thus confirming a behavior of the CVA6 core under these conditions needs hard-

1796 34th USENIX Security Symposium USENIX Association

Listing 2: Proof of concept code snippets for bypassing PMP data and control flow entanglement, where both LR and SC protection on CVA6. share the same address operand, and 3) the use of random- 1 auipc a3 , 0 x0 // a3 = [PC] ness combined with strategic decision-making in instruction 2 addi a3 , a3 , 0 x1c // a3 = [protected address] generation, such as inserting a standard store instruction. 3 addi ,→t1 ,= [PC + 0x1c] Upon executing this test case, our vulnerability detection mod- 4 srl t1 , x0 , 0 x2 // t1 = 0x2 ule identified that the Store-Conditional (SC) operation failed a3 , t1 // t1 = a3 >> 0x2 5 csrw pmpaddr0 , t1 // pmpaddr0 = t1 because the reservation was broken by the effect of a normal 6 addi t0 , x0 , 0 x94 // t0 = 0x94 store operation to a random address. The ISA implementa- 78 csrw pmpcfg0 , t0 // pmpcfg0 = t0 tions in both Boom and Rocket seem to register reservations auipc,→a3 ,= 0 x0 // a3 = [protected address] for the entire physical memory range. This approach leads to [PC + 0x1c] 9 lbu a4 , 0( a3 ) several issues; most notably, it creates significant overhead since any store operation to an arbitrary address can disrupt the reservation. As a result, achieving a single successful ware revision to enforce correct PMP behavior. Load-Reserved and Store-Conditional operation may require Improper Handling of Load-Reserved/Store-Conditional numerous attempts, particularly in multi-threaded environ- Instructions (V8). In RISC-V architecture, the Load- ments. This inefficiency can lead to potential vulnerabilities Reserved (LR.W, LR.D) and Store-Conditional (SC.W, SC.D) such as information leakage and denial of service, specifically instructions are designed to facilitate atomic read-modify- CWE-1281 and CWE-1421. Listing 3 provides an example se- write operations in a multi-threaded or multi-processor envi- quence of instructions to replicate this behavior. ronment. These instructions help implement synchronization Improper Handling of PMP Execute-Only Memory mechanisms, such as locks or semaphores, to ensure that crit- Regions(V9). In the RISC-V architecture, the Physical Mem- ical code sections are executed atomically, preventing race ory Protection (PMP) mechanism provides fine-grained con- conditions and ensuring data integrity. An implementation trol over memory access permissions, allowing regions to be can allocate an arbitrary subset of the memory space during configured with Read (R), Write (W), and Execute (X) permis- each Load-Reserved operation, and multiple LR reservations sions. According to the RISC-V privileged specification [50], can coexist concurrently for a single hardware thread (hart). an execute-only region should have permissions set as R=0, The success of a Store-Conditional operation is contingent W=0, X=1, allowing code execution within this region while upon the absence of external accesses by other harts to the preventing any load or store operations. The combination designated address between the SC.D instruction and the most R=0 and W=1 is specifically reserved and should not occur. recent LR.D instruction that reserved the same address within However, in the BOOM core implementation, setting a region the hart. Notably, this LR.D instruction might have referenced to be execute-only (R=0, W=0, X=1) incorrectly triggers an a different address, but it reserved the memory subset that Instruction Access Fault when attempting to execute includes the address designated by the SC [49, 50]. an instruction within that region. This behavior deviates from the RISC-V ISA specification [50], which expects instruc- Listing 3: Proof of concept code snippets to trigger and break tions in an execute-only region to execute without fault. The LR.D and SC.W conditonal store on Boom and Rocket. issue can be reproduced with the code snippet in Listing 4. 1 auipc t0 , 0 xff // t0 = PC Listing 4: Proof of concept code snippets to trigger improper 23 c. lui t1 , 0 x2 // t1 = 0x2000 Handling of PMP execute-only memory regions on BOOM. lr .d t3 , ( t0 ) // Reserve on [t0] 4 sd t1 , 512( t0 ) // 1 auipc t0 ,0 x0 // t0 = PC 5 sc .d t3 , t1 , ( t0 ) // Store Conditional on [ 2 addi t0 ,t0 ,0 x43 // t0 = PC + 0x43 ,→ t0] 3 addi t1 ,x0 ,0 x2 // t1 = 0x2 4 srl t1 ,a4 , t1 5 addi t2 ,x0 ,0 x94 // Set R=0, W=0, X=1 However, GenHuzz successfully generated a test case that 6 csrw pmpaddr0 , t1 intelligently reserves an address using the LR instruction. Fol- 7 csrw pmpcfg0 , t2 lowing this, GenHuzz, guided by the inferred semantics of the 8 ... // v-- Mem offset = 0x43 --v RISC-V ISA specification [49, 50], generates a store instruc- 9 auipc,→a4 , 0 x0 // @Exception Instruction Access Fault tion targeting a random address within the program’s address 10 lbu a4 ,0( a4 ) space. It then issues a conditional store instruction (SC) to the same reserved address. This carefully constructed sequence In the BOOM core, executing the Listing 4 results in an of instructions demonstrates GenHuzz’s ability to understand Instruction Access Fault. This inconsistency suggests and apply several key considerations: 1) the semantics of a flaw in the PMP implementation, as other RISC-V cores RISC-V assembly language, ensuring that LR and SC instruc- like Rocket [33], CVA6 [35], and the Spike [43] simulator tions are ordered correctly and related within a program, 2) correctly allow execution within execute-only regions without

USENIX Association 34th USENIX Security Symposium 1797

raising an exception. This incorrect behavior in the BOOM stagnate. We conduct an ablation study to assess its effective- core could lead to unexpected behavior, i.e., CWE-1281, po- ness by disabling the reset module and comparing the results tentially affecting system stability and the correct execution against the original implementation. As shown in the green of software that relies on execute-only regions for code pro- line in Fig. 7, coverage growth plateaus prematurely without tection and security. the reset module. In contrast, with the reset module engaged, GenHuzz demonstrates sustained coverage improvement over 7 Performance Evaluation time. This finding confirms that the reset module is crucial in encouraging broader state exploration and preventing the In this section, we investigate GenHuzz ’s performance by fuzzing process from getting stuck in suboptimal policy. evaluating three key aspects: the stability of the fuzzing pro- Impact of Reward Assignment. The reward structure in cess, the necessity of the reset module, and the impact of GenHuzz is designed to guide the fuzzer toward reaching different reward assignments. Aligned with Fig. 6, we show significant hardware states while ensuring the validity and experimental results on RocketChip with condition coverage diversity of generated test cases. To evaluate the impact of metric in Fig. 7. different reward components on performance, we conducted experiments using GenHuzz with only the hardware coverage reward, excluding the additional terms rvalid and rbonus from 0.810 Eq. 5. As shown in Fig. 7, this simplified reward structure still allows GenHuzz to improve coverage. However, the rate of 0.805 improvement and the final coverage achieved are both lower compared to the comprehensive reward setup. The absence z 0.800 of rvalid often results in the fuzzer generating syntactically 5 0.795 incorrect instructions, rendering the test cases invalid. While ©5 pd the reset module can mitigate this issue by reinitializing the 0.790 , fuzzer, the lack of rbonus leads to another challenge: the gen- 0.7851;[ —— Original (mean and variance) eration of repetitive instructions. This phenomenon, known -==- No reset module as reward hacking [51], occurs when the fuzzing policy max- ! Rewards with HW imizes the reward without fulfilling the intended objective. 0780 0 10000 20000 30000 40000 50000 Incorporating rbonus addresses this issue by encouraging the Number of test cases fuzzer to explore new states, resulting in better overall per- formance. These findings highlight that additional incentives Figure 7: Performance evaluation of GenHuzz. for generating valid and diverse test cases, beyond merely maximizing coverage, are crucial for achieving richer state Stability of the Fuzzer. We repeat the fuzzing process ten exploration and superior outcomes. times under different random seeds to validate that GenHuzz ’s results remain stable despite inherent randomness. Each run 8 begins with a unique initial condition, and we compare the Discussion resulting coverage trends and final coverage values. As shown GenHuzz is a novel hardware fuzzing framework that com- in Fig. 7 (the mean and the standard deviation are in red and bines reinforcement learning (RL) and language models (LM) pink, respectively), GenHuzz consistently converges to high to address the challenge of generating test cases that can coverage levels regardless of the starting seed. Moreover, the trigger deeper vulnerabilities. While RL has been explored lower bound consistently outperforms all other fuzzers (see in software fuzzing, hardware fuzzing presents unique chal- Fig. 6), demonstrating its robust performance. Indeed, three lenges, such as tighter coupling between instructions and key mechanisms foster this stability: 1) the HGRL frame- states, deterministic timing, and a more complex state space. work constrains policy updates, preventing drastic shifts and Impact of RL and LM Integration. Unlike conventional maintaining smoother training dynamics; 2) the hardware RL-based fuzzers, which rely primarily on adaptive search coverage metric directly guides the instruction generator’s policies without deeper semantic insights, GenHuzz integrates fine-tuning, ensuring a focus on producing more effective test RL with LM to generate contextually meaningful instruction cases; and 3) the reset module actively monitors progress, in- sequences. By enriching the search process with a model that tervening to prevent the model from settling into local optima. understands the structure and semantics of the code, Gen- These design choices ensure that GenHuzz ’s RL-language Huzz can systematically identify coverage gaps and detect model-driven approach is inherently stable. nuanced vulnerabilities. This semantic-driven strategy, com- The Need for the Reset Module. GenHuzz incorporates a monly called semantic-aware fuzzing [52], allows GenHuzz to reset module to break out of local optima. This module is uncover vulnerabilities often missed by random or mutation- activated dynamically when the fuzzing process begins to based fuzzing methods.

1798 34th USENIX Security Symposium USENIX Association

ID Core Bug Description New CVSS CWE V1 CVA6 Bypassing PMP Memory Protection in NA4 Mode ✔ 9.3 1281 V2 CVA6 Incorrect Exception Handling for Misaligned Load-Reserved ✔ 7.3 1298 V3 CVA6 Incorrect tval Register Value for Breakpoint Exceptions ✔ 6.8 1281 V4 CVA6 Incorrect NaN Boxing in FP fsqrt.s Instruction ✔ 6.8 1281 V5 CVA6 Incorrect NaN Boxing in Single-Precision FP Division Instructions ✔ 6.8 1281 V6 CVA6 Missing exceptions for some illegal instructions ✗ 4.0 1281 V7 CVA6 Incorrect Setting of NX Flag for FP Instructions ✗ 5.1 1281 V8 BOOM Any store-related instruction breaks the LR and SC ✔ 8.5 1281 V9 BOOM Improper Handling of Execute-Only Memory Regions in PMP ✔ 8.5 1281 V10 BOOM No exception for invalid rm in single-precision operation ✔ 4.4 1281 V11 BOOM Improper Zeroisation of minstret CSR ✗ 6.5 1239 V12 RocketChip Any store-related instruction breaks the LR and SC ✔ 8.5 1281 V13 RocketChip No exception for invalid rm in single-precision operation ✔ 4.4 1281 V14 RocketChip Incorrect minstret value for after EBREAK ✗ 3.3 1281

Table 1: Detected Vulnerabilities and Bugs (The Maximum CVSS Score is 10). CWE (Common Weakness Enumeration) is a standardized list of software and hardware vulnerability categories. Since detected bugs and vulnerabilities are inside the hardware, the CWE column only contains hardware CWEs. The corresponding Common Vulnerabilities and Exposures (CVE) numbers are detailed in the Open Science section.

Computation Efficiency. The computational efficiency of volve multiple instructions. Integrating LLM with HGRL opti- a fuzzer directly influences its overall performance. In Gen- mizes the process by reducing the test case number to achieve Huzz, the main overhead arises from 1) test case generation comprehensive coverage, thus minimizing the computational (Section 3.2), 2) HGRL processing (Section 3.3), and 3) in- cost of hardware fuzzing. strumentation. Since GenHuzz uses the same DUT as prior Adaptability. One key strength of GenHuzz is its adaptability work, instrumentation time remains the same across fuzzers. to almost any RISC-V implementation, enabled by RISC-V’s The HGRL module imposes negligible overhead, complet- open nature and the abundance of modifiable RTL (Regis- ing tuning in under a second. Therefore, we assess GenHuzz ter Transfer Level) designs. For GenHuzz (or any white-box ’s efficiency primarily via test case generation time. Addi- fuzzing framework) to be transferable to ARM and x86 ar- tionally, computational resources significantly affect fuzzing chitectures, three key requirements must be met. First, there efficiency. Although GenHuzz executes largely on a GPU, must be open-source RTL designs for the target ARM and we measure GenHuzz’s CPU and GPU performance against x86 cores. These designs should be sufficiently detailed and state-of-the-art fuzzers for a fair comparison. functional to enable accurate simulation and vulnerability de- On an AMD EPYC 9684X CPU, GenHuzz demonstrates an tection. Next, GenHuzz requires coverage feedback from the execution time of 4.46 seconds per test case while achieving DUT to guide test case refinement. For RISC-V, open-source a significantly reduced execution time of 1.58 seconds on an cores provide direct access to such feedback. For ARM and A6000 GPU. Although the CPU overhead is marginally higher x86, standardized methods for extracting coverage feedback than that of state-of-the-art fuzzers such as CASCADE (2.06 (either via open hardware or specialized development kits) seconds) and TheHuzz (2.47 seconds), this discrepancy arises must be established. Finally, an accessible and modifiable from the fact that machine learning models are inherently toolchain, including simulators and debuggers for ARM and optimized for efficient execution on GPUs. Consequently, the x86, is crucial for pre-training and GenHuzz’s pipeline. CPU overhead observed in GenHuzz is slightly larger. The ad- Unfortunately, no fully open-source, standardized ARM or ditional overhead arises from the real-time tuning mechanism x86 cores exist, as both ISAs are commercially licensed with of GenHuzz (see Fig. 4), which restricts high parallelization closed RTL. Lacking modifiable RTL prevents detailed cov- during the generation of test cases. When fully utilizing paral- erage extraction, which is the foundation of GenHuzz. Be- lelization in input generation, GenHuzz achieves a throughput sides, proprietary simulators typically do not offer the level of more than 10 000 test cases per second. Additionally, even of transparency required for white-box fuzzing. The absence in a non-parallelized configuration, GenHuzz demonstrates of open-source RTL designs and accessible toolchains for superior efficiency by effectively exploring hardware states, ARM and x86 not only limits the adaptability of GenHuzz achieving comparable coverage to state-of-the-art fuzzers but also highlights a broader issue in hardware security re- while using only 1% of the test cases. Furthermore, it shows search. Proprietary architectures hinder efforts to develop and effectiveness in uncovering complex vulnerabilities that in- evaluate novel hardware vulnerability detection techniques.

USENIX Association 34th USENIX Security Symposium 1799

This limitation underscores the need for increased openness security properties (generic cost function) that detect vulner- or at least controlled accessibility within the ARM and x86 abilities in the DUT. Finally, Cascade [27] aims to improve development ecosystems. We strongly advocate for greater instruction execution efficiency by building long programs availability of open-source or accessible designs for ARM and eliminating control flow influences. Unfortunately, overly and x86 architectures. Such access would enable researchers long and complex test cases lead to high time consumption. to apply advanced techniques like white-box fuzzing, uncover The overlook of input semantics and low utilization of the previously unknown hardware vulnerabilities, and develop hardware feedback constrains the test case’s variety and po- robust defenses. Moreover, fostering an open research envi- tentially reduces the bug detection capability. ChatFuzz [25] ronment around these architectures would accelerate innova- employs reinforcement learning to guide fuzz test case gen- tion in offensive and defensive hardware security, benefiting eration but fine-tunes a pre-trained human language model, academia and industry. resulting in suboptimal performance. In contrast, GenHuzz is designed specifically for hardware fuzzing, leveraging a cus- 9 Related Work tomized GPT model integrated with a tailored HGRL pipeline. By incorporating comprehensive hardware coverage into the Generic Fuzzers. Laeufer et al. propose RFuzz [53], the first fuzzing policy refinement, GenHuzz efficiently uncovers sub- FPGA emulation-based generic hardware fuzzing technique, tle vulnerabilities while maintaining high test case efficiency. which relies on multiplexer control signals to generate in- puts based on AFL-based mutation functions. However, this 10 Conclusions method is contained by its coverage metrics detecting vulner- abilities and is computationally expensive. Li et al. proposed We introduce a novel white-box hardware fuzzer, GenHuzz, a new metric, Full Multiplexer Toggle Coverage, for better designed to actively interact with the Device Under Test hardware coverage performance [54]. Both approaches are (DUT) to refine its fuzzing policy, thereby significantly en- closely tied to the Chisel HDL, limiting their broader appli- hancing vulnerability and bug detection capabilities with a cability, while monitoring multiplexers in complex designs limited number of test cases. By leveraging a customized introduces significant performance overhead [31]. In contrast, language model-based fuzzer with a hardware-guided rein- Trippel et al. [23] proposed fuzzing hardware-like software forcement learning (HGRL) framework, GenHuzz achieves by fuzzing the hardware simulation binary rather than porting broader hardware state coverage compared to existing fuzzers. software fuzzers directly on the hardware designs. Verila- The fuzzing results on three RISC-V cores show that Gen- tor [55] translates the hardware to an equivalent software Huzz identifies all previously reported bugs and uncovers model. This approach allows hardware fuzzing to utilize ex- ten new vulnerabilities, including five of high severity. De- isting coverage metrics commonly used by software fuzzers, tecting these vulnerabilities requires complex combinations such as basic block and edge coverage [31]. Still, it faces of multiple instructions, which are nearly impossible to con- scalability problems when, e.g., fuzzing an entire processor. struct using existing fuzzing techniques. In contrast, GenHuzz Processor Fuzzers. TheHuzz [20] simulates the RTL design successfully generates these instruction sequences by under- of the processor with the binary format of the instruction us- standing inter-instruction semantics. ing Synopsys VCS [48] that traces the code coverage through various metrics, including branch, condition, toggle, FSM, Acknowledgment and functional coverage. However, this method suffers from low computation efficiency and hardware coverage: the code Our research work was partially funded by Intel’s Scalable As- coverage is only 2.86% higher than that of random methods. surance Program, Deutsche Forschungsgemeinschaft (DFG) – DifuzzRTL [28] generates instructions and collects control SFB 1119 – 236615297, the European Union under Horizon register coverage to guide the fuzzing process. Following Europe Programme – Grant Agreement 101070537 – Cross- this work, MorFuzz [29] achieves a final coverage that is 4.4 Con, and the European Research Council under the ERC Pro- times higher than DifuzzRTL by generating fuzzing seeds gramme - Grant 101055025 - HYDRANOS. This work does based on syntax and semantics and using run-time informa- not in any way constitute an Intel endorsement of a product or tion feedback to mutate instructions. ProcessorFuzz [31] is a supplier. Any opinions, findings, conclusions, or recommen- concurrent work that generates instructions and collects cov- dations expressed herein are those of the authors and do not erage of control and status registers. However, these works necessarily reflect those of Intel, the European Union, and the only focus on the coverage of registers generating the select European Research Council. signals of MUXes, leading to missing bugs and vulnerabilities. To increase design coverage and detect more vulnerabilities, HyPFuzz [24] was proposed to guide the fuzzer using formal verification tools that reach hard-to-reach design spaces. Al- ternatively, SoC Fuzzer [32] directs the fuzzing based on the

1800 34th USENIX Security Symposium    USENIX Association

Ethics and Disclosures References

GenHuzz is a hardware fuzzing tool developed to advance the [1] EMVCo, “EMV specifications (2001),” functional and security verification of hardware, particularly https://www.emvco.com/. processors. Its intended users include security researchers, [2] I. 15408, “Common criteria v3.1 (2017),” hardware manufacturers and designers, and hardware secu- https://www.commoncriteriaportal.org/cc/index.cfm?. rity companies. By using the GenHuzz framework, users can discover new bugs and vulnerabilities in hardware designs [3] O. Mutlu and J. S. Kim, “Rowhammer: A retrospec- under test. We thoroughly evaluated GenHuzz on three widely tive,” IEEE Transactions on Computer-Aided Design recognized benchmarks, discovering ten new bugs and vul- of Integrated Circuits and Systems, vol. 39, no. 8, pp. nerabilities. In line with the Menlo Report principles [56], 1555–1571, 2019. particularly the principle of "Respect for Persons", we have [4] U. Rührmair, J. Sölter, F. Sehnke, X. Xu, A. Mahmoud, ensured that the security vulnerabilities identified by Gen- V. Stoyanova, G. Dror, J. Schmidhuber, W. Burleson, Huzz were promptly communicated to the responsible teams and S. Devadas, “Puf modeling attacks on simulated for CVA6 [35], BOOM [34], and RocketChip [33] in August and silicon data,” IEEE transactions on information 2024. This timely disclosure was essential to mitigate risks forensics and security, vol. 8, no. 11, pp. 1876–1891, and protect individuals from potential harm that could arise 2013. if adversaries were to uncover these vulnerabilities indepen- dently. The responsible parties have acknowledged the issues [5] P. Kocher, J. Horn, A. Fogh, D. Genkin, D. Gruss, and are actively working on implementing fixes to address W. Haas, M. Hamburg, M. Lipp, S. Mangard, T. Prescher the concerns raised in this paper. et al., “Spectre attacks: Exploiting speculative execu- To further adhere to the principle of "Justice", which requires tion,” Communications of the ACM, vol. 63, no. 7, pp. fairness in distributing benefits and burdens, we have carefully 93–101, 2020. considered the potential risks associated with the misuse of [6] M. Lipp, M. Schwarz, D. Gruss, T. Prescher, W. Haas, GenHuzz. To prevent harm and ensure that the framework is S. Mangard, P. Kocher, D. Genkin, Y. Yarom, used to advance research and enhance hardware security, ac- and M. Hamburg, “Meltdown,” arXiv preprint cess to the source code for GenHuzz will be restricted. It will arXiv:1801.01207, 2018. be made available only upon request and with confirmation that it will be used exclusively for responsible research by [7] C. Canella, D. Genkin, L. Giner, D. Gruss, M. Lipp, academic users and hardware manufacturers. This approach M. Minkin, D. Moghimi, F. Piessens, M. Schwarz, aligns with the Menlo Report [56] emphasis on promoting B. Sunar et al., “Fallout: Leaking data on meltdown- social value while protecting individual rights and privacy, resistant cpus,” in Proceedings of the 2019 ACM ensuring that GenHuzz contributes positively to the hardware SIGSAC Conference on Computer and Communications security community without introducing unnecessary risks. Security, 2019, pp. 769–784. [8] MITRE, “Hardware design cwes,” https://cwe.mitre.org/ Open Science data/definitions/1194.html, 2019. [9] Intel, “Machine check error avoidance on page size In alignment with USENIX Security’s open science policy, we change/cve2018-12207,” https://www.intel.com/ release the GenHuzz framework and all vulnerability-related content/www/us/en/developer/articles/troubleshooting/ test cases in https://zenodo.org/records/14727632, en- software-security-guidance/advisory-guidance/ abling the community to validate and extend our work. machine-check-error-avoidance-page-size-change. In our responsible disclosure, we received confirmation from html, 2019. the responsible teams and requested Common Vulnerabili- [10] S. R. Sarangi, A. Tiwari, and J. Torrellas, “Phoenix: De- ties and Exposures (CVE)3 numbers for these vulnerabili- tecting and recovering from permanent processor design ties. Upon submission, CVE numbers were assigned for V2 bugs with programmable hardware,” in 2006 39th An- (CVE-2024-44791), V4 (CVE-2024-44790), V5 (CVE-2024- nual IEEE/ACM International Symposium on Microar- 47789), V8 (CVE-2024-46029), and V12 (CVE-2023-46028). chitecture (MICRO’06). IEEE, 2006, pp. 26–37. The CVE numbers for the remaining vulnerabilities are pend- ing and will be updated on the above link once they are issued. [11] C. Deutschbein and C. Sturton, “Mining security critical linear temporal logic specifications for processors,” in 3CVE is a system for identifying and naming publicly known vulnerabili- 2018 19th International Workshop on Microprocessor ties. Each CVE entry is assigned to one vulnerability and has a unique ID and SOC Test and Verification (MTV). IEEE, 2018, pp. and basic details about a vulnerability. 18–23.

USENIX Association 34th USENIX Security Symposium 1801

[12] B. Wile, J. Goss, and W. Roesner, Comprehensive func- [22] Google, “Americal fuzzy loop,” https://github.com/ tional verification: The complete industry cycle. Mor- google/AFL, 2019. gan Kaufmann, 2005. [23] T. Trippel, K. G. Shin, A. Chernyakhovsky, G. Kelly, [13] G. Dessouky, D. Gens, P. Haney, G. Persyn, A. Kanu- D. Rizzo, and M. Hicks, “Fuzzing hardware like soft- parthi, H. Khattri, J. M. Fung, A.-R. Sadeghi, and ware,” in 31st USENIX Security Symposium (USENIX J. Rajendran, “{HardFails}: insights into {software- Security 22), 2022, pp. 3237–3254. exploitable} hardware bugs,” in 28th USENIX Security [24] C. Chen, R. Kande, N. Nguyen, F. Andersen, A. Tyagi, Symposium (USENIX Security 19), 2019, pp. 213–230. A.-R. Sadeghi, and J. Rajendran, “{HyPFuzz}:{Formal- [14] E. M. Clarke, W. Klieber, M. Nováˇcek, and P. Zu- Assisted} processor fuzzing,” in 32nd USENIX Secu- liani, “Model checking and the state explosion prob- rity Symposium (USENIX Security 23), 2023, pp. 1361– lem,” in LASER Summer School on Software Engineer- 1378. ing. Springer, 2011, pp. 1–30. [25] M. Rostami, M. Chilese, S. Zeitouni, R. Kande, J. Ra- [15] I. Wagner and V. Bertacco, “Engineering trust with se- jendran, and A.-R. Sadeghi, “Beyond random inputs: mantic guardians,” in 2007 Design, Automation & Test A novel ml-based hardware fuzzing,” in 2024 Design, in Europe Conference & Exhibition. IEEE, 2007, pp. Automation & Test in Europe Conference & Exhibition 1–6. (DATE). IEEE, 2024, pp. 1–6. [16] M. Hicks, C. Sturton, S. T. King, and J. M. Smith, [26] M. Rostami, S. Zeitouni, R. Kande, C. Chen, P. Mah- “Specs: A lightweight runtime mechanism for protecting moody, J. Rajendran, and A.-R. Sadeghi, “Lost and software from security-critical processor bugs,” in Pro- found in speculation: Hybrid speculative vulnerability ceedings of the Twentieth International Conference on detection,” in 61st ACM/IEEE Design Automation Con- Architectural Support for Programming Languages and ference (DAC ’24), June 23–27, 2024, San Francisco, Operating Systems, 2015, pp. 517–529. CA, USA, ser. DAC ’22, 2024, pp. 192–197. [27] F. Solt, K. Ceesay-Seitz, and K. Razavi, “Cascade: Cpu [17] X. Li, M. Tiwari, J. K. Oberg, V. Kashyap, F. T. Chong, fuzzing via intricate program generation,” in Proc. 33rd T. Sherwood, and B. Hardekopf, “Caisson: a hardware USENIX Secur. Symp, 2024, pp. 1–18. description language for secure information flow,” ACM Sigplan Notices, vol. 46, no. 6, pp. 109–120, 2011. [28] J. Hur, S. Song, D. Kwon, E. Baek, J. Kim, and B. Lee, “Difuzzrtl: Differential fuzz testing to find cpu bugs,” in [18] X. Li, V. Kashyap, J. K. Oberg, M. Tiwari, V. R. Ra- 2021 IEEE Symposium on Security and Privacy (SP). jarathinam, R. Kastner, T. Sherwood, B. Hardekopf, and IEEE, 2021, pp. 1286–1303. F. T. Chong, “Sapper: A language for hardware-level [29] J. Xu, Y. Liu, S. He, H. Lin, Y. Zhou, and C. Wang, security policy enforcement,” in Proceedings of the 19th “{MorFuzz}: Fuzzing processor via runtime instruction international conference on Architectural support for morphing enhanced synchronizable co-simulation,” in programming languages and operating systems, 2014, 32nd USENIX Security Symposium (USENIX Security pp. 97–112. 23), 2023, pp. 1307–1324. [19] D. Zhang, Y. Wang, G. E. Suh, and A. C. Myers, “A hard- [30] P. Borkar, C. Chen, M. Rostami, N. Singh, R. Kande, A.- ware design language for timing-sensitive information- R. Sadeghi, C. Rebeiro, and J. Rajendran, “Whisperfuzz: flow security,” Acm Sigplan Notices, vol. 50, no. 4, pp. White-box fuzzing for detecting and locating timing 503–516, 2015. vulnerabilities in processors,” 2024. [Online]. Available: [20] R. Kande, A. Crump, G. Persyn, P. Jauernig, A.-R. https://arxiv.org/abs/2402.03704 Sadeghi, A. Tyagi, and J. Rajendran, “{TheHuzz}: [31] S. Canakci, C. Rajapaksha, L. Delshadtehrani, Instruction fuzzing of processors using {Golden- A. Nataraja, M. B. Taylor, M. Egele, and A. Joshi, Reference} models for finding {Software-Exploitable} “Processorfuzz: Processor fuzzing with control and vulnerabilities,” in 31st USENIX Security Symposium status registers guidance,” in 2023 IEEE International (USENIX Security 22), 2022, pp. 3219–3236. Symposium on Hardware Oriented Security and Trust [21] M. Rostami, C. Chen, R. Kande, H. Li, J. Rajendran, (HOST). IEEE, 2023, pp. 1–12. and A.-R. Sadeghi, “Fuzzerfly effect: Hardware fuzzing [32] M. M. Hossain, A. Vafaei, K. Z. Azar, F. Rahman, for memory safety,” IEEE Security and Privacy, vol. 22, F. Farahmandi, and M. Tehranipoor, “Socfuzzer: Soc no. 4, pp. 76–86, 2024. vulnerability detection using cost function enabled fuzz

1802 34th USENIX Security Symposium    USENIX Association

testing,” in 2023 Design, Automation & Test in Europe [45] I. Shumailov, Z. Shumaylov, Y. Zhao, Y. Gal, N. Paper- Conference & Exhibition (DATE). IEEE, 2023, pp. not, and R. Anderson, “The curse of recursion: Training 1–6. on generated data makes models forget,” arXiv preprint [33] A. et al., “The Rocket Chip Generator,” arXiv:2305.17493, 2023. no. UCB/EECS-2016-17, 2016. [Online]. Avail- [46] NXP, “Introduction to device trees,” https://www.nxp. able: http://www2.eecs.berkeley.edu/Pubs/TechRpts/ com/docs/en/application-note/AN5125.pdf, 2015. 2016/EECS-2016-17.html [47] M. Andrychowicz, A. Raichuk, P. Sta´nczyk, M. Orsini, [34] J. Zhao, B. Korpan, A. Gonzalez, and K. Asanovic, “Son- S. Girgin, R. Marinier, L. Hussenot, M. Geist, icBOOM: The 3rd Generation Berkeley Out-of-Order O. Pietquin, M. Michalski et al., “What matters for on- Machine,” 4th Workshop on Computer Architecture Re- policy deep actor-critic methods? a large-scale study,” search with RISC-V, 2020. in International conference on learning representations, [35] F. Zaruba and L. Benini, “The cost of application-class 2021. processing: Energy and performance analysis of a linux- [48] Synopsys, “Vcs, the industry’s highest performance ready 1.7-ghz 64-bit risc-v core in 22-nm fdsoi tech- simulation solution,” https://www.synopsys.com/ nology,” IEEE Transactions on Very Large Scale Inte- verification/simulation/vcs.html. gration (VLSI) Systems, vol. 27, no. 11, pp. 2629–2640, [49] RISC-V, “The risc-v instruction set man- 2019. ual volume i: Unprivileged isa,” https: [36] NIST, “Common vulnerability scoring system,” https: //github.com/riscv/riscv-isa-manual/releases/tag/ //nvd.nist.gov/vuln-metrics/cvss/v3-calculator, 2019. riscv-isa-release-8b9dc50-2024-08-30, 2024. [37] R. Saravanan and S. M. Pudukotai Dinakarrao, “The [50] ——, “The risc-v instruction set manual volume ii: Priv- fuzz odyssey: A survey on hardware fuzzing frameworks ileged isa,” https://github.com/riscv/riscv-isa-manual/ for hardware design verification,” in Proceedings of the releases/tag/riscv-isa-release-8b9dc50-2024-08-30, Great Lakes Symposium on VLSI 2024, 2024, pp. 192– 2024. 197. [51] D. Amodei, C. Olah, J. Steinhardt, P. Christiano, J. Schul- [38] R. S. Sutton and A. G. Barto, Reinforcement learning: man, and D. Mané, “Concrete problems in ai safety,” An introduction. MIT press, 2018. arXiv preprint arXiv:1606.06565, 2016. [39] B. Mahesh, “Machine learning algorithms-a re- [52] R. Padhye, C. Lemieux, K. Sen, M. Papadakis, and view,” International Journal of Science and Research Y. Le Traon, “Semantic fuzzing with zest,” in Proceed- (IJSR).[Internet], vol. 9, no. 1, pp. 381–386, 2020. ings of the 28th ACM SIGSOFT International Sympo- sium on Software Testing and Analysis, 2019, pp. 329– [40] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, 340. L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “At- tention is all you need,” Advances in neural information [53] K. Laeufer, J. Koenig, D. Kim, J. Bachrach, and K. Sen, processing systems, vol. 30, 2017. “Rfuzz: Coverage-directed fuzz testing of rtl on fp- gas,” in 2018 IEEE/ACM International Conference on [41] A. Radford, K. Narasimhan, T. Salimans, and Computer-Aided Design (ICCAD). IEEE, 2018, pp. I. Sutskever, “Improving language understanding with 1–8. unsupervised learning,” 2018. [54] T. Li, H. Zou, D. Luo, and W. Qu, “Symbolic simulation [42] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, enhanced coverage-directed fuzz testing of rtl design,” “Bert: Pre-training of deep bidirectional transform- in 2021 IEEE International Symposium on Circuits and ers for language understanding,” arXiv preprint Systems (ISCAS). IEEE, 2021, pp. 1–5. arXiv:1810.04805, 2018. [55] Verilator, “Welcome to verilator,” https://www.veripool. [43] RISC-V, “Spike risc-v isa simulator,” https://github.com/ org/verilator/, 2019. riscv-software-src/riscv-isa-sim. [56] “The menlo report: Ethical principles guiding in- [44] E. Barron and H. Ishii, “The bellman equation for mini- formation and communication technology research,” mizing the maximum cost.” NONLINEAR ANAL. THE- https://www.dhs.gov/sites/default/files/publications/ ORY METHODS APPLIC., vol. 13, no. 9, pp. 1067– CSD-MenloPrinciplesCORE-20120803_1.pdf, 2012. 1090, 1989.

USENIX Association 34th USENIX Security Symposium 1803

Appendix Misaligned exception is correctly raised, aligning with the expected behavior. A Fuzzer Training Pipeline The incorrect exception handling introduces a critical issue, potentially causing confusion in software error handling rou- The training pipeline of the fuzzer is summarized as follows: tines and leading to improper software responses to alignment First, word embeddings are generated. Positional and segment errors. embeddings are then added to these word embeddings, with Incorrect tval Register Value for Breakpoint and Fault positional embeddings indicating each token’s position in the Exceptions (V3). In the RISC-V architecture, the tval, i.e., sequence and segment embeddings distinguishing between trap value, a register is used to hold additional information different input segments. This sequence of embeddings is fed about exceptions that occur during instruction fetches, loads, through attention layers that use a multi-head masked self- or stores. Based on RISC-V ISA specification [49], When attention mechanism to produce rich contextual embeddings. a breakpoint, address-misaligned, access-fault or page-fault These embeddings capture the context of each token by incor- exception occurs, tval should contain the faulting virtual porating information from other tokens in the sequence. The address if the exception is related to an instruction fetch, contextual embedding of the final token is passed to the clas- load, or store and if a nonzero value is written to tval. This sification head, which generates unnormalized probabilities mechanism helps debug by providing the exact virtual address for each token in the tokenizer’s dictionary. A single token is where the exception occurred. sampled from these probabilities and appended to the end of However, in the latest version of the CVA6 core, when such the sequence. exceptions occur, the instruction value (i.e., the opcode of the instruction that caused the exception) is incorrectly placed B Bugs and Vulnerabilities in the tval register instead of the faulting virtual address. This behavior can be reproduced with the code snippets in Incorrect Exception Handling for Misaligned Load- Listing 6. Reserved Instructions (V2). In the RISC-V architecture, Load-Reserved (LR.W, LR.D) and Store-Conditional (SC.W, Listing 6: Proof of concept code snippets to trigger tval SC.D) instructions are intended to perform atomic read- CVA6 modify-write operations. These operations require the target 1 c. ebreak addresses to properly align according to the instruction size 2 // tval= 0x9002 which is the opcode of c. to ensure atomicity and data integrity. For example, the LR.D ,→ ebreak instruction expects an eight-byte aligned address for a double- word load. If a misaligned address is provided, the RISC-V When the PoC code is executed on the CVA6 core, the tval specification dictates that a Load Address Misaligned ex- register is set to 0x9002, which corresponds to the opcode ception should be raised to indicate the improper alignment of the c.ebreak instruction. This is incorrect because the of the memory access. RISC-V specification expects the tval register to contain However, in the CVA6 [35] core, executing a Load-Reserved the program counter PC value at the time of the exception, instruction such as LR.D with a misaligned address incorrectly not the opcode of the instruction that caused the exception. triggers a Store Address Misaligned exception instead In contrast, executing the same code on the Spike RISC-V of the expected Load Address Misaligned exception. This ISA simulator correctly places the pc value in the tval reg- deviation from the expected behavior can be reproduced with ister, demonstrating adherence to the RISC-V specification. the code snippets in Listing 5. This incorrect behavior in the CVA6 core represents a signifi- cant deviation from the RISC-V specification and can lead to Listing 5: Proof of concept code snippets to trigger misaligned difficulties in debugging and error handling. on CVA6 Incorrect NaN Boxing in Single-Precision Floating-Point 1 // Load an address in t0 which is not 8-byte SQRT Instruction (V4). In the RISC-V architecture, single- ,→ aligned, e.g., 0x0000000080001054 precision floating-point operations should adhere to the NaN- 2 auipc t0 ,0 x0 // t0 = PC = 0x0000000080001054 boxing rules as specified in the RISC-V ISA manual [49, 50]. 34 lr .d s1 , ( t0 ) NaN-boxing is a mechanism to extend narrower floating-point // Exception @ Cause: Store Address Misaligned values into wider registers while maintaining compatibility, e.g., in Machine Learning algorithms. According to the RISC- When the code is executed on the CVA6 core, an excep- V ISA specification [49, 50], when an operation like SQRT.S tion with the cause Store Address Misaligned is thrown, (square root, single precision) is executed with a value that which is incorrect according to the RISC-V specification is not properly NaN-boxed, i.e., where the most significant 32 [49, 50]. On the other hand, when the same code is exe- bits are not all set to 1, the result should be a 32-bit canonical cuted on the Spike RISC-V ISA simulator, a Load Address NaN, i.e., 0x7fc00000 extended appropriately for the wider

1804 34th USENIX Security Symposium USENIX Association

floating-point register. Listing 8: Proof of concept code snippets to trigger incor- rect NaN boxing and exception Handling in single-precision Listing 7: Proof of concept code snippets for incorrect NaN floating-Point division on CVA6 [35] boxing in single-precision floating-point SQRT instruction on 1 CVA6 [35] 2 lui t5 , 0 xbd131 fmv .w.x ft3 , t5 // ft3 = 0xffffffffbd131000 1 lui t0 , 0 x3d0f4 3 lui s4 , 0 xee72f 2 // t0 :000000003d0f4000 4 fmv .w.x fa7 , s4 // fa7 = 0xffffffffee72f000 3 fcvt .d.w ft5 , t0 5 fcvt .d.l fa7 , s4 // fa7 = 0xc1b18d1000000000 4 // ft5 :41ce87a000000000 6 fdiv .s, ft3 , ft3 , fa7 // ft3 = 0 5 fsqrt .s fa5 , ft5 7 → xffffffffff800000 6 // fa5 :ffffffff00000000 csrr a4 , fflags // a4 = 0x0000000000000008 7 csrr gp , fflags

However, in CVA6 cores, executing a single-precision accurate floating-point calculations. This is particularly con- floating-point square root operation on an invalid NaN-boxed cerning for applications in machine learning, where precise value incorrectly results in a boxed single-precision floating- floating-point arithmetic is essential for tasks such as floating- point value of 0xffffffff00000000. This is incorrect ac- point precision conversion, a process that is widely used. Er- cording to the RISC-V specification [49, 50], which requires rors in these calculations can lead to inaccuracies in model the result to be a 32-bit canonical NaN. The incorrect be- training, predictions, and overall system performance, poten- havior can be reproduced with the code in listing 7. When tially compromising the reliability and effectiveness of ma- the code in listing 7 executed on the CVA6 core, the fa5 chine learning algorithms. register contains the value 0xffffffff00000000, which is No exception for invalid rm in single-precision not the correct canonical NaN. In contrast, executing the operation(V10). same code on the Spike RISC-V ISA simulator results in In the RISC-V architecture, the rounding mode (rm) field ft5=0xffffffff7fc00000, correctly adhering to the NaN- in single-precision floating-point instructions specifies the boxing rules specified in the RISC-V ISA manual. rounding behavior for floating-point operations. This field is This incorrect handling of NaN-boxing in the CVA6 core part of the floating-point control and status register (fcsr) can lead to significant software issues that rely on accurate and guides how the processor should manage rounding when floating-point arithmetic, particularly in machine learning the result of a floating-point calculation cannot be precisely training, cryptography computation, or any domain requiring represented. The rounding mode can be directly encoded precise floating-point operations. The incorrect result returned within the instruction (rm field) or dynamically set using the from floating-point instructions could lead to undetected flaws, floating-point rounding mode register (frm). If an invalid potentially impacting the stability of software systems and rounding mode value (101–111 based on RISC-V ISA Speci- consequently leading to security issues, e.g., decrease the fication [49,50]) is specified, the architecture requires that any quality of machine learning inference. floating-point operation using this invalid mode must trigger Incorrect NaN Boxing and Exception Handling in Single- an illegal instruction exception, as outlined in the RISC-V Precision Floating-Point Division (V5). As mentioned in specification [49, 50]. appendix B, in the RISC-V architecture, single-precision Using GenHuzz, we generated a test case that inten- floating-point operations are expected to follow specific rules tionally included an invalid rounding mode in a single- for handling NaN-boxed values. The GenHuzz found that in precision floating-point operation. Upon running this test the CVA6 core, performing any single-precision floating-point case, it was observed that the BOOM core did not raise a division with an invalid NaN-boxed value incorrectly results trap_illegal_instruction exception, as would be ex- in a non-canonical value 0xffffffffff800000 and raises pected when executing instructions with an invalid rm field. the "divide by zero" exception flag fflags set to 0x8. The This deviation indicates non-compliance with the RISC-V expected behavior can be reproduced with the code in listing standard, which treats such operations as illegal. This finding 8. suggests a potential flaw in the core’s handling of rounding On the CVA6 core, executing the PoC code results in ft3 modes, which could have further implications for software being set to 0xffffffffff800000 and fflags containing that relies on correct floating-point rounding behavior. 0x8, indicating a divide by zero exception. However, based on While this issue had been identified manually in earlier ver- RISC-V ISA specification [49, 50], if we ran the same code, sions of the BOOM core, our automated testing—conducted the ft3 register should contain the canonical NaN value, i.e., without any previous knowledge of this bug—revealed that 0xffffffff7fc00000, and fflags should be 0x0, which the problem persists in the current version. This emphasizes indicating no exceptions were raised. As discussed in ap- the importance of thorough automated testing methodologies pendix B, this incorrect behavior in the CVA6 core could to uncover issues that may have been overlooked in manual result in significant problems for operations that depend on inspections.

USENIX Association 34th USENIX Security Symposium 1805